What Makes a Good Penetration Test

Share on linkedin
Share on facebook
Share on twitter

Table of Contents

There’s a considerable difference between a “check the box” penetration test and a full-blown professional penetration test. In order to get the best out of your investment, you want to contract a penetration testing company that will leverage high standards and quality that helps your business both meet compliance standards and implement measures that protect you from cyberattacks. A pentest that only aims to “check the box” will provide you with a false sense of security and can lead you to spend resources inefficiently.

As a stakeholder, however, you may not always be aware of the elements that makes a good penetration test. Here are some of the elements you should look out for in a penetration test:

1. The provider defines the scope properly

Before you perform a pentest, it is crucial that you clearly define your scope with your provider to ensure that all your bases are covered and that you identify your most prominent risks. For example, if you’re performing an internal network pentest, it is important for the pentesting company to define how many servers / workstations will be targeted and if any cloud-based service are used (AWS, Azure, VMs, etc..). In other contexts, if you’re conducting a web application pentest, you want to make sure that your most critical features and API connectors (if any are present) are in scope of the test. Any of these details should be explicitly mentioned to your penetration testing company during the scoping phase to make sure that they tested and that components requiring further attention are identified ahead of the test. If your provider attempts to get your project started ASAP with a vague and undefined scope, you should be wary of many other criteria brought up in this article before moving forward.

Why is it important? Defining your scope rigorously not only helps you hold your provider accountable, but also ensures that you are leaving no stones unturned. Failing to define the scope properly might leave some critical assets aside which should have been tested, leaving a gap open in your risk management strategy and creating a false sense of security.

A professional pentesting supplier will help you define the scope properly and will provide a list of requirements to scope each type of penetration test your organization needs. The way your provider scopes your test is a good indicator of the level of quality you can expect from them, but there are many other things to consider as well.

2. The specialists are certified and the company is accredited

Why are they important ? – When you choose a pentesting company, you should look for which certifications their team holds and look for well-known and recognized penetration testing certifications. These credentials, such as CISSP, GIAC, CEH, or OSCP, provide a great benchmark of the experience and expertise that will be leveraged during the test. Most of them require extensive knowledge of the techniques used by hackers as well as the latest technologies to be obtained. They allow the specialist to refine their skills and to succeed various exploitation scenarios used in real-world situations on systems that reflect the ones used by organizations today. Some advanced certifications, such as OSCP, can only be acquired after passing a series of challenges and an assessment that lasts 24h consecutively.

In addition to these certifications held by the specialist, you should also take the company’s accreditations into consideration during your selection process. For example, providers who are ISO9001 compliant demonstrate a will to continuously improve and a commitment to the quality of their deliverables. This way, you can be certain that each of their penetration test follows a systematic and documented approach.

Accredited companies with certified testers, given that their credentials are valid and recognized by the industry, will deliver a penetration test you can rely on to implement measures and make decisions that secure your company. Along with these certifications, the approach and methodologies used are also a great indicator of a good penetration test, as discussed in the following point.

3. The test is more manual than automated

As discussed in a previous article that compares automated vs manual penetration testing, these methodologies have their fair share of pros and cons, but there is a clear winner when it comes to their return on investment and the depth of their assessment.

Although automated pentesting is cheaper and requires a lesser degree of expertise, it is important to note that it can still be helpful for your IT team to reveal common vulnerabilities associated with the technologies and versions currently used. It should not, however, be your only way to assess your cybersecurity risks for many reasons that follow. In simple terms, a fully automated test only check your company’s doors to see if they are locked and in some cases it might say that a critical access is not locked and requires immediate attention, although a security guard is sitting right behind it or without considering that it leads to a public bathroom. A penetration test on the other hand, will not only verify that every door is locked, but it will attempt to bypass locked doors and enter unlocked doors to see what can be done once opened to help accurately prioritize each risks faced by the company.

Why is it important? – Fully automated testing do not contextualize each vulnerability and is unable to determine their risk with the same degree of precision that a full blown penetration test will. They often generate false positives which could lead to a waste of resources if the analysis is considered as fully reliable and if its results are not validated by an experienced specialist. They can also create a false sense of security for companies relying solely on this technique.

A good penetration test will use a combination of both techniques, with a large portion focusing on manual techniques to identify and safely exploit vulnerabilities. They still use automated tools to optimize their time because they might otherwise spend an unreasonable amount of time on the reconnaissance phase, but the majority of the test follows a manual methodology.

4. It leverages recognized standards and methodologies

Pentesters can use a variety of frameworks to achieve their goals, whether it’s OWASP, OSSTMM, CVE, CAPEC, CWE, CVSS, etc. A good way to determine if you’re paying for a good penetration test is by asking the methodologies they use. You should look for a company that base their tests on recognized penetration testing methodologies and see if they are aligned with the type of penetration test you’re looking for. If the company can’t provide the methodologies used or if they do not leverage industry-recognized standards, then you should look out for other elements that helps determine the quality of their penetration tests before you move forward with the provider.

5. The final report is clear and contains relevant information

The quality of the final report is one of the most, if not the single most important component that determines whether you’re looking at a good penetration test or not. Based on the results provided in this report, your stakeholders will make decisions and plan cybersecurity investments in order to secure your company’s assets. You should look for these 5 items in your penetration testing report to make sure that it delivers recommendations that your management can rely on to successfully implement strong cybersecurity measures and to fix your vulnerabilities. The report will directly reflect the quality of your provider’s pentest.

6. Retests are included

Along with the 5 elements covered previously, you want to make sure that this penetration test supports you throughout the application of the recommendations it provided. A good penetration test will offer retests at no additional charges, allowing you to confirm that your vulnerabilities have been fixed successfully and that no additional vulnerabilities were introduced while they were fixed. After you have closed those security gaps, you should receive an attestation or an official statement, showing that a proper pentest was done and that any vulnerabilities found during the process were corrected. This statement will let your clients know that you’re safe to do business with: that you’ve taken the necessary steps to protect them and yourself, which ultimately makes you a better business partner or provider. Without retesting, on the other hand, you might not know for sure that you’ve actually fixed those vulnerabilities–and you don’t have anything to prove it other than a pentest report that discloses your security issues. While your IT team can effectively fix and validate their corrective measures on their own, having a third-party’s validation is always much more valuable, especially in a context of compliance with standards such as PCI-DSS or security questionnaires from your business partners.


Not all pentesting teams are created equal and offer the same level of expertise. Recently, there has been a major shift in the skills desired by companies seeking pentesting services. The skills once regarded as largely unimportant have now become critical for ensuring the safety of an organization. Always be diligent when shopping for your annual penetration test, especially for critical systems.


A penetration test is a simulated hacking attempt that identifies opportunities for real hackers to break through your defences and perform various malicious acts. It generally leverages tools used by hackers and various professional methodologies to replicate the steps that modern hackers would take to intrude into your IT systems.

A pentest attempts to exploit your vulnerabilities to determine their potential impact, should they be used in a real hacking scenario. They provide a list of vulnerabilities with their respective level of severity, as well as technical recommendations to help your team apply corrective measures and focus on the most critical vulnerabilities.

These services allow your organization to answer the following questions, among several others:

  • Can a hacker gain access to any sensitive information?
  • Can a hacker hijack my technologies for any malicious acts?
  • Could a malware infection spread through the network?
  • Can an attacker escalate access to an administrative user?

Learn more about penetration testing →

There are many contexts in which a penetration test should be performed.

Here are some common use cases for a pentest:

  • As part of the development cycle of an application. (To test the security of a new feature/app)
  • To comply with security requirements. (3rd-parties, PCI, ISO27001, etc.)
  • To secure sensitive data from exfiltration.
  • To prevent infections by malware. (Ransomware, spyware, etc.)
  • To prevent disruptive cyberattacks. (Such as denial of service)
  • As part of a cybersecurity risk management strategy.

All businesses are advised to conduct a penetration test at least once a year, as well as after any significant upgrades or modifications to the company network. Given the rapid rate at which new exploits are discovered, we generally recommend that quarterly tests are performed.

Various steps are taken over the course of the project to prevent the potential impact of our tests on the stability of your technological environment and the continuity of your business operations.

For this reason, a communication plan will be put in place at the beginning of the project to prevent and mitigate any potential impact. A representative of your organization will be identified to act as the main point of contact to ensure rapid communication in the event of a situation directly impacting the conduct of your daily operations, or if any critical vulnerabilities are identified, for which  corrective measures need to be implemented quickly.

While we use a simple 4 levels risk rating approach (Critical, High, Moderate, Low), our risk assessment is actually based on the Common Vulnerability Scoring System (CVSS) standard. Two main criteria are considered when  assessing the risk level of each vulnerability:

  • Potential impact: The potential impact of an attack based on a vulnerability, combined with its  potential effect on the availability of the system, as well as the confidentiality and integrity of  the data.
  • Exploitability: The potential exploitability of a vulnerability; a vulnerability that is easier to  exploit increases the number of potential attackers and thus the likelihood of an attack.  Different factors are considered when evaluating the exploitability potential of a vulnerability  (e.g.: access vector, authentication, operational complexity, etc.)

Related Vumetric Blog Posts

Cyberattack impact

How Cyberattacks Impact Your Organization

A cyberattack is a malicious assault by cybercriminals aiming to damage a computer network or …

Read The Article
penetration test vs bug bounty

Penetration Testing vs Bug Bounty

Due to the recent spate of ransomware incidents, organizations and nervous IT administrators are wondering …

Read The Article
How Wordpress Gets Hacked and How to Prevent

How WordPress Sites Get Hacked And Fixes to Prevent it

WordPress sites get hacked on a regular basis, as it is by far the most …

Read The Article


We've Earned Internationally-Recognized Certifications

Contact a Certified Expert

Talk with a real expert. No engagement. We answer within 24h.
penetration testing provider

Stay Updated on Cyber Risks!

Subscribe to the Vumetric Monthly Bulletin to keep up with breaking news in the cybersecurity industry.

Looking For A High-Quality Penetration Test?

or give us a call directly at: