Why Automated App Pentests Are Not Enough | Vumetric

Why Automated App Pentests Are Not Enough

Share on linkedin
Share on facebook
Share on twitter

Table of Contents

With the ever-growing amount of applications provided to customers, the prospect of performing Application Penetration Testing on each application, with limited budgets and scarce resources, becomes increasingly daunting and seemingly impossible for most organizations. Application risks will never be sufficiently mitigated by relying on automated scanning alone, due to the custom nature of their implementation.

The volume and complexity of cyberattacks continue to increase at an alarming rate and are predicted to continue in the following years. To ensure sufficient protection, organizations must be able to answer these questions:

  • Do we have vulnerabilities that an attacker could find?
  • If an attacker found them, could they be exploited?
  • If exploited, what damage could they do to our business?
  • What should be done to fix the vulnerabilities?

Are Automated Scans Good Enough?

Application risks cannot be sufficiently mitigated by relying on automated scans alone. Typically, there are three main dynamic options practiced today, and they vary in depth, accuracy, and cost.

  1. Automated Scans
  2. Automated Scans with Manual Validation
  3. Application Penetration Tests (Automated Scanning with Manual Validation, combined with Manual Testing)

To determine which option is right for each of your applications, it is common to take a risk-based approach to prioritize applications. This risk-based approach is used to influence the type of assessment that each application requires.

Although risk-based classification is an effective way to prioritize limited resources, it leads to the conclusion that automated scanning alone is acceptable for some applications, when in fact, this is rarely the case. The wide majority of application still require a regular manual penetration test, especially following major changes.

Automated Scans Do Not Identify Critical Vulnerabilities

Even with manual validation, automated scans consistently miss high and critical vulnerabilities that can leave organizations exposed.

Automated scans are well suited to identify certain types of application vulnerabilities found in the OWASP top 10, including Cross-site Scripting, SQL Injection, and Server-Side Request Forgery. Automated scans are also efficient at identifying particular misconfigurations, including incorrectly implemented TLS or the absence of recommended security-focused HTTP headers and cookie attributes.

However, automated scans fail to identify complex vulnerabilities that can have a critical impact on your application, such as Authentication Bypasses, many types of Access Control Weaknesses, and flaws in business logic. Also, automated scans contain a large number of false-positives, and use generic risk ratings that can lead to a waste of effort and resources on measures that will not secure your application adequately.

While a manual validation of automated scan results by certified professionals removes false-positives and adjusts risk ratings to an organization’s context, manual validation does not improve the depth of the analysis, which does not make them as reliable as manual testing to secure critical applications.

Automated Scans Leave Companies Exposed to Cyberattacks

As shown in the examples below, relying on automated scans with manual result validation can leave organizations with a false sense of security and expose them to cyberattacks, as they will feel that their application has been properly secured and no longer requires an assessment. The findings highlighted in green represent vulnerabilities that would not have been addressed if the organization had relied solely on Automated Scans, leaving them open to various scenarios of attack.

Testing Scenario #1: Public-facing app

Risk Level



Critical Authorization Bypass through User-Controlled Key Penetration Testing
High Weak access control – Page accessible without authentication Penetration Testing
High Server-side Request Forgery Automated Scanning
High Encryption Not Enforced Automated Scanning
Medium Improper Access Control – Forceful Browsing Penetration Testing
Medium Insufficient Session Expiration Penetration Testing
Medium Client-Side Security Control Without Server-side Enforcement Penetration Testing
Medium Missing HTTP-only attribute in session cookie Automated Scanning
Medium Vulnerable software components Automated Scanning
Low Disclosure of Private IP and Hostnames Automated Scanning
Low Information Disclosure – <Redacted> Endpoints Automated Scanning

In this application, an automated scan would have missed 5 significant vulnerabilities, including a critical vulnerability that could have allowed an attacker to gain access to sensitive customer data.

Testing Scenario #2: E-commerce app

Risk Level



Critical Encryption Not Enforced Automated Scanning
Critical Privilege Escalation by Authentication Manipulation Penetration Testing
Critical Unverified Password Change Leads to Privilege Escalation Penetration Testing
Critical Authorization Bypass through User-Controlled Key Penetration Testing
Medium Missing Secure Attribute in Session Cookie Automated Scanning

In this application, an automated assessment would have missed various ways in which a user could access another user’s data and permissions, leaving the company exposed to a potential incident.

Perform Manual Tests Frequently And Adjust Based on Risks

These examples demonstrate that, while automated scans can help identify various vulnerabilities that do need to be corrected, it is critical to be aware that it is far from sufficient to successfully secure a critical application. Without manual tests, the organizations would have left many stones unturned, exposing them to potentially dangerous vulnerabilities.

With today’s ever-evolving cyber threats, it is essential that all applications be manually tested by experienced and certified Penetration Testers. If they aren’t, many sophisticated and high-risk vulnerabilities will be left aside. If these vulnerabilities are overlooked, it will leave sensitive data and corporate systems at risk.

Rather than wonder “Which of my applications should be tested manually?”, you should wonder “How frequently does each application need a manual penetration test?”


To learn more about the differences between manual and automated testing or to get started, get in touch with one of our experts.


Need a Manual Web App Pentest?

or give us a call directly at:

A penetration test is a simulated hacking attempt that identifies opportunities for real hackers to break through your defences and perform various malicious acts. It generally leverages tools used by hackers and various professional methodologies to replicate the steps that modern hackers would take to intrude into your IT systems.

A pentest attempts to exploit your vulnerabilities to determine their potential impact, should they be used in a real hacking scenario. They provide a list of vulnerabilities with their respective level of severity, as well as technical recommendations to help your team apply corrective measures and focus on the most critical vulnerabilities.

These services allow your organization to answer the following questions, among several others:

  • Can a hacker gain access to any sensitive information?
  • Can a hacker hijack my technologies for any malicious acts?
  • Could a malware infection spread through the network?
  • Can an attacker escalate access to an administrative user?

Learn more about penetration testing →

There are many contexts in which a penetration test should be performed.

Here are some common use cases for a pentest:
  • As part of the development cycle of an application. (To test the security of a new feature/app)
  • To comply with security requirements. (3rd-parties, PCI, ISO27001, etc.)
  • To secure sensitive data from exfiltration.
  • To prevent infections by malware. (Ransomware, spyware, etc.)
  • To prevent disruptive cyberattacks. (Such as denial of service)
  • As part of a cybersecurity risk management strategy.
All businesses are advised to conduct a penetration test at least once a year, as well as after any significant upgrades or modifications to the company network. Given the rapid rate at which new exploits are discovered, we generally recommend that quarterly tests are performed.

The time required to successfully execute a penetration test depends on the scope and type of test. Most penetration tests can be performed within a couple of days, but some can span over several weeks, sometimes even months depending on the complexity of the project.

Various steps are taken over the course of the project to prevent the potential impact of our tests on the stability of your technological environment and the continuity of your business operations.

For this reason, a communication plan will be put in place at the beginning of the project to prevent and mitigate any potential impact. A representative of your organization will be identified to act as the main point of contact to ensure rapid communication in the event of a situation directly impacting the conduct of your daily operations, or if any critical vulnerabilities are identified, for which  corrective measures need to be implemented quickly.

While we use a simple 4 levels risk rating approach (Critical, High, Moderate, Low), our risk assessment is actually based on the Common Vulnerability Scoring System (CVSS) standard. Two main criteria are considered when  assessing the risk level of each vulnerability:

  • Potential impact: The potential impact of an attack based on a vulnerability, combined with its  potential effect on the availability of the system, as well as the confidentiality and integrity of  the data.
  • Exploitability: The potential exploitability of a vulnerability; a vulnerability that is easier to  exploit increases the number of potential attackers and thus the likelihood of an attack.  Different factors are considered when evaluating the exploitability potential of a vulnerability  (e.g.: access vector, authentication, operational complexity, etc.)

Need to Identify Your Cybersecurity Risks?

A specialist will reach out to:

Mailbox Icon
stay informed!
Subscribe to stay on top of the latest trends, threats, news and statistics in the cybersecurity industry.