Why Automated Penetration Testing For Applications is Insufficient

Table of Contents

With the ever-growing amount of applications provided to consumers, the prospect of performing application penetration tests on each application with a limited budget becomes increasingly daunting and seemingly impossible for most organizations. With that said, application risks will never be sufficiently mitigated by relying solely on automated scanning alone due to the custom nature of modern applications. This can lead to hard decisions when it comes to establishing a cost-effective strategy for application security.

This article will go over the main challenges that comes with security testing for web applications, the various shortcomings of automated scans as well as potential solutions for organizations with limited resources still looking to secure their mobile or web applications from cyberattacks.

Which technique is the best suited for your needs?

Every professional in the industry agrees that applications cannot be sufficiently secured by relying on automated scans alone. This goes without saying that automated scanning itself is not an issue, but rather the context in which it is often performed. It can be a great starting point for those who lack a sufficient budget to undergo frequent security testing for all of their applications, but should not be your only resort. Typically, there are three main approaches used to test the security of an application, and they vary in depth, accuracy, and cost. These techniques should be used at a different frequency due to the different results they yield and should be combined for a bulletproof risk management strategy.

  1. Automated Scans
  2. Automated Scans + Manual Validation
  3. Application Penetration Tests (Automated Scanning + Manual Validation + Manual Testing)

To determine which option is right for each app, it is common to take a risk-based approach to prioritize applications. This risk-based approach is used to influence the type of assessment that each application requires and on which basis, according to a variety of factors such as: how sensitive the data being handled is, what kind of changes were recently made to the app, the number of third-party integrations, when the last in-depth assessment was performed, etc. Depending on these factors, your stakeholders may decide to opt for a fully automated scan or an in-depth manual test.

Although a risk-based approach is an effective way to determine which type of assessment to use for an application, it often leads to the conclusion that automated scanning alone is acceptable for some apps, when in fact, this is rarely the case. The wide majority of application still require a regular manual penetration test, though its frequency can be adjusted based on a series of factors brought up previously. This next point will go over some of the reasons why it should not be your only resort to test your application.

Relying 100% on automated scans exposes companies to cyberattacks

While automated scans play an important role in an efficient cybersecurity strategy, it should not be your only option to assess your application’s cybersecurity.  The simple reason being that most applications contain too many proprietary elements and unique business logic that is impossible for an automated scanner to accurately contextualize. The validations performed by a scanner cannot accurately and systematically identify most critical to high severity vulnerabilities present in modern applications, such as Authentication Bypasses or Access Control Weaknesses. These vulnerabilities can have serious consequences when exploited by hackers and require hands-on testing in order to be identified, as they are often tied to the unique properties and actions found in your application.

Automated testing often leads to a false sense of security for organizations who rely 100% on them, as they tend to assume their application has been successfully secured, when in fact many critical vulnerabilities were left aside. As shown in the examples below, automated testing, even with manual validation, can expose organizations to cyberattacks as they will feel that their application no longer requires an assessment. The findings highlighted in green represent vulnerabilities that would not have been addressed if the organization had relied solely on automated scans, leaving them open to various scenarios of attack.

Testing Scenario #1: Public-facing app

Risk Level



Critical Authorization Bypass through User-Controlled Key Penetration Testing
High Weak access control – Page accessible without authentication Penetration Testing
High Server-side Request Forgery Automated Scanning
High Encryption Not Enforced Automated Scanning
Medium Improper Access Control – Forceful Browsing Penetration Testing
Medium Insufficient Session Expiration Penetration Testing
Medium Client-Side Security Control Without Server-side Enforcement Penetration Testing
Medium Missing HTTP-only attribute in session cookie Automated Scanning
Medium Vulnerable software components Automated Scanning
Low Disclosure of Private IP and Hostnames Automated Scanning
Low Information Disclosure – <Redacted> Endpoints Automated Scanning

In this application, an automated scan would have missed 5 significant vulnerabilities that could have allowed an attacker to access sensitive customer data and in some instances, escalate privileges to access administrative features.

Testing Scenario #2: E-commerce app

Risk Level



Critical Encryption Not Enforced Automated Scanning
Critical Privilege Escalation by Authentication Manipulation Penetration Testing
Critical Unverified Password Change Leads to Privilege Escalation Penetration Testing
Critical Authorization Bypass through User-Controlled Key Penetration Testing
Medium Missing Secure Attribute in Session Cookie Automated Scanning

In this application, an automated assessment would have missed various ways in which a user could access another user’s data and permissions, leaving the company exposed to a potential incident.

Want to know why 1,000+ organizations rely on Vumetric's manual penetration testing expertise?

No matter the complexity of your application, our experts are renowned to identify and help you fix the most intricate application vulnerabilities that could lead to a disastrous cyberattack.

When to prioritize manual penetration testing

These two case studies demonstrate that, while automated scans can help identify various vulnerabilities that do need to be corrected, it is far from sufficient to successfully secure a critical application. Without manual tests, the organizations would have left many stones unturned, exposing them to potentially damaging vulnerabilities. Rather than wonder “Which of my applications should be tested manually?”, you should wonder “How frequently does each application need a manual penetration test?”.

With today’s ever-evolving cyber threats, it is essential that all applications be manually tested by experienced and certified Penetration Testers at least once a year, allowing you to stay on top of the latest hacking tools and techniques used to circumvent your application’s security measures. This is why many standards, such as PCI-DSS, mandate a yearly pentest in order to stay compliant.

On another note, it’s imperative that your applications are manually tested when new features are introduced or major changes are done to its network infrastructure, as some new vulnerabilities may have been introduced that leaves you vulnerable to a potential incident.

When to use automated testing solutions for your app

If your application has gone through a manual penetration test recently, it’s safe to say that you’ve covered all your bases. However, technologies and frameworks on which your applications are built are constantly faced with new vulnerabilities, sometimes critical. Your organization might not afford to undergo hands-on penetration tests every month to consistently identify new security flaws, but it’s still essential that your team fixes these vulnerabilities on an ongoing basis. This is where scans come in and fill the gaps.

Automated scans are well suited to identify certain types of application vulnerabilities found in the OWASP top 10, including Cross-site Scripting, SQL Injection, and Server-Side Request Forgery. Automated scans are also efficient at identifying particular misconfigurations, including incorrectly implemented TLS or the absence of recommended security-focused HTTP headers and cookie attributes. Their validations are done by leveraging large databases of vulnerabilities associated with each of your technologies in place, as well as specific issues related to the version of software you have currently installed. They can be performed on a recurring basis and do not require a very specialized expertise, making them a cost-effective solution to keep your application secure until your next full-blown penetration test can be performed.

Automated scanning with manual validations

The main shortcoming of automated scanning resides in the large amount of false positives it generates. It can lead to an inefficient use of your resources since your team might spend time fixing vulnerabilities that represent little to no risk for your company. A manual validation of automated scans performed by a certified professional allows you to adjust risk ratings to your context and prioritize your next steps, helping you save resources in the long run. Though it’s important to note that manual validation does not improve the depth of the analysis, which does not make them as reliable as manual testing to secure critical applications.

A penetration test is a simulated hacking attempt that identifies opportunities for real hackers to break through your defences and perform various malicious acts. It generally leverages tools used by hackers and various professional methodologies to replicate the steps that modern hackers would take to intrude into your IT systems.

A pentest attempts to exploit your vulnerabilities to determine their potential impact, should they be used in a real hacking scenario. They provide a list of vulnerabilities with their respective level of severity, as well as technical recommendations to help your team apply corrective measures and focus on the most critical vulnerabilities.

These services allow your organization to answer the following questions, among several others:

  • Can a hacker gain access to any sensitive information?
  • Can a hacker hijack my technologies for any malicious acts?
  • Could a malware infection spread through the network?
  • Can an attacker escalate access to an administrative user?

Learn more about penetration testing →

There are many contexts in which a penetration test should be performed.

Here are some common use cases for a pentest:

  • As part of the development cycle of an application. (To test the security of a new feature/app)
  • To comply with security requirements. (3rd-parties, PCI, ISO27001, etc.)
  • To secure sensitive data from exfiltration.
  • To prevent infections by malware. (Ransomware, spyware, etc.)
  • To prevent disruptive cyberattacks. (Such as denial of service)
  • As part of a cybersecurity risk management strategy.

All businesses are advised to conduct a penetration test at least once a year, as well as after any significant upgrades or modifications to the company network. Given the rapid rate at which new exploits are discovered, we generally recommend that quarterly tests are performed.

Various steps are taken over the course of the project to prevent the potential impact of our tests on the stability of your technological environment and the continuity of your business operations.

For this reason, a communication plan will be put in place at the beginning of the project to prevent and mitigate any potential impact. A representative of your organization will be identified to act as the main point of contact to ensure rapid communication in the event of a situation directly impacting the conduct of your daily operations, or if any critical vulnerabilities are identified, for which  corrective measures need to be implemented quickly.

While we use a simple 4 levels risk rating approach (Critical, High, Moderate, Low), our risk assessment is actually based on the Common Vulnerability Scoring System (CVSS) standard. Two main criteria are considered when  assessing the risk level of each vulnerability:

  • Potential impact: The potential impact of an attack based on a vulnerability, combined with its  potential effect on the availability of the system, as well as the confidentiality and integrity of  the data.
  • Exploitability: The potential exploitability of a vulnerability; a vulnerability that is easier to  exploit increases the number of potential attackers and thus the likelihood of an attack.  Different factors are considered when evaluating the exploitability potential of a vulnerability  (e.g.: access vector, authentication, operational complexity, etc.)

Related Blog Articles

What Are the Security Risks Associated with Public Wi-Fi?

What Are the Security Risks Associated with Public Wi-Fi?

Wireless networks are convenient, but can also be dangerous, as they can leave your device vulnerable to hackers. This article reviews the primary risks to prepare against.

Read The Article
8 Tips to Secure Your E-Commerce Website

8 Tips to Secure Your E-Commerce Website

Over the past two years, the rise of e-commerce as a priority channel for consumer-facing businesses has only accelerated. Therefore, the need to provide your consumers with a secure e-commerce website has never been so critical to your success.

Read The Article
What is OWASP and Why Does it Matter?

What is OWASP and Why Does it Matter?

OWASP is an international organization that focuses on improving software security. OWASP develops and maintains a variety of tools, checklists, and guides related to web application security.

Read The Article

Tell us about your needs.
Get an answer the same business day.

Got an urgent request? Call us at 1-877-805-7475 or Book a meeting.

What happens next:

  • We reach out to learn about your objectives
  • We work together to define your project's scope
  • You get an all-inclusive, no engagement proposal

No engagement. We answer within 24h.
This site is registered on wpml.org as a development site.