Why Automated Penetration Testing For Applications is Insufficient

Table of Contents

With the ever-growing amount of applications provided to consumers, the prospect of performing application penetration tests on each application with a limited budget becomes increasingly daunting and seemingly impossible for most organizations. With that said, application risks will never be sufficiently mitigated by relying solely on automated scanning alone due to the custom nature of modern applications. This can lead to hard decisions when it comes to establishing a cost-effective strategy for application security.

This article will go over the main challenges that comes with security testing for web applications, the various shortcomings of automated scans as well as potential solutions for organizations with limited resources still looking to secure their mobile or web applications from cyberattacks.

Which technique is the best suited for your needs?

Every professional in the industry agrees that applications cannot be sufficiently secured by relying on automated scans alone. This goes without saying that automated scanning itself is not an issue, but rather the context in which it is often performed. It can be a great starting point for those who lack a sufficient budget to undergo frequent security testing for all of their applications, but should not be your only resort. Typically, there are three main approaches used to test the security of an application, and they vary in depth, accuracy, and cost. These techniques should be used at a different frequency due to the different results they yield and should be combined for a bulletproof risk management strategy.

  1. Automated Scans
  2. Automated Scans + Manual Validation
  3. Application Penetration Tests (Automated Scanning + Manual Validation + Manual Testing)

To determine which option is right for each app, it is common to take a risk-based approach to prioritize applications. This risk-based approach is used to influence the type of assessment that each application requires and on which basis, according to a variety of factors such as: how sensitive the data being handled is, what kind of changes were recently made to the app, the number of third-party integrations, when the last in-depth assessment was performed, etc. Depending on these factors, your stakeholders may decide to opt for a fully automated scan or an in-depth manual test.

Although a risk-based approach is an effective way to determine which type of assessment to use for an application, it often leads to the conclusion that automated scanning alone is acceptable for some apps, when in fact, this is rarely the case. The wide majority of application still require a regular manual penetration test, though its frequency can be adjusted based on a series of factors brought up previously. This next point will go over some of the reasons why it should not be your only resort to test your application.

Relying 100% on automated scans exposes companies to cyberattacks

While automated scans play an important role in an efficient cybersecurity strategy, it should not be your only option to assess your application’s cybersecurity.  The simple reason being that most applications contain too many proprietary elements and unique business logic that is impossible for an automated scanner to accurately contextualize. The validations performed by a scanner cannot accurately and systematically identify most critical to high severity vulnerabilities present in modern applications, such as Authentication Bypasses or Access Control Weaknesses. These vulnerabilities can have serious consequences when exploited by hackers and require hands-on testing in order to be identified, as they are often tied to the unique properties and actions found in your application.

Automated testing often leads to a false sense of security for organizations who rely 100% on them, as they tend to assume their application has been successfully secured, when in fact many critical vulnerabilities were left aside. As shown in the examples below, automated testing, even with manual validation, can expose organizations to cyberattacks as they will feel that their application no longer requires an assessment. The findings highlighted in green represent vulnerabilities that would not have been addressed if the organization had relied solely on automated scans, leaving them open to various scenarios of attack.

Testing Scenario #1: Public-facing app

Risk Level Vulnerability Methodology
Critical Authorization Bypass through User-Controlled Key Penetration Testing
High Weak access control – Page accessible without authentication Penetration Testing
High Server-side Request Forgery Automated Scanning
High Encryption Not Enforced Automated Scanning
Medium Improper Access Control – Forceful Browsing Penetration Testing
Medium Insufficient Session Expiration Penetration Testing
Medium Client-Side Security Control Without Server-side Enforcement Penetration Testing
Medium Missing HTTP-only attribute in session cookie Automated Scanning
Medium Vulnerable software components Automated Scanning
Low Disclosure of Private IP and Hostnames Automated Scanning
Low Information Disclosure – <Redacted> Endpoints Automated Scanning

In this application, an automated scan would have missed 5 significant vulnerabilities that could have allowed an attacker to access sensitive customer data and in some instances, escalate privileges to access administrative features.

Testing Scenario #2: E-commerce app

Risk Level Vulnerability Methodology
Critical Encryption Not Enforced Automated Scanning
Critical Privilege Escalation by Authentication Manipulation Penetration Testing
Critical Unverified Password Change Leads to Privilege Escalation Penetration Testing
Critical Authorization Bypass through User-Controlled Key Penetration Testing
Medium Missing Secure Attribute in Session Cookie Automated Scanning

In this application, an automated assessment would have missed various ways in which a user could access another user’s data and permissions, leaving the company exposed to a potential incident.

When to prioritize manual penetration testing

These two case studies demonstrate that, while automated scans can help identify various vulnerabilities that do need to be corrected, it is far from sufficient to successfully secure a critical application. Without manual tests, the organizations would have left many stones unturned, exposing them to potentially damaging vulnerabilities. Rather than wonder “Which of my applications should be tested manually?”, you should wonder “How frequently does each application need a manual penetration test?”.

With today’s ever-evolving cyber threats, it is essential that all applications be manually tested by experienced and certified Penetration Testers at least once a year, allowing you to stay on top of the latest hacking tools and techniques used to circumvent your application’s security measures. This is why many standards, such as PCI-DSS, mandate a yearly pentest in order to stay compliant.

On another note, it’s imperative that your applications are manually tested when new features are introduced or major changes are done to its network infrastructure, as some new vulnerabilities may have been introduced that leaves you vulnerable to a potential incident.

When to use automated testing solutions for your app

If your application has gone through a manual penetration test recently, it’s safe to say that you’ve covered all your bases. However, technologies and frameworks on which your applications are built are constantly faced with new vulnerabilities, sometimes critical. Your organization might not afford to undergo hands-on penetration tests every month to consistently identify new security flaws, but it’s still essential that your team fixes these vulnerabilities on an ongoing basis. This is where scans come in and fill the gaps.

Automated scans are well suited to identify certain types of application vulnerabilities found in the OWASP top 10, including Cross-site Scripting, SQL Injection, and Server-Side Request Forgery. Automated scans are also efficient at identifying particular misconfigurations, including incorrectly implemented TLS or the absence of recommended security-focused HTTP headers and cookie attributes. Their validations are done by leveraging large databases of vulnerabilities associated with each of your technologies in place, as well as specific issues related to the version of software you have currently installed. They can be performed on a recurring basis and do not require a very specialized expertise, making them a cost-effective solution to keep your application secure until your next full-blown penetration test can be performed.

Automated scanning with manual validations

The main shortcoming of automated scanning resides in the large amount of false positives it generates. It can lead to an inefficient use of your resources since your team might spend time fixing vulnerabilities that represent little to no risk for your company. A manual validation of automated scans performed by a certified professional allows you to adjust risk ratings to your context and prioritize your next steps, helping you save resources in the long run. Though it’s important to note that manual validation does not improve the depth of the analysis, which does not make them as reliable as manual testing to secure critical applications.

Subscribe to Our Newsletter!
Stay on top of cybersecurity risks, evolving threats and industry news.
This field is for validation purposes and should be left unchanged.

Share this article on social media:

Recent Blog Posts

Featured Services


The Latest Blog Articles From Vumetric

From industry trends,  to recommended best practices, read it here first:


Enter your Email Address

This field is for validation purposes and should be left unchanged.

* No free email provider (e.g: gmail.com, hotmail.com, etc.)


Penetration Testing Buyer's Guide

Everything You Need to Know

Gain full confidence in your future cybersecurity assessments by learning to plan, scope and execute projects.
This site is registered on wpml.org as a development site. Switch to a production site key to remove this banner.