Why Manual Penetration Testing Is Still Needed

When you have multiple testing tools available to you to detect software security vulnerabilities, it is tempting to forgo expensive and time-consuming manual penetration tests. While scanners (both ones we use and ones we don't) help reduce the need for manual penetration tests, they don't eliminate them. To see why, let's debunk three misconceptions about scanning (especially DAST/Active scanning) and how it relates to penetration testing.

A DAST scan isn't a penetration test

First, it's important to mention that even though some unethical or incompetent security professionals try to pass off a DAST scan as a full penetration test (I'm the only one who has seen this, read this and scroll down to the paragraph that starts "special note on ethics"), a true penetration test starts, not ends, with an automated scan. A true penetration test involves a human looking at your website and exploring items that look odd or potentially exploitable. Manual tests tell you two things that an automated test (currently) cannot:

1. Are there any vulnerabilities that require multiple steps to exploit?
2. What is the potential impact of a particular vulnerability? Can I use it to attack the server? Others? Or is it something that I could use to "attack" myself?

Scanners don't come close to catching everything

Unfortunately, I don't have research that I'm able to publish yet, but I have done apples-to-apples comparisons with many scanners and have found that even the best scanners don't catch nearly all of the potential security issues with your website. I have to believe that sites like this one must be using severely over-simplified versions of vulnerabilities, since I've never had a scanner find much more than 50% of vulnerabilities in my tests. I recommend using multiple scanners to help get around this issue, but performing a manual penetration test can still provide more information.

Scanners aren't smart enough to look for application-specific problems

Scanners are built to look for generic issues like SQL injection, Cross-Site Scripting (XSS), and the like, largely because scanning for these types of issues is relatively straightforward. If you put <script>alert(1)</script> into a textbox, submit the form, and see an alert with the text "1", you're reasonably certain that a XSS vulnerability exists. But what if:

• The scanner is able to find items by changing the query string? Is that a vulnerability or is that intended behavior?
• You have a vulnerability in an uncommon scenario, such as incorrectly implementing functionality in a little-known JavaScript framework?
• An attacker can indirectly discern information by application behavior, such as pulling usernames from your system based on the amount of time it takes to process your login?

Automated scanners are great at finding easy items, but a human is still needed for the difficult ones.