Use Pen Testing to Gauge Software Development Life Cycle Health

The health of your software development life cycle (SDLC) is an important indicator of your organization’s quality assurance, cost efficiency, customer satisfaction and compliance.

Even though the executive order (EO) issued in May 2021 to improve the nation’s cybersecurity only required Software Bill of Materials (SBOMs) for government agencies, SBOMs are now widely promoted and used as an industry best practice because of the EO.

The EO has also caused many organizations to look at their software development processes again and take steps to make their software supply lines more secure and reliable. It also shows that organizations need to pay more attention to their development and health practices because SBOM creation can show the world their “dirty laundry.”

This is why you should stop using pen tests only to find problems in your software and start using them to find problems in your processes.

While developers have long used third-party web app and API pen tests to find application security defects, pen tests are also a great way to gauge the health of an SDLC. Third-party automatic pen tests, which are often required by regulatory agencies, do a good job of finding security flaws that need to be fixed.

Customers don’t need much experience with tools or training to use them, and they don’t need to know much about tools to use them. That’s why they’re popular. From both sides, it’s a pretty satisfying activity. The pen tester gets to give the developers a list of flaws that sound scary, and the developers get a list of bugs to add to their backlog. And of course professional penetration testing tools are getting more advanced all the time. This means they can detect more types of security vulnerabilities in less time, keeping costs and complexity low.

But automatic pen tests from a third party can’t take the place of a person doing the testing. While humans are slow and more expensive than automated defect-discovery tooling, because they can mimic human hackers, humans are better at evaluating an application’s response to a pen test and can possibly catch responses that automated software may miss.

One way to think about how you use pen tests is to think about how much it costs to find bugs. For example, using a static application security testing (SAST) tool is a much better way to find a cross-site scripting bug than using a pen test.

When you use pen tests to find security holes, each one takes up a small amount of your testing time, a small part of your checklist and a small amount of your time to write the report. And when companies run pen tests on an annual basis, the cost per defect rises as vulnerabilities get patched; the critical and high vulnerabilities eventually dwindle to nothing.

So if pen testing doesn’t work as well as it used to as a way to find bugs, how can you make the most of the expensive but more accurate human part of pen tests?

Where pen testing really starts to pay off is when organizations leave routine defect discovery to automated tools and shift the human effort toward AppSec program assurance.

Just as there are multiple points in a piece of software that could validate, detect or encode a cross-site scripting attack, there are multiple points in an SDLC that can predict, prevent, detect or mitigate various classes of vulnerabilities.

By shifting pen-testing efforts to finding those inflection points in your SDLC where vulnerabilities can be avoided, you’re using the human and financial costs of pen testing more efficiently than if you’re just focused on defect discovery.

Using pen testing in this way can help you find the SDLC processes that let vulnerabilities in. If you start changing those processes, you’ll also reduce the number of vulnerabilities in the future.

Instead of depending on pen tests to find security flaws one by one that need to be fixed, they should be used to do a blameless postmortem and figure out if changes need to be made to ensure that potential failures are caught at certain points in the SDLC.

There are many defect identification systems that can find single kinds of issues, but they are generally not great at finding edge cases. For that, you need human beings, and this is where the pen test can really work for your business.

There are many ways to use the results of a pen test to make your SDLC and code safer.

Apply policy and standards: Update your policies and standards to make it clear that production flaws are not allowed. Pay for training and tools, put finding and fixing flaws at the top of the list.

Based on the results of pen tests, senior leaders can agree to and make changes to policies and standards.

• Requirements: Start writing security standards that can be used more than once, can be put into action, can be tested and can address the OWASP top 10 vulnerabilities. When pen tests find security flaws, you can fix them by writing user stories and approval criteria.
• Training: When your pen tests find a security flaw, check to see if this problem is covered in your training. How can developers meet security standards without having received instruction on how to stop security faults? If developers aren’t trained to avoid security flaws, the training program has failed, not the development team.
• Review of the design and modeling of threats: By looking at the design, you can find and stop whole classes of problems before they happen. Use your pen-testing results to modify threat-modeling checklists to cover design decisions that could prevent or mitigate security faults and to create secure design patterns that allow developers to rule out entire classes of defects.
• Instead of depending on pen testing, shift to in-band SDLC flaw discovery tooling to discover and remediate entire classes of flaws.
  • Use static application security testing (AST) or code review checklists to stop devs who don’t know what they’re doing from missing well-known security holes. When reviewing the results of a pen test, check to see if you can use static defect discovery to find these problems and adjust your testing to make sure they are found.
  • You can find runtime bugs and setup problems that pen tests usually find with dynamic application security testing (DAST) or interactive application security testing (IAST).
  • QA-based security tests can be used instead of pen tests to find odd cases and logic flaws that scanning rule sets miss. When the architecture design review and threat modeling find dangerous logic, abuse scenarios should be made and run by the automated process.
  • Instead of testing after the fact, your production scans should practice like they play. By making the testing environment as close as possible to the deployment environment in terms of configuration settings, rights and infrastructure, configuration-related flaws in the gold image can be found and fixed before it is sent to production.
• Once pen testing finds flaws, you need to build gating methods to catch them and stop them from moving down the SDLC. After figuring out a defect’s requirement, usage cases, implementation guidelines and testing method, add checks to the life-cycle gates to catch and stop them from going into production.
• Production risk management should look for and fix production flaws, because it’s never too late to do things right. Using the results of your pen testing, you can automate detection and send changes back to be fixed, or you can set up manual push-to-release checks that lower environmental risk.

After these security problems have been fixed, you can start to move pen testing to a different place in the risk management portfolio.

When delivering penetration test results, there should be tools and information in place to help teams and AppSec program owners talk about which of their capabilities may not be working as well as they think they are. When you add pen-test results to your SDLC, the results can help you prioritize recommendations.

However, pen testing can’t provide direct evidence for training needs, compliance enforcement, vulnerability management needs, security champions or other key capabilities that don’t directly prevent or find pen-test security defects.

In conclusion, it’s time to rethink how pen tests are used in risk management. There are cheaper ways to find defects and faster ways to reduce risk in an application, but the pen test’s human-driven nature means that it will thoroughly exercise whichever capabilities are not being capitalized on.