Defining PCI DSS – where does application testing fit in?

When clients approach us for PCI DSS (Payment Card Industry Data Security Standard) compliance checks, they are usually only looking for internal and external infrastructure tests. While it is rare that we are asked to do a web application test as part of a PCI assessment, many recent breaches have involved attackers gaining access through an organisation’s public-facing web applications.

This raises some important questions – why are there are not stricter rules in place for the testing of web applications? Or are the PCI DSS requirements simply not being adhered to properly?

The challenge of understanding PCI DSS

Section six of PCI DSS , “develop and maintain secure systems and applications” has two notable sections, 6.5 and 6.6.
Section 6.5 outlines common vulnerabilities that are often found in applications during the development process, including injection flaws (SQL injection), insecure cryptographic storage, cross-site scripting and cross-site request forgery – a good baseline of vulnerabilities, which all sites should be checked against.

Section 6.6 notes that public-facing web applications should address new threats and vulnerabilities on a regular basis, and that applications should be protected against known attacks. To do this it advises that one of two methods are used:

• The first is a “vulnerability security assessment”, performed either manually or automatically. This specifies that all the common vulnerabilities mentioned in section 6.5 should be checked for and resolved at least annually.
• The second option is to use an automated technical solution that detects and prevents web-based attacks. However, this would suggest that you have the option of implementing a solution such as a web-application firewall (WAF) or automated web vulnerability scan, instead of performing tests to check against everything in section 6.5.

Similarly, section 11.3 of the specification outlines the requirements for penetration tests and appears to contradict the general PCI interpretation paradigm of section 6.6. The Information Supplement: Penetration Testing Guidance document has a lot of important information about the scope and type of tests needed to be carried out in a penetration test. The “Scope” section of this document specifies that “testing must include both application-layer and network-layer assessments”, and the next section (page 9) states that “any software written by or specifically for the organisation that is part of the penetration test scope should be subject to both an application and network-layer penetration test”. Unlike requirement 6.6, this document does not mention using an automated technical solution instead of testing the application for vulnerabilities, and implies that just having an automated system in place is not enough to become compliant.

Requirement 11.3 in the full PCI DSS specification also states that “application-layer penetration tests to include, at a minimum the vulnerabilities listed in requirement 6.5”. This requirement implies that all applications should have been tested against everything in requirement 6.5. Again, this contradicts requirement 6.6 as having the application tested against everything mentioned in requirement 6.5 is one of two options for that requirement.