Study Guide
Assurance and the Limits of Penetration Testing
What assessments, certifications and penetration tests can and cannot show, why a breach can follow a clean result, and how to use assurance as part of an ongoing programme.
Lesson 5.8 Governance, Risk & Compliance
Managers, customers and regulators all want to know whether an organisation’s security actually works. Assessments, certificates and penetration tests help answer that question, but each answers a narrower question than it may seem to. This lesson works out what each kind of assurance shows, what it cannot show, and why a breach shortly after a clean result is not necessarily a contradiction.
Stage: Think
The house survey
Think it through
A surveyor checks a house before you buy it and finds nothing wrong with the roof. A year later the roof leaks. List the reasons this could happen without the survey having been wrong.
- Hint 1: What could the surveyor not see?
- Hint 2: What happened after the survey?
Show what happens
The survey looked at certain things on one day with certain methods; the roof may have aged or been damaged since; and the surveyor may never have been able to see the part that failed. Every form of security assurance has the same kinds of limit.
Think it through
Show the answer
Answer: A. Little was found, within its limits.
That testers found little within that scope and time. It says nothing about what was not tested, or about later changes.
Stage: Predict
Assurance answers a specific question
Think it through
Any check of security has to decide three things before it starts. What might they be?
- Hint 1: Think of the surveyor: what, when and how?
Show what happens
What it will look at (its scope), when (its time) and how (its method). Its result is reliable for that question and no further.
Every form of assurance looks at a defined scope, at a particular time, using a particular method. Its result is reliable for that question and no further. GRC introduces the short version of this point: compliance is not the same as security.
| Form | What it shows | What it cannot show |
|---|---|---|
| Self-assessment, such as Cyber Essentials | That the organisation has answered defined questions about specified controls, and the answers have been verified. | That every answer matches reality, or that threats outside the controls are covered. |
| Independent technical verification, such as Cyber Essentials Plus | That an assessor has tested a sample of the same controls within scope. | That untested systems, or the same systems next month, are in the same state. |
| Certification audit, such as ISO/IEC 27001 | That the management system in the certified scope met the standard’s requirements when audited. | That every control is effective or that no breach can occur. |
| Penetration test | What authorised testers could find and exploit within an agreed scope and time. | That nothing else is exploitable, or that an attacker with more time would fail. |
| Continuous monitoring, such as a SOC | That activity is being watched and alerts can be acted on. | That every attack will be noticed in time. |
Check your understanding
Show the answer
Answer: C. That someone watches activity.
That activity is being watched and alerts can be acted on, not that every attack will be noticed in time.
For example, an organisation can complete a Cyber Essentials self-assessment while malware remains undetected on its network. Likewise, an organisation can hold ISO/IEC 27001 certification and run a 24/7 Security Operations Centre, yet still suffer a breach. Neither point means assurance is worthless; each means it has limits.
Stage: Predict
What a penetration test is
Think it through
A tester and a criminal might use exactly the same techniques against the same website. What makes one a test and the other an attack?
- Hint 1: What does the tester have that the criminal does not?
Show what happens
Permission. A test is authorised in advance, in writing, within agreed limits; an attack has no such permission.
A penetration test is an authorised attempt to find and exploit weaknesses, as an attacker would, within limits agreed in advance. Ethics, Authorisation and Responsible Disclosure explains why written testing authorisation is what separates a test from an attack. A test is bounded in several ways:
- Scope: which systems, applications, sites or people may be tested, and which may not.
- Timeframe: the days or weeks in which testing happens.
- Rules of engagement: which techniques are permitted, when testing may take place and who to contact if something breaks.
- Output: a report of findings with their severity and recommendations. Such reports are highly sensitive, because they describe how to attack the organisation.
Check your understanding
Show the answer
Answer: C. It shows how to attack it.
Because it describes how to attack the organisation: its findings are a map of weaknesses.
Each of those bounds narrows what the final report can speak for.
Four stages, each smaller than the one before. A clean report describes the last stage only. Everything left behind at an earlier stage was not examined.
Everything the organisation relies on
Its own systems, its staff, its buildings, its suppliers and every way in through each of them.
narrows to
What was in scope
The systems the organisation chose to have tested, such as one website. Email, suppliers and staff are usually left out.
narrows to
What there was time for
A fortnight of work by a small team. An attacker can take months.
narrows to
What the report describes
The weaknesses those testers found in that scope, with those techniques, in that fortnight.
Go deeper Kinds of penetration test named in the Security+ objectives
By how much the testers are told beforehand.
- Known environment: the testers are given full details, such as network diagrams and accounts. Thorough, and efficient with time.
- Partially known environment: they are given some information, for example an ordinary user’s account.
- Unknown environment: they are told almost nothing and start where an outside attacker would.
By which side is being exercised.
- Offensive: testers play the attacker and try to get in.
- Defensive: the exercise tests how well the organisation’s defenders notice and respond.
- Integrated: the two sides work together and share what they learn as they go.
By how the testers gather information first (reconnaissance).
- Passive reconnaissance: collecting what is already public, without touching the target’s systems.
- Active reconnaissance: interacting with the target’s systems, for example by scanning them.
Tests of locks, doors and reception desks are physical penetration tests. Every kind is bounded in the same four ways.
Stage: Predict
Why a breach can follow a clean test
Think it through
Before reading on, give two reasons a company could be breached a month after a clean penetration test.
- Hint 1: Did the test cover everything, for ever?
Show what happens
The test was bounded in scope and time and the attacker is not; systems change, new vulnerabilities appear, and the attacker may use a route the test never examined.
What does it mean if an organisation is breached shortly after a penetration test? Two limits explain it:
- A test is bounded. It is performed within a defined scope and timeframe, so it cannot guarantee that every vulnerability will be found.
- An attacker is not. A malicious attacker may have considerably more time to study a target than a tester working within a formal engagement.
Further reasons add to these: systems change after the test, new vulnerabilities are disclosed, the attacker may use a route outside the agreed scope, such as a supplier or a phishing email, and findings may not yet have been fixed.
A fictional company commissions a two-week penetration test of its customer website. The report finds only minor issues, which are fixed.
- Month 1. The test covered the website. The staff email system and the company’s IT support supplier were out of scope.
- Month 3. A new vulnerability in the website’s content system is disclosed. It did not exist in any public form during the test.
- Month 4. An attacker instead phishes an employee of the IT support supplier and uses that company’s remote access.
The test was not wrong: within its scope and time it found what it could. The breach came through a route it never examined, after conditions had changed. The lesson is to know what the test covered, test again after significant change and manage the other routes through other controls.
Check your understanding
Show the answer
Answer: B. Through an out-of-scope supplier.
Through the IT support supplier’s remote access, which was outside the test’s scope.
Stage: Predict
Thinking like an attacker
LulzSec was a small group of hackers who, in 2011, broke into the systems of a series of large organisations. Ryan Ackroyd was one of its members, and was later convicted in the UK for his part in it. The case is used here for one lesson, about assumptions. Defenders work within defined frameworks, processes and expected behaviours. Attackers are not bound by any of them, and look for the places the structure does not reach, including the gaps between one team’s responsibility and the next. In short: defenders follow defined structures, while attackers look for the gaps between them.
Think it through
Defenders follow defined structures. Where might an attacker look that a structure does not reach?
- Hint 1: Who is responsible for the space between two teams?
Show what happens
The gaps between teams and responsibilities, routes nobody owns, and interactions nobody planned for, such as a supplier’s access or a forgotten system.
For assurance, this means asking what an assessment’s structure might miss, not only whether the organisation passed it. Effective defenders think creatively about bypasses and unexpected interactions, while still working within their authorisation.
Stage: Predict
Using assurance well
Think it through
A penetration test report lists twelve findings. What has to happen next for the test to have been worth doing?
- Hint 1: Does a report fix anything by itself?
Show what happens
Each finding needs an owner and has to be fixed, accepted or escalated; fixes need retesting; and the next test should follow significant change.
Penetration testing is one component of an ongoing security programme, not a one-time guarantee. Several practices follow from that:
- Know the scope. Read what an assessment or certificate covered before relying on it, including a supplier’s.
- Act on findings. A report changes nothing until its findings are assigned owners and fixed, accepted or escalated, as described in Treating Risk.
- Retest. Check that fixes worked, and test again after significant changes.
- Combine forms. A certificate shows the management system; a test shows exploitable weaknesses; monitoring watches for attacks in between. Each covers gaps the others leave.
When a supplier or colleague says “we’re certified” or “we passed our pen test”, the useful follow-up questions are: what exactly was in scope, when was it done, what was found and has it been fixed? The answers show how much the result can be relied on.
Stage: Apply
Apply it in a new situation
These questions use the lesson in situations it has not shown. For the strongest test of your understanding, come back to them on another day and answer each one at the first attempt.
Apply it
Show the answer
Answer: B. What scope did it cover?
What the certified scope covered, and whether the service you will use is inside it.
Apply it
Show the answer
Answer: C. Test it again after the change.
A retest: a significant change, especially one handling payments, falls outside what the first test could show.
Apply it
Show the answer
Answer: A. A scoped penetration test.
A penetration test scoped to that route, because it shows what authorised testers could actually find and exploit.
Stage: Check
Check your understanding
These questions come from the approved revision bank. Choose an answer and check it; the explanation and its sources follow.
Show answer
Answer: False
Assurance is limited by scope, evidence and timing. As with an MOT, passing does not eliminate later faults or risks outside the assessment; ongoing effectiveness still matters.
Revision sources: Session 3: Governance, Risk and Compliance (Classroom session record) · Session 4: Governance, Risk and Compliance — In Depth (Classroom session record) · Governance, Risk and Compliance (GRC) (Supplementary Study Guide explanation)
Show answer
Answer: Independent technical verification of the same control areas, in addition to verified self-assessment.
Cyber Essentials uses verified self-assessment; Plus adds technical verification. Scope and timing still limit assurance, and the schemes do not confer ISO certification or guaranteed security.
Revision sources: Session 3: Governance, Risk and Compliance (Classroom session record) · Session 4: Governance, Risk and Compliance — In Depth (Classroom session record) · Security Frameworks and Standards (Supplementary Study Guide explanation)
Show answer
Answer: The test worked within a defined scope and timeframe, so it could not guarantee finding every vulnerability, and an attacker may have far more time than the testers.
A penetration test is performed within a defined scope and timeframe, so it cannot guarantee that every vulnerability is found, and a malicious attacker may have considerably more time to study the target than the testers had. A breach after a clean test therefore does not prove the test was faulty, and no single result guarantees security for a set period. Penetration tests assess technical weaknesses; they are not documentation checks.
Revision sources: Session 2: Cyber Fundamentals Part 2 (Classroom session record)
Show answer
Answer: Keep security as an ongoing programme: a penetration test is one component, not a one-time guarantee, and certification or a 24/7 SOC cannot guarantee every attack is prevented.
Penetration testing is one component of an ongoing security programme, not a one-time guarantee that an organisation cannot be compromised. ISO/IEC 27001 certification and a 24/7 Security Operations Centre provide assurance and capability, but neither can guarantee that every attack is prevented. More frequent testing still leaves periods and routes that a test does not cover, so it cannot replace ongoing security activity.
Revision sources: Session 2: Cyber Fundamentals Part 2 (Classroom session record) · Session 4: Governance, Risk and Compliance — In Depth (Classroom session record)
Show answer
Answer: A penetration test shows what authorised testers could find within an agreed scope and time.; Continuous monitoring shows that activity is being watched, not that every attack will be noticed in time.
A test works within a defined scope and timeframe and cannot guarantee finding every vulnerability, so it says nothing about what lies outside its scope. Certification provides assurance but cannot guarantee every attack is prevented. Monitoring shows that activity is watched rather than that every attack is caught. Systems also change and new vulnerabilities emerge after a test, so a result describes the systems as they were tested.
Revision sources: Session 2: Cyber Fundamentals Part 2 (Classroom session record) · Session 4: Governance, Risk and Compliance — In Depth (Classroom session record) · Assurance and the Limits of Penetration Testing (Supplementary Study Guide explanation)
Show answer
Answer: A penetration test of the website within an agreed scope and timeframe.
Each form of assurance answers a narrower question. A penetration test shows what authorised testers could find and exploit within an agreed scope and time, which is the board’s question. A self-assessment shows that answers about specified controls were verified; independent technical verification, as in Cyber Essentials Plus, tests a sample of those controls; a certification audit checks the management system within its scope; and monitoring watches for activity. There is also a limit: even a clean test cannot guarantee that every vulnerability was found.
Revision sources: Session 2: Cyber Fundamentals Part 2 (Classroom session record) · Assurance and the Limits of Penetration Testing (Supplementary Study Guide explanation)
Study and revision
Six approved questions relate to this lesson. Compliance versus security and the Cyber Essentials Plus distinction are practised through the Governance & Policy topic. A breach after a clean test, penetration testing as part of an ongoing programme, what forms of assurance can show and which form answers a given question are practised, with a flashcard on penetration-test limits, through the Security Assessment & Penetration Testing topic.
Stage: Consolidate
What to remember
- Every form of assurance has a scope, a time and a method, and is reliable for that question only.
- A penetration test is an authorised attempt to find and exploit weaknesses within agreed limits; its report is highly sensitive.
- A breach can follow a clean test: the test was bounded, the attacker is not, and conditions change.
- Attackers look for the gaps between defined structures.
- Use assurance well: know the scope, act on findings, retest after change and combine forms.
Key terms
- Assurance
Evidence that gives confidence that security requirements are met or controls are working, within a defined scope and time.
- Penetration test
An authorised assessment that looks for weaknesses in systems and safely demonstrates them, within an agreed scope and timeframe.
Common misunderstandings
It shows what testers found within an agreed scope and time. Other routes, later changes and new vulnerabilities are outside it.
Certification confirms the scoped management system met requirements when audited. Breaches can still occur.
Compliance shows specified requirements were met. Security depends on how well controls address the actual threats.
Lesson sources and acknowledgements
Sources and scope
- Taught in Session 2 (see the Session 2 Learning Log): the question I raised about a breach following a penetration test; tests performed within a defined scope and timeframe; attackers having more time; penetration testing as one component of an ongoing programme.
- Taught in Session 3 (see the Session 3 Learning Log): compliance is not the same as security; the Cyber Essentials self-assessment with undetected malware; penetration-test reports as highly sensitive information; Cyber Essentials Plus adding independent technical verification.
- Taught in Session 4 (see the Session 4 Learning Log): ISO/IEC 27001 certification and a 24/7 SOC not guaranteeing prevention; the Ryan Ackroyd and LulzSec discussion; defenders following structures while attackers look for gaps. This reflects the classroom discussion, not a claim about any organisation’s controls.
- Supplementary explanation: the comparison of forms of assurance, how a test is bounded, the narrowing diagram, the “Go deeper” list of test types (their names are those of CompTIA Security+ objective 5.5), the two sentences saying who LulzSec and Ryan Ackroyd were, the additional reasons a breach can follow a test, the fictional company, the practices for using assurance, the practical note and the misunderstandings.
- Guided version (published 1 November 2026, awaiting the site owner’s editorial review): the house-survey opening questions, every question asked in the lesson with its feedback and hints, the application questions and the closing summary are supplementary explanation written for the Study Guide. The settings are fictional. The questions in the lesson’s steps are not revision-bank items; those in “Check your understanding” are, unchanged.
- No practical work is claimed. I have not carried out or commissioned a penetration test, audit or certification assessment.
Evidence that gives confidence that security requirements are met or controls are working, within a defined scope and time.
Meeting applicable legal, contractual and internal requirements.
A measure intended to modify risk, for example by restricting access, detecting misuse or supporting recovery.
A UK certification scheme covering five technical control areas and based on verified self-assessment. Cyber Essentials Plus adds technical verification.
The international standard specifying requirements for an ISMS within a defined scope. Organisations can be certified against it.
Malicious software designed to disrupt, damage or gain unauthorised access to devices, networks or data. Ransomware is one type.
An authorised assessment that looks for weaknesses in systems and safely demonstrates them, within an agreed scope and timeframe.
A deceptive message designed to trick the recipient into an action, such as revealing sign-in details or opening a malicious link.
A team that continuously monitors an organisation’s systems, detecting, analysing and investigating possible security incidents and escalating confirmed ones.
A document that specifies requirements or recognised practices to be met.
A potential cause of harm, such as an attacker seeking unauthorised access.
A weakness that a threat may exploit.
Your progress on this lesson
Saved in this browser only. It is not sent anywhere, does not follow you to another device, and is lost if you clear this site’s data.
This browser is not allowing the site to save progress, so it will be forgotten when you leave the page.