What is penetration testing and when should you commission one?

Most businesses have a rough sense that penetration testing is something security-conscious organisations do. Fewer have a clear understanding of what it actually involves, why it is meaningfully different from a vulnerability scan, which type of test their business needs, or how to know when the time is right to commission one. This guide answers all of those questions — in plain language, without the jargon that makes cybersecurity unnecessarily opaque.

If you have ever wondered whether your business needs a penetration test, the short answer is almost certainly yes. The longer answer depends on what you are trying to protect, what regulatory obligations apply to you, and where your highest risk areas are — and that is what this guide is designed to help you work out.

What Is Penetration Testing?

Penetration testing — commonly shortened to pen testing — is the practice of commissioning skilled security professionals to attempt to breach your systems using the same techniques, tools, and approaches that real attackers would use. The critical difference between a penetration test and an actual cyberattack is permission: a pen test is carried out under a formal written agreement that authorises the testers to probe your defences within a defined scope.

The goal is to find vulnerabilities — weaknesses in your websites, applications, infrastructure, or processes — before malicious actors do, document what was found and how it was accessed, and provide you with a prioritised roadmap for fixing the issues identified. A well-conducted penetration test does not just tell you that you have vulnerabilities; it shows you what an attacker could actually reach if they exploited them, which is the context that makes the findings genuinely actionable.

This distinguishes penetration testing from vulnerability scanning, which is an automated process that identifies known weaknesses by comparing your systems against databases of known vulnerabilities. Automated scanning is fast and useful — but it cannot determine whether vulnerabilities are actually exploitable in combination, it misses logic flaws that require human judgment to identify, and it cannot replicate the creative problem-solving that characterises a real attacker. Penetration testing is the human layer that validates and extends what automated scanning finds.

How Penetration Testing Works in Practice

A penetration test follows a structured methodology. The OWASP Web Security Testing Guide is the industry-standard framework for web application testing; CREST publishes methodology guides for network and infrastructure testing. Whatever the specific framework, the process follows consistent phases:

Scoping and authorisation. Before any testing begins, the scope is formally agreed in writing — which systems will be tested, what methods are permitted, what is explicitly out of scope, and the timeframe for testing. The authorisation document protects both parties: the tester has written permission to conduct what would otherwise be illegal activity, and you have a clear record of what was agreed.

Reconnaissance and information gathering. The tester collects information about your systems from publicly available sources — DNS records, SSL certificate details, technology stack indicators, employee information on LinkedIn, exposed ports and services. This mirrors what a real attacker would do in the early stages of targeting an organisation.

Vulnerability identification. Using a combination of automated tools and manual analysis, the tester identifies potential weaknesses — outdated software, misconfigured services, injection vulnerabilities, insecure authentication, exposed sensitive data. This phase produces a list of candidates for exploitation.

Exploitation. The tester attempts to actively exploit identified vulnerabilities to determine whether they can be used to gain unauthorised access, escalate privileges, access sensitive data, or move laterally through your systems. This is the phase that distinguishes a penetration test from a vulnerability assessment — the tester is actively trying to breach your defences, not just cataloguing potential weaknesses.

Post-exploitation analysis. When access is gained, the tester documents what could be reached from that position — which data, which systems, which further attack paths. This contextualises the risk: a vulnerability that allows access only to a public-facing server with no sensitive data is a different severity level from one that provides a path to your customer database.

Reporting. The deliverable is a comprehensive report documenting every finding, the exploitation path used to access it, the potential business impact, and specific, actionable remediation steps. A good report has an executive summary accessible to non-technical leadership alongside detailed technical findings for the engineering or IT team.

Remediation and re-testing. Findings are fixed by your team, and a re-test confirms that the vulnerabilities have been correctly remediated. Some testing providers include a re-test within the original engagement; others offer it as a separate service.

The Five Types of Penetration Test

Web application penetration testing

The most commonly commissioned type of penetration test, web application testing, assesses your websites, web applications, customer portals, and APIs for security vulnerabilities. The testing methodology follows frameworks like the OWASP Top 10 — the definitive list of the most critical web application security risks — and covers SQL injection, cross-site scripting (XSS), authentication and session management flaws, access control vulnerabilities, security misconfiguration, and more.

If your business has a website that customers log into, an e-commerce platform, a customer portal, or any web-based application, web application penetration testing is the most directly relevant test to your risk profile. It is the starting point for most businesses that have not previously conducted formal security testing.

Network and infrastructure penetration testing

Network penetration testing assesses your server infrastructure, cloud environment, firewalls, VPNs, and internal network for vulnerabilities. It can be conducted from an external perspective — simulating an attacker on the internet trying to breach your perimeter — or from an internal perspective, simulating an attacker who has already gained access to your internal network through phishing or a compromised endpoint.

External network testing is relevant for any business with internet-facing server infrastructure. Internal network testing is particularly important for businesses where the consequences of an insider threat or a phished employee would be severe — financial services, healthcare, businesses with valuable intellectual property.

Social engineering testing

Social engineering testing assesses the human layer of your security — whether your employees can be manipulated into disclosing information or taking actions that create security vulnerabilities. Phishing simulations send realistic fake phishing emails to your staff and measure what percentage click, enter credentials, or open attachments. Vishing (voice phishing) tests involve phone calls attempting to extract sensitive information from staff. Physical security testing may include attempting to tailgate into secure premises or leaving USB drives in car parks to see whether they are plugged in.

According to IBM’s Cost of a Data Breach report, phishing is consistently one of the leading initial attack vectors in confirmed data breaches. If your technical defences are strong but your people have not been tested, you have a blind spot. Social engineering testing reveals it in a controlled way.

Mobile application and API testing

If your business has a mobile app — on iOS, Android, or both — it requires dedicated security testing that accounts for the specific attack surface of mobile applications: local data storage security, transport security (whether data in transit is properly encrypted), authentication and session handling, and the security of the API endpoints the app communicates with. Mobile testing is often underinvested relative to web application testing, despite mobile apps frequently handling sensitive customer data.

Red team exercise

A red team exercise is the most comprehensive form of penetration testing — a full attack simulation that combines technical, physical, and social engineering techniques over an extended period, typically weeks or months. The objective is not just to find vulnerabilities but to test your organisation’s ability to detect, respond to, and contain a sophisticated attack. Red team exercises are most appropriate for larger organisations with mature security programmes who want to test the effectiveness of their detection and response capability, not just their preventive controls.

Black Box, Grey Box, and White Box: Choosing Your Methodology

Beyond the type of test, you will need to agree the knowledge level the pen tester begins with — a choice that affects both the realism of the test and its efficiency.

Black box testing gives the tester no prior information about your systems. They begin exactly as a real attacker would — searching for information from public sources, probing for open ports and services, and working methodically to find a way in. This provides the most realistic simulation of an external attacker, but it is less efficient than other approaches because significant time is spent on reconnaissance that your own team could short-circuit.

Grey box testing gives the tester partial information — for example, user-level credentials to a web application, or documentation about the technology stack — but not full administrative access. This is the most commonly recommended approach for most businesses, because it combines realistic attack simulation with enough context to allow the tester to focus their time on the highest-risk areas rather than spending it on reconnaissance. For a given budget, grey box testing typically produces a more thorough and useful result than black box testing.

White box testing gives the tester complete information — source code, architecture documentation, admin credentials, network diagrams. This allows the most thorough review of code-level vulnerabilities and is particularly valuable when the objective is a comprehensive security audit of application code rather than a realistic attack simulation. Development teams commissioning security review of their codebase before a major release typically choose white box testing.

When Should You Commission a Penetration Test?

There are two categories of timing for penetration testing: trigger events that should prompt immediate action, and scheduled cadences that should be maintained regardless of whether a specific event has occurred.

Immediate trigger events

After a security incident. If your business has experienced a breach, ransomware attack, or any other security incident, a penetration test is part of the remediation process. The incident response process identifies and fixes the specific vulnerability that was exploited — but a penetration test reveals whether other vulnerabilities of similar severity exist that the attacker did not exploit but that remain open. Returning to normal operations after an incident without testing for residual vulnerabilities is a common and serious mistake.

Before a major launch or deployment. A new ecommerce platform, a customer portal, a mobile app, a significant website rebuild — any major new system that will handle customer data or provide access to your business systems should be tested before it goes live, not after. The cost of finding and fixing a vulnerability in a controlled pre-launch environment is a fraction of the cost of remediating it after a breach.

After significant infrastructure changes. A cloud migration, a new data centre deployment, a major CMS upgrade, or the integration of significant new third-party services changes your attack surface. The security configuration of the new environment needs to be validated through testing, not assumed to be correct because the deployment appeared to go smoothly.

When clients or contracts require it. Enterprise clients, public sector organisations, and businesses in regulated industries are increasingly including penetration testing requirements in supplier security questionnaires and contracts. A current pen test report — typically from within the past twelve months — is becoming a standard expectation for businesses seeking to work with larger organisations. For guidance on common vulnerabilities and how to prevent them, our dedicated guide covers the technical detail in depth.

Scheduled regular testing

Beyond trigger events, penetration testing should be a recurring scheduled activity — not a one-off exercise. Vulnerabilities are introduced continuously through software updates, configuration changes, new integrations, and the ongoing discovery of new attack techniques. A test that found no critical vulnerabilities twelve months ago does not mean no critical vulnerabilities exist today.

The appropriate frequency depends on your risk profile: annually is the baseline recommendation for most businesses; every six months for ecommerce businesses processing payment data and other high-volume customer transaction platforms; quarterly for financial services, healthcare, and other businesses in regulated industries handling the most sensitive data categories.

What to Look for in a Penetration Testing Provider

The quality of penetration testing varies significantly between providers. The key indicators of a credible provider are:

CREST accreditation. CREST is the independent body that accredits penetration testing firms and certifies individual testers. CREST accreditation means the firm’s methodologies, processes, and tester competence have been independently assessed. For web application and infrastructure testing, CREST accreditation in the provider should be a minimum requirement.

Sector-relevant experience. A provider who has tested businesses similar to yours — in size, sector, and technology stack — will be more efficient and more likely to find the vulnerabilities that are most relevant to your specific environment. Ask specifically about comparable engagements.

Clear, written scope of work. Before any testing begins, the scope, methodology, permitted techniques, excluded systems, timeline, and deliverables should all be documented and agreed in writing. This protects both parties and ensures there is no ambiguity about what the engagement covers.

Reports that non-technical stakeholders can read. Ask to see a sample report. It should include an executive summary that communicates risk in business terms — not just technical severity ratings — alongside the detailed technical findings. If the report would be incomprehensible to your leadership team, its findings will not be acted on.

Formal remediation support. The test is only valuable if the findings are fixed. Ask whether the provider offers guidance on remediation, and whether a re-test to confirm fixes is included or available. For comprehensive ongoing security support following a penetration test, the managed technical support team at Matrix Internet can help implement and verify recommended fixes.

What a Penetration Test Does Not Cover

Understanding the limitations of penetration testing is as important as understanding its value. A penetration test is a point-in-time assessment — it tells you the state of your security at the moment of testing, not six months later when new vulnerabilities have been introduced. It is not a substitute for ongoing security practices: regular patching, access control management, security monitoring, and staff awareness training.

A penetration test also works within an agreed scope. Systems excluded from the scope are not tested. If your scope agreement excludes a third-party payment processor, for example, vulnerabilities in that integration will not appear in your report. Scope decisions made to limit disruption or cost can create blind spots — be explicit with your provider about what you want tested and why.

Finally, a penetration test report is only as valuable as the remediation effort that follows it. A report that sits unread or whose findings are partially addressed and then forgotten has provided a false sense of security rather than genuine protection. The test must be followed by a structured remediation process with defined ownership, timelines, and verification. For guidance on recovering from a security incident if the worst happens, our dedicated guide covers the process in full.

Summary

Penetration testing is the practice of commissioning authorised ethical hackers to test your systems using real attack techniques, producing findings and recommendations that allow you to fix vulnerabilities before they are exploited by malicious actors. It is meaningfully different from vulnerability scanning because it validates exploitability, requires human judgment, and reveals what an attacker could actually access rather than just what weaknesses theoretically exist.

The five main types — web application, network infrastructure, social engineering, mobile and API, and red team — address different parts of your attack surface. For most businesses, web application testing is the most immediately relevant starting point. Grey box testing is typically the most efficient methodology. Annual testing is the minimum cadence for most organisations, with more frequent testing for businesses in regulated industries or those processing significant volumes of payment or health data.

The right time to commission your first penetration test is before an incident makes it urgent. If your business has never been formally tested, handles customer data, processes payments online, or is growing into new systems and markets, the conversation about penetration testing is overdue. The cybersecurity and technical support team at Matrix Internet works with businesses across Ireland and Europe to scope and deliver penetration testing engagements, from initial web application tests through to comprehensive infrastructure assessments. Get in touch to discuss what your business needs.