Skip to main content
  1. Posts/

Intrusion prevention systems: what they actually do, what they miss, and how to test them

··2751 words·13 mins·
Table of Contents
Testing an IPS against systems you don’t own or have written permission to test is the cloud-equivalent of unannounced Nmap scans against a non-scoped network: the techniques work, the legal exposure is real, and the engagement letter is what makes the difference. The blue-team side of this post is also operator-relevant on engagement, because every red team finding includes a paragraph about what the defender’s IPS did or didn’t catch.

The Intrusion Prevention System (IPS) is the technology that sits inline between the network and the internet and tries to stop bad traffic before it reaches the protected segment. The Intrusion Detection System (IDS) is the same technology in monitor mode, generating alerts without blocking. Both have been deployed for thirty years; both are still the spine of most enterprise network defense; and both have evolved into a fuzzier category called Network Detection and Response (NDR) that adds machine learning to what was originally rule-based pattern matching.

This post covers IPS and IDS as a working operator would think about them in 2026: what the detection models actually do, where the technology gets deployed, the open-source and commercial products that dominate the market, the testing tradecraft for evaluating an IPS deployment, and the corrected versions of the historical case studies that get cited (often incorrectly) in this kind of post.

What an IPS does
#

The job is straightforward in concept: read every packet on the wire, decide whether it’s malicious, and take action. The interesting work is in the deciding. Three detection models cover most deployed IPS technology:

Signature-based detection matches packet content (or session content, after the IPS reassembles it) against a database of patterns known to indicate specific attacks. A Snort rule, a Suricata rule, a Cisco SourceFire signature; they’re all instances of the same idea. Signature detection is fast, deterministic, and effective against attacks that match known patterns. It’s also blind to anything new and easy to bypass with simple obfuscation if the rule isn’t carefully written. The signature database has to be continuously updated; vendors like Cisco Talos, Emerging Threats, and Proofpoint publish updated rulesets daily or hourly.

Anomaly-based detection establishes a baseline of normal network behavior and flags deviations. Hosts that talk to unusual destinations, services that suddenly produce ten times their normal traffic, protocols appearing on ports they shouldn’t appear on. Useful against zero-day attacks that don’t match any signature; prone to false positives during normal change events (a marketing campaign launches and the web servers do ten times their normal traffic, legitimately). Hard to tune.

Behavioral analysis is the catch-all term for ML-driven detection that looks at multiple signals (process behavior on endpoints if you have agents there, network flow patterns, DNS query patterns, authentication events) and identifies attack-like behavior across the noise. Vectra AI, Darktrace, ExtraHop, Corelight (the commercial Zeek), and Microsoft Defender XDR all fit here. Modern NDR (Network Detection and Response) products lean heavily on behavioral analysis with signature-based detection as a complementary layer rather than the primary one. Gartner published its first NDR Magic Quadrant in May 2025, naming Vectra AI, Darktrace, ExtraHop, and Corelight as the Leaders quadrant; the NDR category is increasingly absorbing the standalone IPS market as enterprises consolidate on XDR platforms.

Deployment patterns
#

IPS goes one of three places on the network:

Inline at the perimeter, sitting between the internet and the network’s outside edge. Every packet crosses the IPS; the IPS can drop, modify, or alert on anything it doesn’t like. Performance becomes a constraint (a 10 Gbps IPS that drops to 2 Gbps under deep-inspection load is a 2 Gbps perimeter) and a failure mode (an IPS hardware failure inline is a network outage unless fail-open is configured). Most modern NGFW deployments are inline IPS.

Passive at the perimeter, with traffic mirrored to the IPS via a SPAN port or a network TAP. The IPS sees everything but can only alert; it can’t block. Lower performance impact (the IPS isn’t in the critical path) and no failure-mode risk to the network. Useful for IDS-style deployments where the SOC handles response separately.

Internal, deployed at network segment boundaries (between user VLANs and server VLANs, between corporate and OT/SCADA networks, between segments holding different sensitivity classes of data). Catches lateral movement after the perimeter is breached. Often passive in this deployment so the IPS doesn’t disrupt internal application traffic.

Most large enterprises run some combination: NGFW-with-IPS at the perimeter, dedicated IDS in monitoring positions at internal segment boundaries, host-based IDS / EDR on endpoints, plus a SIEM correlating across all of them. The IPS is one layer; the defense-in-depth model assumes it’ll miss things.

The open-source stack
#

Three products that operators meet in real environments:

Snort is the original, written by Martin Roesch starting in 1998. Sourcefire was founded around the project in 2001; Cisco acquired Sourcefire in October 2013 for $2.7 billion, which is why Cisco’s commercial IPS lineage runs through Snort. Snort 3, released GA in January 2021, was a near-total rewrite from C to C++ that added multi-threading (Snort 2.x was single-threaded), HTTP/2 inspection, and a more flexible rule language. The open-source version is still under active development; the commercial Cisco-backed productization is Cisco Secure IPS (formerly Cisco Firepower).

Suricata is the OISF’s (Open Information Security Foundation) IPS, originally funded by the U.S. Department of Homeland Security and first publicly released in 2010. Multi-threaded from day one, rule-compatible with Snort (you can use most Snort rules in Suricata and vice versa), with extensive protocol parsers and a JSON event output format that ships cleanly into a SIEM. Suricata is the open-source IPS most modern deployments reach for first; the Snort 2.x single-threaded limitation drove a lot of organizations to it, and most of those organizations stayed even after Snort 3 closed the gap.

Zeek is what Bro was renamed to, announced at BroCon in October 2018 (the original name was a cultural awkwardness Robin Sommer and the team had wanted to fix for years; the rename took effect with the 3.0.0 release in 2019). Zeek isn’t strictly an IPS; it’s a network security monitoring framework with a domain-specific scripting language that lets you express complex detection logic in code rather than in pattern-match rules. Zeek’s output is high-fidelity protocol metadata (every HTTP request, every DNS query, every SSL session, with the relevant fields extracted) that downstream tools can correlate. Corelight is the commercial vendor that productizes Zeek and is what most large enterprises buy for visibility-heavy deployments. Zeek pairs naturally with Suricata in a deployment where Suricata does signature detection and Zeek does behavioral logging.

The commercial NGFW market
#

Three dominant vendors in the commercial NGFW-with-IPS space:

Cisco Secure Firewall (the renamed Cisco Firepower, with the rebrand in flight since 2022) traces its lineage through the Sourcefire acquisition. Snort-derived IPS engine, AnyConnect VPN integration, Talos threat intelligence feed. The IPS engine is what made Sourcefire central to the acquisition; the rest of the platform is built around it.

Palo Alto Networks NGFW was founded in March 2005 by Nir Zuk, who had been one of the engineers behind Check Point’s stateful inspection technology and the founder of OneSecure before Palo Alto. The architectural distinction is the single-pass parallel processing (SP3) model: every packet gets inspected once for all checks (application identification via App-ID, user identification via User-ID, content scanning via Content-ID) in parallel rather than serially. The performance characteristic is what made Palo Alto the enterprise NGFW market leader for over a decade. Zuk announced his retirement in 2025 after twenty years.

Fortinet FortiGate uses ASIC-based content processors (NP and CP series) for line-rate inspection of large pipes. Founded by brothers Ken and Michael Xie in 2000. The pitch is performance at price: FortiGate boxes do more inspection per dollar than the Cisco or Palo Alto equivalents, with the trade-off that the management interface has historically been less polished and the threat intelligence less timely.

Beyond these three, the market includes Check Point (the original stateful-inspection firewall vendor, founded 1993, still significant in financial services and government), Juniper (SRX series), Sophos (acquired by Thoma Bravo 2020), and several smaller vendors. The cloud-native equivalents (AWS Network Firewall, GCP Cloud Firewall Plus, Azure Firewall Premium) are now meaningful competitors for cloud-native workloads.

Testing an IPS
#

The operator-side question is whether the IPS actually detects and blocks the attacks the customer expects it to. The testing workflow:

Run known-malicious traffic. Generate traffic that matches well-known signatures and verify the IPS catches it. Tools that do this:

  • hping3, nmap, and nping for packet-crafting and basic network-attack simulations.
  • Metasploit for known-exploit traffic. A Metasploit auxiliary scanner against a Suricata-inspected segment lights up most rule sets within seconds.
  • PCAP replay using tcpreplay against captured attack PCAPs (Mendel/MalwareBazaar, abuse.ch, and the Honeynet Project all publish samples).

Test evasion. The operator side of this is the same techniques attackers use to bypass IPS:

  • Fragmentation. Splitting an attack across multiple IP fragments or TCP segments to break signature matching on the payload. Tools: fragroute, custom Scapy scripts.
  • Encoding and obfuscation. Base64 encoding, URL encoding, Unicode encoding, hex encoding. Most modern IPS normalize before matching, but custom encodings still slip through.
  • Tunneling. Wrapping the attack in DNS, HTTPS, SSH, or another protocol the IPS doesn’t decrypt or fully inspect.
  • Timing manipulation. Slow scans (one packet per minute) that fall below threshold detection.
  • Protocol abuse. HTTP smuggling, HTTP/2 frame manipulation, TLS fingerprint spoofing.

Generate normal-looking traffic to test false-positive rates. A high-quality IPS deployment isn’t just one that catches attacks; it’s one that doesn’t alert on every benign user activity. Run scripted-but-benign traffic (legitimate web browsing, file downloads, software updates) and measure how many false alerts the IPS produces.

Coverage testing against MITRE ATT&CK. The modern framing is “which ATT&CK techniques does our IPS detect?” Tools like Atomic Red Team (Red Canary), Caldera (MITRE), and MITRE D3FEND/Engage map specific techniques to specific detection coverage. Run a technique, see whether the IPS catches it, document the gap, write a rule.

The historical tools-and-techniques list in older posts mentions LOIC for DDoS testing and CANVAS for exploitation. LOIC is irrelevant against any modern network defense. CANVAS (originally Immunity Inc., Dave Aitel’s commercial framework, with Immunity acquired by Cyxtera in 2019 and Aitel departing in 2020) continues to be sold but has been largely displaced by Metasploit Pro, Core Impact, and Cobalt Strike for commercial exploitation work.

Where the IPS misses
#

The honest part of any IPS evaluation is what the technology fundamentally can’t catch:

Encrypted traffic. IPS can either decrypt (with SSL inspection, requires intercepting CA in the trust store and adds significant performance cost and privacy concerns) or inspect blind (JA3/JA4 TLS fingerprinting, certificate inspection, behavioral patterns). The decrypting deployment catches more but breaks any TLS pinning the application does and is illegal or quasi-illegal in many regulated environments.

Encrypted protocols designed for evasion. DNS-over-HTTPS, DNS-over-TLS, Encrypted Client Hello (ECH) for TLS, QUIC. Each of these makes some inspection harder; together they’re moving the IPS toward a metadata-only inspection model where the actual content is opaque.

Insider threats. An authenticated user accessing data they have legitimate access to looks like normal traffic to an IPS. Detection requires user-behavior analytics rather than network IPS.

Living-off-the-land binaries (LOLBins) on the host. PowerShell, mshta, certutil, regsvr32 running from inside an authenticated session don’t produce network signatures the IPS will catch. Endpoint detection (EDR) is the answer here, not network IPS.

Targeted attacks with custom tooling. Nation-state-level adversaries write new malware per campaign. The IPS signature database catches commodity malware; bespoke campaigns slip through until the signature gets written after the attack.

Case studies (corrected)
#

Three historical examples that get cited frequently and often inaccurately.

Stuxnet (discovered June 2010)
#

The post version that gets passed around says “Stuxnet was ultimately detected by an IPS system that was in place at the target facility.” This is not what happened. Stuxnet was discovered in June 2010 by VirusBlokAda, a Belarusian antivirus company, on a customer’s machine in Iran that was crashing repeatedly. VirusBlokAda’s analysis identified the unusual zero-day exploits in the binary; Symantec and Kaspersky did the follow-up reverse engineering that mapped the worm’s specific targeting of Siemens Step 7 SCADA systems and confirmed Natanz as the target.

The Natanz facility itself was air-gapped, and Stuxnet’s propagation vector was USB drives, so there was no perimeter IPS that “caught” the attack. The actual containment was after-the-fact: once the worm was understood, defenders worldwide could write signatures, but Stuxnet had already accomplished its objective by the time the antivirus industry got its hands on a sample.

The lesson is the opposite of the one the original framing suggests: a sophisticated, targeted, air-gap-jumping attack will not be caught by a network IPS at the target facility, because the target probably doesn’t have one in the relevant position and the attack probably isn’t going to look like anything the IPS has a signature for.

WannaCry (May 2017)
#

The actual story: WannaCry propagated via the EternalBlue exploit against Microsoft SMBv1 (CVE-2017-0144), leaked by the Shadow Brokers from NSA tooling. Microsoft had released the MS17-010 patch in March 2017; organizations that applied it were not vulnerable. Organizations that didn’t were susceptible to a wormable spread that infected approximately 230,000 computers in over 150 countries within a single day.

The actual global containment was Marcus Hutchins (MalwareTech) registering the kill switch domain iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com on May 12, 2017, which halted propagation because the malware checked whether that domain resolved before continuing to spread. IPS rules for EternalBlue/SMBv1 existed and helped some organizations block the propagation, but the dominant containment factor was the kill switch plus the patch. Crediting “IPS technology” with stopping WannaCry overstates the actual contribution.

The lesson is more nuanced: IPS rules helped organizations that had them in the right place, but the technique that actually saved the world that weekend was a $10.69 domain registration by a researcher in his bedroom in Devon.

Target (December 2013)
#

The post version says “attackers gained access by exploiting a vulnerability in the company’s HVAC system.” This is wrong in the specifics, though the third-party-vendor framing is right.

The actual attack chain: attackers phished credentials from Fazio Mechanical Services, a Pennsylvania-based HVAC contractor that had vendor portal access to Target’s network for billing, contract submission, and project management. Fazio’s credentials gave the attackers access to Target’s vendor portal, from which they moved laterally to Target’s internal network, then to the POS infrastructure, where they installed memory-scraping malware on the point-of-sale terminals to capture payment card data as it was processed. Roughly 40 million payment card records were stolen between November 27 and December 18, 2013.

The HVAC system itself wasn’t the vulnerability. The vulnerability was the vendor portal’s network reachability, the lack of network segmentation between vendor-portal-accessible systems and the POS infrastructure, and Fazio’s susceptibility to phishing. Target settled for $18.5 million with state attorneys general in 2017 plus separate settlements with banks and consumers.

The lesson is about supply-chain security and network segmentation. An IPS at the segment boundary between vendor-portal systems and POS systems might have caught the lateral movement; that’s a real failing. But the framing of “Target didn’t have an IPS” undersells the deeper issue, which was network design.

What this comes down to
#

IPS is one of the older categories of network security and still one of the foundational ones. The technology has evolved from pure signature matching into a richer ecosystem of NDR products that incorporate machine learning, encrypted-traffic analysis, and behavioral modeling. The fundamental limitations haven’t gone away: encrypted traffic remains hard to inspect, insider threats remain invisible to network-layer defenses, and the signature-based model remains reactive to new attacks.

The operator’s job on engagement is to assess how well the customer’s IPS is actually doing its job, which usually means a combination of generating known-malicious traffic and verifying detection, generating evasion-modified traffic and measuring failure modes, and mapping detection coverage against MITRE ATT&CK. The defender’s job is to combine IPS with EDR, identity-based detections, and a SIEM that correlates across all of it, because no single layer catches everything and the gaps between layers are where attackers live.

Pick the open-source stack that fits your scale (Suricata + Zeek for most modern deployments) or the commercial vendor that fits your budget and existing infrastructure (Cisco, Palo Alto, Fortinet, Check Point). The marketing differences between them are mostly noise; the deployment differences (where the IPS sits, how it’s tuned, how its output gets correlated) are where the actual security value lives.

UncleSp1d3r
Author
UncleSp1d3r
As a computer security professional, I’m passionate about building secure systems and exploring new technologies to enhance threat detection and response capabilities. My experience with Rails development has enabled me to create efficient and scalable web applications. At the same time, my passion for learning Rust has allowed me to develop more secure and high-performance software. I’m also interested in Nim and love creating custom security tools.