How Intrusion Detection Systems (IDS) Work: One Part of Your Security Arsenal

All cyberattacks that violate the confidentiality, integrity and availability (CIA) of the network and data demonstrate some form of anomalous behavior. The starting point of this behavior may be an unauthorized intrusion into the network and, then, unauthorized use of the resources contained within the network.

If you can identify an unauthorized network intrusion attempt, you can maintain the CIA of your data assets and network resources. This is precisely the goal of an Intrusion Detection System (IDS).

What are intrusion detection systems?

An Intrusion Detection System refers to the technology that monitors network traffic for anomalous behavior such as cyberattacks or security policy violations.

Once the event is identified as an anomaly, it reports the administrators or issues an automation control action to the integrated Security Information and Event Management (SIEM) tool. The SIEM tool then uses advanced filtering techniques and protocols to distinguish a legitimate intrusion attempt from false alarms when raising an alert.

(Explore Splunk Enterprise Security, a leading SIEM solution.)

IDS vs firewall

This process is different from a firewall mechanism, which simply filters and implicitly prevents a possible network intrusion.

So why not just a firewall system instead? Modern enterprise IT networks are complex. The networks include thousands of network endpoints and nodes communicating between each other — no fixed set of rules can encompass a holistic and uniform security policy for the entire network. Therefore, IDS systems are deployed at various network nodes to determine potential violation of a network security policy.

(Curious about prevention? IPS handles that.)

Types of intrusion detection systems

Intrusion Detection Systems can be broadly categorized into the following groups:

Signature-based Detection (SD)

Signature-based detection systems use existing knowledge of attack signatures to identify intrusion attempts. If a traffic request matures a previous unauthorized intrusion attempt, an alarm goes off. A database of attack signatures is maintained and used to compare against current attempts to access the network. These systems are highly accurate in matching known attack signatures.

However, a Zero-Day exploit may not contain any signature knowledge in the database. If such an attack does not demonstrate characteristics and patterns from previously known and available list of attack signatures, it will not be identified by the IDS that relies on SD techniques. After all, SD is a simple detection system that uses contextual knowledge for simple security policy enforcement decisions.

Signature-based detection does have drawbacks:

• Requires continuous updates to the signatures database, otherwise you’re quickly out of date.
• Inadequate understanding of the TCP/IP protocols and states means that it cannot develop an adequate and intelligent understanding of signature patterns.
• Ineffective against Zero-Day exploits.
• Intensive on manual configurations and administrative work to keep the signature database up to date.

Anomaly-based Detection (AD)

The limitations of Signature-based Detection are overcome by Anomaly-based Detection systems, which model the behavior of the systems using statistical functions, knowledge-based methods or machine learning techniques.

The models train and generalize on the network system’s response to allowed traffic and known attack signatures. Any deviation from the expected system response — allowing legitimate traffic and rejecting traffic that contains patterns of attack signatures — triggers an alert.

The positives of AD systems are that they are less dependent on the underlying technology stack and OS. New vulnerabilities can be easily detected as long the model is sufficiently trained to classify a legitimate traffic request from an unauthorized intrusion attempt. New vulnerabilities such as Zero-Days exploits are less concerning, as explicit signature knowledge is not required. However, the drawbacks of AD systems include:

• Modeling complex network systems is a hard problem. It requires ongoing training of the models as the traffic patterns evolve.
• As the observed events constantly change, building correct traffic profiles becomes a challenging task.
• Alerts may not be raised in real-time, or require explicit training, before a malicious intrusion attempt with slightly anomalous deviations is correctly classified as an authorized traffic request.

(Solve common anomaly detection challenges.)

Stateful Protocol Analysis (SPA)

This system evaluates protocols of the TCP/IP stack. The intrusion engine runs at the application layer and uses predefined protocol profiles for each protocol state activity as provided by the vendor.  These are universal and standardized profiles that describe how a protocol should govern traffic flows. Any deviation constitutes an anomalous behavior and hence, triggers an alarm.

An example would be that the intrusion attempt initiates an unexpected sequence of attempts without issuing prerequisite commands. The SPA system would check for the protocol profile characteristics, such as length of the command and order sequence, to determine a potentially unauthorized network intrusion attempt.

On the positive, these systems are well positioned to distinguish between traffic protocol sequences, especially as the states are explicitly known and tracked. The information is universally available and standardized across technology vendors. Now to the difficult part:

• Implementation remains challenging. Internal expertise and tools may be required to carefully understand and classify traffic behaviors based on state information.
• States information alone may be inadequate to determine the legitimacy of a traffic request. Additional analysis techniques may be required, which study traffic content and signatures.
• If the technology is incompatible with specific operating systems and APIs, the SPA system may need to be reconfigured and customized to extract the required information around protocols and state profiles.

IDS = one part of your security arsenal

These IDS systems don’t provide an actual defense against malicious intrusion attempts — they’re not firewall systems, but a piece of the large security puzzle.

The decision of rejecting network traffic requests may be difficult to represent as a single policy or rules that are enforced by a firewall system. Instead, Intrusion Detection Systems help InfoSec teams understand traffic behavior and make well-informed decisions based on true contextual knowledge, instead of relying on fixed and predefined policies.

What is Splunk?

This posting does not necessarily represent Splunk's position, strategies or opinion.