Bypassing the Bypass: Detecting Okta Classic Application Sign-On Policy Evasion

This blog was co-authored by Splunker Jason Swierk.

In September 27, 2024, Okta disclosed a critical vulnerability affecting their Classic environment that created a concerning security gap in identity protection. The vulnerability, active since July 17, 2024, allowed attackers with valid credentials to bypass application-specific sign-on policies by simply modifying their user-agent string. This bypass enabled unauthorized users to circumvent crucial security controls, including network zones, device-type restrictions, and custom authentication requirements.

In this blog post, we will analyze the technical details of the Okta policy bypass vulnerability, explore its detection through existing analytic stories, and provide comprehensive hunting strategies for security teams. We'll examine how organizations can leverage Splunk's detection capabilities to identify both active exploitation attempts and historical incidents that may have occurred during the vulnerability window. Additionally, we'll provide detailed guidance on implementing effective monitoring strategies and establishing behavioral baselines to detect similar bypass attempts in the future.

Let's begin by examining the key aspects of this vulnerability before diving deeper into detection strategies, hunting techniques, and the ways defenders can protect their identity infrastructure.

Quick Facts

• Vulnerability: Application Sign-On Policy Bypass in Okta Classic
• Affected Timeline: July 17, 2024 - October 4, 2024
• Impact: Unauthorized access to applications despite configured security policies
• Attack Vector: Use of "unknown" user-agents with valid credentials
• Affected Systems: Okta Classic environments using application-specific sign-on policies
• Status: Patched as of October 4, 2024- Complexity: Low (requires only valid credentials and modified user-agent)

Technical Deep Dive

The Vulnerability

The vulnerability stems from a flaw in how Okta Classic processes authentication requests from non-standard user-agents. When a request came from a user-agent that Okta classifies as "unknown" (such as Python scripts or uncommon browsers), the system failed to properly enforce application-specific sign-on policies.

This bypass affected several critical security controls:

• Network zone restrictions
• Device-type enforcement
• Step-up authentication requirements
• Custom authentication policies

_{Figure 1: Flow of Attack (large image here), Splunk 2024}

The sequence diagram above illustrates both the exploitation path and detection opportunities for the Okta Classic Application Sign-On Policy bypass vulnerability that existed between July 17 and October 4, 2024. The flow shows how an attacker with valid credentials could bypass application-specific policies simply by using an unknown user-agent string, circumventing critical security controls such as network zones and device restrictions. The right side of the diagram highlights the parallel detection strategies defenders should employ, including user-agent analysis, historical access comparison, failed authentication attempt correlation, and behavioral analytics. These detection methods, when combined, help identify both active exploitation attempts and historical incidents that may have occurred during the vulnerability window.

Exploit Mechanics

The exploitation process follows these steps:

1. Credential Acquisition:

• Attacker obtains valid username and password credentials
• This could be through various means: phishing, credential stuffing, or other attack vectors

2. Policy Identification:

• Target organization must have application-specific sign-on policies
• Particularly vulnerable are applications with default policy rules (e.g., Microsoft Office 365, Radius)

3. Authentication Request:

• Attacker modifies their user-agent to one Okta classifies as "unknown"
• Submits authentication request with valid credentials
• Policy enforcement fails, granting access despite restrictions

4. Access Granted:

• Successful authentication bypasses intended security controls
• Attacker gains access to protected applications

Example Attack Sequence

POST /api/v1/authn HTTP/1.1
Host: customer.okta.com
User-Agent: PythonRequests/2.31.0z
Content-Type: application/json

{
  "username": "user@company.com",
  "password": "ValidPassword123",
  "options": {
    "multiOptionalFactorEnroll": false,
    "warnBeforePasswordExpired": true
  }
}

This seemingly innocuous request, when successful, could bypass carefully crafted security policies due to its non-standard user-agent.

Existing Detection Coverage

The Splunk Threat Research Team maintains several Okta-focused analytic stories that provide valuable detection coverage for this vulnerability. Three key stories are particularly relevant:

1. Okta MFA Exhaustion

This analytic story focuses on detecting attempts to overwhelm or bypass multi-factor authentication mechanisms. Key detections include:

• Okta MFA Exhaustion Hunt
• Okta Two or More Rejected Okta Pushes
• Okta Mismatch Between Source and Response
• Okta Account Locked Out

These detections are particularly relevant to the policy bypass vulnerability, as attackers might attempt MFA exhaustion techniques in conjunction with unknown user-agents to maximize their chances of success.

2. Okta Account Takeover

This comprehensive analytic story includes detections for various account compromise scenarios:

• Okta Authentication Failed During MFA Challenge
• Okta Multiple Failed MFA Requests For User
• Okta Multiple Users Failing To Authenticate From IP
• Okta New Device Enrolled on Account
• Okta User Logins from Multiple Cities

These detections help identify unusual authentication patterns that could indicate exploitation of the policy bypass vulnerability, especially when combined with unknown user-agent strings.

3. Suspicious Okta Activity

This analytic story provides broader coverage of suspicious behavior:

• Okta Suspicious Use of a Session Cookie
• Okta Account Lockout Events
• Okta Failed SSO Attempts
• Okta ThreatInsight Login Failure with High Unknown users

Detection Correlation

The real power comes from correlating these detections, which you can do using the following correlation search:

| tstats `security_content_summariesonly` 
values(All_Risk.analyticstories) as analyticstories 
sum(All_Risk.calculated_risk_score) as risk_score, 
count(All_Risk.calculated_risk_score) as 
risk_event_count,values(All_Risk.annotations.mitre_attack.mitre_tactic_id) 
as annotations.mitre_attack.mitre_tactic_id
WHERE All_Risk.risk_object_type = user All_Risk.analyticstories IN 
("Okta Account Takeover", "Suspicious Okta Activity","Okta MFA Exhaustion")

This correlation helps identify:

1. Users experiencing multiple types of suspicious activity
2. Sequential patterns of attacks (e.g., MFA exhaustion followed by policy bypass attempts)
3. Geographical anomalies combined with unknown user-agents

Hunting for Suspicious Identity Behaviors in Okta

While our analytic stories provide excellent detection coverage, hunting through Okta logs can uncover subtle patterns and behaviors that might indicate exploitation attempts or suspicious activity. The following hunting queries are designed to help security analysts identify potentially suspicious identity behaviors; however, it's important to note that not all identified behaviors are inherently malicious.

Important Considerations for Hunting

Before diving into specific queries, consider these key points:

1. Environmental Context

• Filter out known good IP addresses (e.g., VPN endpoints, office locations)
• Consider your organization's geographic footprint
• Account for legitimate business hours across time zones
• Document known automation tools and their user-agents

2. Baseline Establishment

• Understand normal authentication patterns
• Document expected MFA behaviors
• Map typical application access patterns
• Identify standard device types and user-agents

3. False Positive Reduction

• Maintain lists of expected service accounts
• Document known automation workflows
• Consider legitimate business travel patterns
• Account for remote work scenarios

Identity Behavior Analysis Hunt

This first query focuses on analyzing authentication patterns and MFA behaviors across your Okta environment.

`okta` earliest=-30d latest=now() 
(eventType IN ("user.authentication.auth_via_mfa", 
"user.mfa.factor.*", "user.authentication.auth_via_inbound_delauth", 
"user.authentication.auth_via_social", "user.authentication.sso"))
| eval time_period=if(_time >= relative_time(now(), "-15d"), 
"current", "previous")
| stats 
    values(eventType) as event_types,
    values(client.geographicalContext.country) as countries,
    values(client.ipAddress) as ip_addresses,
    dc(client.geographicalContext.country) as unique_countries,
    values(target{}.type) as target_apps,
    values(outcome.result) as outcome_results,
    values(outcome.reason) as outcome_reasons,
    count(eval(eventType IN ("user.mfa.factor.activate", 
"user.mfa.factor.update", "user.mfa.factor.deactivate"))) 
as mfa_changes,
    count(eval(outcome.result="FAILURE")) as failed_auth_attempts,
    count(eval(eventType="user.authentication.auth_via_mfa" AND 
outcome.result="SUCCESS" AND (client.device="Unknown" OR 
client.device="unknown"))) as unknown_device_mfa_success,
    count(eval(eventType="user.authentication.sso" AND 
outcome.result="SUCCESS" AND (client.device="Unknown" OR 
client.device="unknown"))) as unknown_device_sso_success
    by user
| where mfa_changes > 0 OR failed_auth_attempts > 0 OR 
unknown_device_mfa_success > 0 OR unknown_device_sso_success > 0
| eval geo_anomaly_score = case(
    unique_countries > 3, 3,
    unique_countries > 1, 2,
    1=1, 0
)
| eval suspicion_score = geo_anomaly_score + (mfa_changes * 2) + 
(failed_auth_attempts * 1) + (unknown_device_mfa_success * 3) + 
(unknown_device_sso_success * 2)
| sort - suspicion_score
| table user, suspicion_score, geo_anomaly_score, mfa_changes, 
failed_auth_attempts, unknown_device_mfa_success, 
unknown_device_sso_success, event_types, countries, ip_addresses, 
outcome_results, outcome_reasons, target_apps

_{Figure 2: Results over 30 days, Splunk 2024}

Purpose: This hunt helps identify users exhibiting unusual authentication patterns by analyzing:

• MFA configuration changes
• Failed authentication attempts
• Successful authentications from unknown devices
• Geographic dispersion of access
• Authentication outcomes and targeted applications

Key Features:

• Assigns weighted scores to different types of suspicious behavior
• Tracks MFA-related changes and failures
• Monitors unknown device authentications
• Creates a geographic anomaly score
• Provides comprehensive authentication context

Customization Notes:

• Adjust the time window based on your retention and analysis needs
• Modify the scoring weights to match your risk tolerance
• Add custom filters for known good IPs/ranges
• Tune the geo_anomaly_score thresholds for your organization's footprint
• Ensure proper logging of Okta data is in place. Some items may not fill if data is limited.

Advanced Authentication Pattern Analysis

The second query provides deeper insight into authentication patterns and potential policy bypass attempts.

`okta`
(eventType="user.authentication.sso" OR 
eventType="user.authentication.auth_via_mfa")
| eval is_unknown = if(client.device=="Unknown" OR 
client.device=="unknown", 1, 0)
| eval is_success = if(outcome.result=="SUCCESS", 1, 0)
| eval hour = strftime(_time, "%H")
| eval is_odd_hour = if(hour > 24 OR hour < 0, 1, 0)
| stats 
    count(eval(is_success=1 AND is_unknown=1)) as 
successful_unknown_sso,
    count(eval(is_success=0)) as failed_attempts,
    values(client.userAgent.rawUserAgent) as user_agents,
    dc(client.geographicalContext.country) as unique_countries,
    values(client.geographicalContext.country) as countries,
    dc(client.ipAddress) as unique_ips,
    values(client.ipAddress) as ip_addresses,
    dc(hour) as unique_hours,
    sum(is_odd_hour) as odd_hour_logins,
    dc(client.userAgent.rawUserAgent) as unique_user_agents,
    values(client.geographicalContext.city) as cities,
    dc(client.geographicalContext.city) as unique_cities,
    latest(_time) as last_activity,
    earliest(_time) as first_activity,
    values(target{}.type) as target_apps
    by user
| eval time_span = (last_activity - first_activity) / 3600
| eval avg_activity_per_hour = if(time_span > 0,
 (successful_unknown_sso + failed_attempts) / time_span, 0)
| eval suspicion_score = 0
| eval suspicion_score = suspicion_score + (successful_unknown_sso * 3)
| eval suspicion_score = suspicion_score + (failed_attempts * 1)
| eval suspicion_score = suspicion_score + (unique_countries * 2)
| eval suspicion_score = suspicion_score + (unique_ips * 1)
| eval suspicion_score = if(unique_hours > 12, suspicion_score + 5,
 suspicion_score)
| eval suspicion_score = suspicion_score + (odd_hour_logins * 2)
| eval suspicion_score = suspicion_score + (unique_user_agents * 3)
| eval suspicion_score = if(avg_activity_per_hour > 10, 
suspicion_score + 10, suspicion_score)
| eval geo_dispersion = if(unique_countries > 1, 
unique_ips / unique_countries, unique_ips)
| eval suspicion_score = suspicion_score + (geo_dispersion * 2)
| eval unusual_ua_score = if(like(user_agents, "%Python%") OR 
like(user_agents, "%curl%") OR like(user_agents, "%Postman%"), 15, 0)
| eval suspicion_score = suspicion_score + unusual_ua_score
| eval rapid_auth_score = if(avg_activity_per_hour > 5 
AND unique_countries > 1, 20, 0)
| eval suspicion_score = suspicion_score + rapid_auth_score
| eval mfa_bypass_score = if(successful_unknown_sso > 0 AND 
failed_attempts > 5, 25, 0)
| eval suspicion_score = suspicion_score + mfa_bypass_score
| eval city_hopping_score = if(unique_cities > (unique_countries * 
3), 20, 0)
| eval suspicion_score = suspicion_score + city_hopping_score
| eval credential_stuffing_score = if(failed_attempts > 10 AND 
successful_unknown_sso > 0, 30, 0)
| eval suspicion_score = suspicion_score + credential_stuffing_score
| eval policy_bypass_score = if(successful_unknown_sso > 0 AND 
unusual_ua_score > 0, 25, 0)
| eval suspicion_score = suspicion_score + policy_bypass_score
| eval sensitive_app_score = if(like(target_apps, "%Office365%") OR 
like(target_apps, "%Radius%"), 15, 0)
| eval suspicion_score = suspicion_score + sensitive_app_score
| sort - suspicion_score
| table user, suspicion_score, successful_unknown_sso, 
failed_attempts, unique_countries, countries, unique_ips, 
ip_addresses, user_agents, unique_hours, odd_hour_logins, 
unique_user_agents, avg_activity_per_hour, geo_dispersion, 
unusual_ua_score, rapid_auth_score, mfa_bypass_score, 
city_hopping_score, credential_stuffing_score, policy_bypass_score, 
sensitive_app_score, target_apps, cities

^{Figure 3: Output from query, Splunk 2024}

Purpose: This advanced hunt helps identify sophisticated attack patterns by analyzing:

• Unknown device authentications
• Temporal patterns
• Geographic dispersions
• User-agent anomalies
• Application targeting
• Authentication velocities

Key Scoring Components:

• Unusual User-Agent Score: Identifies automation tools and scripts
• Rapid Authentication Score: Detects high-velocity authentication attempts
• MFA Bypass Score: Correlates failures with successful unknown device access
• City Hopping Score: Identifies improbable travel patterns
• Credential Stuffing Score: Detects brute force patterns
• Policy Bypass Score: Specifically targets the policy bypass vulnerability
• Sensitive App Score: Weights access to critical applications

Customization Notes:

• Adjust scoring weights based on your threat model
• Add additional user-agent patterns specific to your environment
• Modify time-based thresholds for your operating hours
• Include custom application sensitivity weights
• Add organization-specific location logic

Effective Use of Hunt Results

When analyzing results from these hunts:

1. Focus on highest scoring users first
2. Look for clusters of related suspicious behaviors
3. Cross-reference with known security incidents
4. Document and tune false positives
5. Use results to improve detection engineering

Remember that these hunts are starting points; they should be customized to your environment and used as part of a broader detection strategy. Regular tuning and adjustment based on findings will improve their effectiveness over time.

Conclusion

The Okta Classic Application Sign-On Policy Bypass vulnerability (July 17 - October 4, 2024) demonstrates how seemingly simple configuration validations—in this case, user-agent verification—can lead to significant security implications when not properly implemented. While Okta has patched this vulnerability, it serves as a reminder that identity and access management systems require continuous monitoring and multiple layers of detection.

The detection strategy we've outlined combines three complementary approaches:

1. Pre-built analytic stories that provide immediate detection capabilities
2. Correlation searches that connect related suspicious activities
3. Advanced hunting queries that uncover subtle patterns and anomalies

What makes this vulnerability particularly concerning is its low complexity—requiring only valid credentials and a modified user-agent—combined with its high impact of bypassing critical security controls. Organizations should consider this incident as an opportunity to:

• Review and enhance their identity monitoring strategies
• Implement behavior-based detection alongside traditional rule-based approaches
• Develop robust hunting programs that can identify novel bypass attempts
• Establish strong baselines for normal authentication patterns
• Maintain comprehensive device and user-agent inventories

The hunting queries provided offer a starting point for organizations to build their own detection capabilities, but they should be customized based on:

• Geographic footprint
• Business hours across time zones
• Known automation tools and workflows
• Legitimate business travel patterns
• Application criticality

As attackers continue to find creative ways to bypass security controls, the ability to detect and respond to suspicious identity behaviors becomes increasingly crucial. Organizations should combine the detections and hunting approaches outlined here with robust security policies, regular access reviews, and continuous monitoring to maintain a strong security posture.