TIPS & TRICKS

Configuring Okta Security Assertion Markup Language (SAML) Single Sign On (SSO) with Splunk Cloud

By Splunk September 01, 2016

As organizations grow, the number of applications and tools utilized to perform a job and support the business of the organization inevitably grows. It is not unheard of for enterprises to literally have hundreds of on premise, SAAS and Cloud based tools and applications. Making sure users of those applications are who they say they are means, at the least, one must authenticate themselves into the application. Although it was effective, people frowned on the practice of sticking a mass of Post-it notes on a monitor with user names and passwords. Password vault tools are a nice alternative to the Post-it, but it still means one has pull up the password vault app to look up a forgotten password to log in to this app, log in to that app, log in again when a session times out, log out, log in again, … ad nauseam.

Enter our glorious days of Single Sign On (SSO)! And that is a bandwagon that has many, many riders these days. The SSO cat has been skinned in many ways by many vendors and niche players. Some provide auto-fill of login/password prompts no matter what type of app or screen is presented. Others provide auto-fill of only web forms that present a user with fields for a username and password. And there are even cracker tools out there that do the same, in an attempt to brute force acquire a valid username/password to mischievously log into apps. You know, to exploit those users out there that like to use passwords like ‘abc123’ or ‘passw0rd’ so they don’t have to have so many stickies on their monitor or reduce the number of times they have to bring up their password vault on their smart phone…

In the spirit of SSO, a while back a bunch of smart folk asked the question “is there a better way to authenticate with SSO into an app without having to present fields for a username and password?” and answered that question with a resounding “yes!”. But first they had to create that better way. They put their heads together, designed a framework and wrote a Request for Comments (RFC) that defined that better way for securely passing authentication requests and responses between applications. They called it the Security Assertion Markup Language (SAML) which is now at version 2.0.

My role at Splunk>, as an Engineer on the Cloud Adoption team as part of our Customer Success organization, means that I exist here to help make our customers happy. Not having to type in a username and password to log into Splunk> to bring up your boss’s ‘TPS Reports’ dashboard is just one small way to bring happiness. So a team in your company must have got together and convinced management to purchase an Identity Provider (IdP) – Okta! Smiles ensued. Another way to ride the highway to bliss is for your organization’s over-worked IT Infrastructure staff to not have to own the hardware, floor space and admin head count to support that awesome instance of Splunk> that you’re getting huge piles of value out of. So that smart team at your company bought Splunk> Cloud and the party keeps on rolling!

By now you’re saying “I’m four paragraphs deep and I’ve yet to learn how to configure Okta!” – so enough of the background and let’s get down to it.

So what do you need? Okta? – check. Splunk> Cloud instance? – check.

Who do you need? 1) An administrator for your Okta instance 2) An administrator for your local Identity Management system (Active Directory, LDAP, etc.) 3) An administrator for your Splunk> Cloud instance. If they’re all the same person (you), you’re in luck. Otherwise you’ll have to run the calendar dice and find time for you all to discuss SAML integration, put in change control, scheduled a time to implement, etc.

Here’s what you do:

Pre-requisite:

This step is requested to be performed so that our Splunk> Cloud Support and Operations staff will know that you are integrating your instance with Okta. It provides a mechanism to more effectively support you in your efforts to integrate with Okta in case anything may go amiss, or you may have further questions around Okta configuration that are not addressed in this posting.

Log into your Splunk> Customer Portal and create a Splunk> Customer support case.
- A Priority of P3 or P4 is adequate.
- Choose ‘Authentication & Security‘ for the Area
- For the ‘Feature / Component / App‘ choose ‘SAML
- In the ‘Subject‘ enter in something along the lines of ‘SAML Integration with Okta
- Add a summary in the ‘Description‘ that you are going to integrate your Splunk> Cloud instance with Okta, and possibly a date/time you will be performing the integration if applicable.
Read all of the below Integration steps. There are some pieces that you may need to perform in your Identity Management environment before you integrate with Okta. There are also possible affects to your current locally defined users in Splunk> Cloud. And there may be other topics that may require further discussions among your team members or questions with Splunk>

Okta Integration:

The following steps are specific to versions 6.4.x of Splunk> Cloud. Okta is supported under Splunk> Cloud v6.3.1551.x and the steps below are nearly identical as well for that older version of Splunk (if your instance has not yet been upgraded).

First have your Splunk> Cloud administrator log into your instance as a user with the ‘admin‘ role. Yep, the ole manually entered username/password thing…
If you have multiple search heads in your Splunk> Cloud environment (aka a general search head at ‘https://<acme>.splunkcloud.com and/or possibly an Enterprise Security search head at https://<es-acme>.splunkcloud.com) you will need to perform a separate Okta integration for each search head independently. In short, you’ll have multiple Okta apps, one for each search head (or search head cluster).
Confirm that your instance is at version 6.3.1551.x or later by going to the top menu option ‘Support & Services‘ -> ‘About
Obtain your search head’s metadata.
This can be obtained by, once logged into a session as an admin role user, entering the URL https://<acme>.splunkcloud.com/saml/spmetadata into your browser’s URL field.
Something similar will be presented in your browser window:

From the metadata, capture the search head’s certificate (masked out above, between the XML tags ‘<ds:X509Certificate>‘ and ‘</ds:X509Certificate>‘. Save the certificate into a non-formatted text file (Notepad for example) and place a row above the certificate with the text ‘—–BEGIN CERTIFICATE—–‘ and a row below the certificate with the text ‘—–END CERTIFICATE—–‘. It should look something similar to:
Have your Okta admin log into your Okta instance as the Admin user.
Enter into the Admin functionality within Okta
Click on the link to ‘Add Applications
Click on the link to ‘Create New App
NOTE: Now – there is a ‘pre-canned’ App in Okta for Splunk> Although some customers have been successful in starting with this App to integrate Okta with their Splunk> Cloud instance, the number of changes to this App are enough where no time is really saved, and there’s been more confusion and mis-steps in the configuration versus starting from scratch with a new Okta app. Also, there have been customers that have gone through a ‘Wizard’ to create a SAML app for Splunk>. We’ve not experienced a clean, easy route to a working configuration through either the utilization of the pre-canned Okta app for Splunk>, nor the app creation Wizard. The most consistent method to reach a working integration is to create a new Okta App from this step forwards.
Choose ‘SAML 2.0‘ and click on the ‘Create‘ button
Enter in an application name and optionally upload a Splunk> logo for the Okta App widget. Suggested app names might be ‘Splunk> Cloud General’ or ‘Splunk> Cloud ES’ or ‘Splunk> Cloud Operations’. Make it something that helps identify which search head functionality your users will be logging into.
In the SAML General settings section, enter the ‘Single Sign On URL‘ in the format similar to ‘https://<acme>.splunkcloud.com/saml/acs‘ where <acme> is the DNS canonical name of the search head you are integrating Okta with.
Make sure the checkbox for ‘Use this for Recipient URL and Destination URL’ is checked.
Enter a unique name for the ‘Audience URI (SP Entity ID)‘. A suggested Entity ID name to uniquely identify the Splunk> Cloud search head is ‘splunk-‘ followed by the first field of the canonical name ‘https://acme.splunkcloud.com‘ – so ‘splunk-acme‘ for instance (Splunk-CustomerName example in the graphic below – do note that this field is case sensitive, so when this is used elsewhere in Okta for the Single Logout as well as when it is used in the Splunk> SAML configuration the case must match)
Set the ‘Name ID Format‘ to ‘Transient‘. And choose the ‘Application username‘ in the format you wish to have users identified within your Splunk> Cloud instance as they come across via SAML.
Here’s one of your first decision points. The ‘Application username’ is the value that is passed via SAML as the ‘nameID‘ attribute. The contents of this attribute will be used as the Splunk> account name once authenticated into Splunk>
If you have existing locally defined accounts in you Splunk Cloud instance, it would be good to have the users that authenticate through SAML to come across into Splunk with the same account name. For instance, if your locally defined user for ‘Joe Schmoe’ has a Splunk> account by the name of ‘jschmoe’, it would be good to have the SAML authenticated user to come across with the ‘nameID‘ as the same string ‘jschmoe’. If, however, Joe Schmoe logs into Okta as ‘jschmoe@acme.com’ then the ‘nameID‘ within the ‘Okta username’ will come across into Splunk> as the string ‘jschmoe@acme.com’ and thus be seen by Splunk> as a net new account.
What does this mean? It means that if Joe Schmoe has a bunch of knowledge objects saved under his old account named ‘jschmoe’, when he logs in via SAML he will no longer have access to those knowledge objects. They are owned by a completely different user under the account ‘jschmoe’ instead of his new account named ‘jschmoe@acme.com’ that was instantiated through the SAML authentication. Not good…
So – luckily there are other options in the pulldown for the ‘Application username‘ as is shown below.
But what if those pulldown choices do NOT match what your user logs into? Now what?! The Custom option will allow you to use the expression language to create a string specific to what you need to have users come across via SAML just as they were named when they were locally defined accounts. If this is a net new instance and you’re just getting up and running with Splunk> the topic is moot. But if you’ve had users for a long time and they have a lot of stuff they don’t want to lose – this piece is important. If you can’t find a pre-formatted pulldown option or an expression language option that will work or you, please do reach out to Splunk> Cloud support for further guidance.
Click on the ‘Show Advanced Settings‘ link to expose additional settings for the Okta App you are building.
Leave ‘Response‘ as ‘Signed‘. ‘Assertion Signature‘ as ‘Signed‘. ‘Signature Algorithm‘ as ‘RSA-SHA256‘. ‘Assertion Encryption‘ as ‘Unencrypted
Click the ‘Enable Single Logout‘ checkbox. Then enter into the ‘Single Logout URL‘ field the URL ‘https://<acme>.splunkcloud.com/saml/logout‘ where ‘<acme>’ again is the search head’s canonical name.
In the ‘SP Issuer‘ field, enter the same value you used in the ‘Audience URI (SP Entity ID)‘ in a previous field – ‘splunk-acme‘ for example. Again, do note that this is case sensitive and should match exactly what was typed into the ‘Audience URI (SP Entity ID)‘.
In this step you will upload the search head’s certificate that you saved into an unformulated text file in step 3. above.
Click on the ‘Browse‘ button to choose the file that contains the certificate.
Click on the ‘Upload Certificate‘ button, you should see a ‘Certificate Updated!‘ notification popup window that will indicate successful upload and parse of the certificate file. The ‘Signature Certificate‘ field should also populate with text similar to the example screen capture below.
Leave the ‘Authentication context class‘, ‘Honor Force Authentication‘ and ‘SAML Issuer ID‘ to their defaults
There are three other attributes that need to be passed to Splunk> Cloud from Okta. These are named ‘mail‘, ‘realName‘ and ‘role‘. Do note that these attributes are case sensitive and must be entered into your Okta App in exactly the case described.The ‘mail‘ attribute is the string you wish to have populated into your Splunk> user’s ‘e-mail’ field for their account. Thus it should be their valid work e-mail address.The ‘realName‘ attribute is the string you wish to have as the ‘Full Name’ of the user in the Splunk> account. This name is used throughout the Splunk> UI. For example it shows as the user’s name in the upper menu bar of the UI where the user can click to enter the menu to change their ‘User Settings‘ or ‘Logout‘ of their Splunk> session. The ‘role‘ is a list of groups that the user is assigned to within Okta and/or your Identity Management system (Active Directory, LDAP, etc.) Enter in “ATTRIBUTE STATEMENTS” section as follows:
Name: mail
Name format: Unspecified
Value: user.emailClick the “Add Another” button to add another Attribute line/entry and enter as follows:
Name: realName
Name format: Unspecified
Value: user.login
(see further notes below)Enter in “GROUP ATTRIBUTE STATEMENTS” section, add one entry as follows:
Name: role
Name format: Unspecified
Filter: Regex
(Filter textbox): .*
(again see notes below)* If you find that your Okta ‘user.login’ does not contain the text that would be nice to have in the ‘Full Name’ field in the Splunk> user account, there are other pre-canned choices in the pull down
However, you can also become quite creative by entering your own Okta expression language formula for the string. A common one that some customers have elected is to have the user’s first name and last name appear in the Full Name within their account. That way in the Splunk UI the user sees their full name – a much more user friendly experience. This can manually be entered with the formula “${user.firstName} ${user.lastName}” (without the quotes) as seen in the below screen shot:

* The role field is important. This is how we will (later) map group or groups the user is set to within your AD/LDAP environment to the Splunk> role or roles they should acquire once authenticated into your Cloud instance. It is not required that your Identity Management administrator create unique groups specific to your usage of Splunk>, however it does make it a bit easier to manage if you do set up groups. A suggested example of some AD/LDAP groups might be a set of groups named like:splunk-acme-admins (for those users that need access to the Splunk> instance and should have the admin Splunk> role)
splunk-acme-user (for those users that just need the user role)
splunk-acme-mycustomrole (for those users that should map to a custom role in Splunk> that you’ve created)
splunk-es-acme-admins (for those users that need to log into the https://es-acme.splunkcloud.com search head and need the admin role)
etc.Thus there may need to be some change control set up to create these groups and assign the specific users to those groups before you integrate Okta with your Splunk> Cloud instance.There is a way to also be more selective of the groups that are passed across in the role attribute from Okta into Splunk>. The regular expression can be more restrictive to only pass over a subset of the groups the user is assigned to. Using the above example named groups, we could enter the regular expression ‘splunk-.*‘ into the filter field of the role attribute and only those groups that match the string starting with ‘splunk-‘ will be passed through.
Click on the ‘Next‘ button to move to the next Okta App configuration panel
Choose the radio button for ‘I’m an Okta customer adding an internal app
Add additional options as you wish or leave blank.
At this point, you will be at the ‘Sign On‘ panel of the Okta application configuration. This screen provides you with a button to ‘View Setup Instructions’ and a link for the ‘Identity Provider metadata‘. Click the ‘Identity Provider metadata‘ and download/save that into a file on your local system. This will be later uploaded to your Splunk> Cloud instance.
The last step in Okta is to assign the new Splunk> Okta App to those people/groups you wish to have access to the Okta App widget you created.
Have the Splunk> Cloud administrator log into the instance if not already still logged in
Go to the Splunk> top menu option ‘Settings‘ -> ‘Access Controls
Click on the ‘Authentication Method‘ link
Choose the ‘External Authentication Method‘ radio button ‘SAML‘, then click on the ‘SAML Settings‘ button
Once in the ‘SAML Settings‘ panel, click on the ‘SAML Configuration‘ in the upper right hand corner
Click on the ‘Select File‘ button to choose and upload the ‘Identity Provider metadata‘ file that was saved from Okta in the steps above. Then click on the ‘Apply‘ button. It will populate several fields in the ‘SAML Configuration‘ panel from values within the metadata file.
Manually enter in the ‘Entity ID‘ field to match the ‘Audience URI (SP Entity ID)‘ that was used in the Okta App – ‘splunk-acme‘ for instance. Again remember that this is case sensitive so it should be typed in exactly as was used in the Okta app.
Scroll down to the ‘Advanced Settings‘ section.
Manually enter in the ‘Fully Qualified Domain Name (FQDN)‘ field the URL of your instance – ‘https://<acme>.splunkcloud.com‘ for instance
Manually enter a ‘0‘ (zero) in the ‘Redirect port – load balancer’s port
Click the ‘Save‘ button to save your configuration, your instance is now set to utilize SAML for authentication! – but the config is not finished yet….
Back at the ‘SAML Settings‘ panel, click on the ‘New Group‘ button in the upper right.
Enter a ‘Group Name‘ that is a group from your AD/LDAP environment. For instance, the example ‘splunk-acme-admins‘ would be the text entered as the group. Then click on one or more roles in the ‘Splunk Roles‘ ‘Available Items‘ selection list. It will copy over to the ‘Selected Item(s)‘ list. Note that it can be a one to many relationship – you can have a group map to one or more Splunk> Roles. Click the ‘Save‘ button to save your mapping – do note that Splunk> will lowercase all text that you enter (as it lower cases everything internally as it comes across in SAML). So if you have a group named ‘SPLUNK-ACME-USERS’ it will be the same as ‘splunk-acme-users’.
With the mapping in place, open up a separate browser or an incognito tab in your browser (and have the SAML tracer plugin running if you have one) and test a login. Either log into Okta as a user that is assigned the new Splunk> Okta app and click on the widget to initiate a SAML login, or simply go directly to your URL ‘https://<acme>.splunkcloud.com‘ and the SAML redirect should occur to authenticate via Okta into Splunk. Use the SAML tracer browser plugin to troubleshoot anything that might be amiss. If you authenticate successfully, be sure to check on the account information in Splunk> within the User Settings panel to make sure the account name (nameID), the ‘Full Name’ (realName), e-mail address (mail) came across as desired. Also check the user’s roles are mapped correctly via the ‘Settings‘ -> ‘Access Controls‘ -> ‘Users‘ list in Splunk>.
NOTE: Splunk> highly recommends that after SAML integration has been performed, any former locally defined user accounts should be removed through the UI by a user with the admin role. This does NOT remove that user’s knowledge objects, all it does it remove the locally defined password. By doing thus, a user account can only log in via a successful SAML authentication.

But what if it’s all messed up and doesn’t work?! How do I get back in to Splunk> Cloud and fix it?!

IMPORTANT: There is a specific URL for your instance that you can point your browser to in order to get to the locally defined user login page for locally defined users. This is the URL “https://<acme>.splunkcloud.com/en-US/account/login?loginType=splunk“. Use that URL, of course specific to your instance, if you need to log into any existing locally defined account. A suggestion is for you to create your own locally defined account that has the admin role assigned to it. That way you always have a way to log in locally with admin role if anything is wrong with SAML – a back door way to fix stuff if it ever becomes needed (without having to log a support case with Splunk in order to have it done for you).
Also test the logout process. Choose the username in the upper menu bar of Splunk> and choose the menu option ‘Logout‘. You should be successfully logged out of the Splunk> instance.
If all is well and you’re rockin and rollin, close the support ticket you opened in the pre-requisite steps.