Splunk is often deployed in environments with high security requirements.
These high security requirements may be due to PCI, SOX, FISMA, and/or HIPAA mandates as well as other factors such as high risk of industrial espionage or operation on a classified or top secret network.
Splunk is able to meet most high security requirements without code modification. Splunk relies whenever possible on open standards, which provides transparency and facilitates accreditation.
All data transmission between Splunk instances (e.g. forwarder to indexer, indexer to client browser) can be strongly encrypted (SSLv2/3).
All data transmission between third-party devices such as firewalls or intrusion detection systems (IDS) can be strongly encrypted as long as the third-party device supports strong ciphering and hashing and can perform a standard handshake with other devices.
Splunk can be hardened in a manner that meets or exceeds most hardening standards.
Aspects of Splunk that facilitate conformation with hardening standards includes but is not limited to:
The most efficient method for Splunk to comply with existing policies and standards is to authenticate and authorize users against an organization's existing directory which conforms to these policies already.
Thus, an LDAP strategy will be implemented which securely authenticates users and performs authorization by mapping user's group memberships to internal Splunk roles.
To ensure that the aforementioned requirement for secure communication is maintained, LDAP over SSL (also known as LDAPS) will be configured to encrypt all traffic between the directory server and Splunk.
This organization has the following information standards for applications:
Splunk provides for all of these requirements through its audit.log, file system change monitor, block signing, and alerting capabilities.
The following hardware was used in this deployment:
The hardware specifications may not seem important to this scenario given the requirements stated above. However, it is important to keep in mind that features such as traffic encryption/decryption and block signing will use additional hardware resources.
The data sources and their respective input methods for this deployment were as follows:
To get these estimates, we analyzed the available archive data and calculated daily averages. In some cases, there was approximately one week of archive data available, while in other cases more thorough archive data was available.
Using the estimates above, the Splunk instance is projected to index approximately 10 GB per day.
The licensing and input assumptions should be validated during the first 30 days in production. It is a low risk endeavor, as Splunk's licensing model will never terminate forwarding or indexing, and furthermore will only disable search if the license is violated 7 times within a 30 day rolling window.
Server installation was performed using the standard linux tarball downloaded from http://www.splunk.com/download. The Splunk binaries and logs will go under /opt, while the data store will be located at /data.
useradd -G splunk -m -u 666 splunk
$SPLUNK_DB=/data
/opt/splunk/bin/splunk enable boot-start -user splunk
To ensure that authentication and authentication to Splunk complies with enterprise security policies and password standards, we configured Splunk to authenticate to Active Directory(AD) and mapped security groups from AD to roles within Splunk. Rather than creating a separate user and group repository meeting policies and standards, it makes far more sense to leverage an existing directory where these policies and standards are already effective.
[role_security_ops] importRoles = Power;Everybody srchFilter = [role_network_ops] importRoles = Power;Everybody srchFilter = [role_hardware_ops] importRoles = Power;Everybody srchFilter =
The below authentication configuration was created in /opt/splunk/etc/system/local/authentication.conf:
[corpdc01] Admin = splunk_admin; security_ops = security_ops; network_ops = network_ops; hardware_ops = hardware_ops; SSLEnabled = 1 bindDN = cn=svc_splunk,ou=service_accounts,ou=users,dc=corp,dc=local bindDNpassword = failsafeLogin = admin failsafePassword = groupBaseDN = ou=splunk,ou=security_groups,dc=corp,dc=local; groupBaseFilter = (objectclass=*) groupMappingAttribute = dn groupMemberAttribute = member groupNameAttribute = cn host = corpdc01 pageSize = 800 port = 636 realNameAttribute = name userBaseDN = ou=users,dc=corp,dc=local userBaseFilter = (objectclass=*) userNameAttribute = sAMAccountName [auth] authSettings = corpdc01 authType = LDAP
To allow Splunk to connect to the LDAPS port on the domain controller corpdc01, the following steps were taken:
TLS_CACERT /opt/splunk/etc/openldap/certs/cert1.cer TLS_CACERTDIR /opt/splunk/etc/openldap/certs
Since Splunk was running as a non-root user, we could not bind to privileged ports (<1024). Thus, the following ports were decided upon:
| SplunkWeb port | Splunkd port | Splunk data port | Syslog Port |
| 8000 | 8089 | 9997 | 1514 |
By default, the SplunkWeb port is 8000 and the Splunkd port is 8089. Communication to and from the Splunkd port is encrypted by default. To enable SplunkWeb encryption, /opt/splunk/etc/system/local/web.conf was created as such:
[settings] enableSplunkWebSSL = true
To force each instance of Splunk to listen on the selected ports as well as to force encryption on the Splunk data port, /opt/splunk/etc/system/local/inputs.conf was created as such:
[SSL] serverCert=/opt/splunk/etc/auth/server.pem password=password rootCA=/opt/splunk/etc/auth/cacert.pem requireClientCert=false [splunktcp-ssl://9997] disabled = false queue = parsingQueue sourcetype = tcp-9997 [udp://1514] disabled = false sourcetype = syslog
The above configurations utilize the certificate and self-signed root certificate authority that are created when Splunk starts for the first time. If the customer had a requirement to leverage their own certificates generated from their enterprise root certificate authority, we would have replaced the serverCert, password, and rootCA values with the path and passwords to their own server certificate and rootCA certificate.
After configuration was completed, the service command was used to ensure that Splunk starts as the "splunk" user:
service splunk start
After changing the network devices to direct its syslog output to the correct UDP port, we installed Splunk in a lightweight forwarder configuration on the Windows servers as follows:
c:\program files\splunk\bin> splunk.exe disable webserver c:\program files\splunk\bin> splunk.exe set server-type forwarder
queue=indexQueue
[tcpout:splunk] server = Splunk.corp.local:9997 [tcpout-server://Splunk.corp.local:9997] sslCertPath=c:\program files\splunk\etc\auth\server.pem sslRootCAPath=c:\program files\splunk\etc\auth\cacert.pem sslPassword=password sslVerifyServerCert=false
c:\program files\splunk\bin> splunk.exe restart
c:\program files\splunk\bin> splunk.exe edit user admin -password r0tsky!~!1
After configuring all Windows forwarders as detailed above, we verified that the Splunk index server received new events from the domain controller event logs and network devices.
By default, auditing is enabled on the Splunk indexing server and audit events flow to the "_audit" index. Also by default, file system monitoring is enabled for the $SPLUNK_HOME/etc directory.
To meet the requirements for integrity monitoring and notification, we enabled block signing for Splunk's indexed data and file system change monitoring for the entire Splunk directory.
We first created an input which overrides the default file system monitoring, then created an input which will monitor the entire $SPLUNK_HOME directory.
[fschange:$SPLUNK_HOME/etc] disabled=true [fschange:$SPLUNK_HOME] disabled=false pollPeriod = 60 recurse=true followLinks=false hashMaxSize=-1 fullEvent=false sendEventMaxSize=-1 filesPerDelay = 10 delayInMills = 100
[main] blockSignSize=100
>source /opt/splunk/bin/setSplunkEnv >python /opt/splunk/bin/genAuditKeys.py
[auditTrail] privateKey = /opt/splunk/etc/auth/audit/private.pem publicKey = /opt/splunk/etc/auth/audit/public.pem
[audit_event alert] action_email = admin@example.com action_rss = 0 counttype = number of events enableSched = 1 quantity = 0 relation = greater than role = Admin schedule = */12 * * * * search = index=_audit path=/opt/splunk/bin* sendresults = 0 userid = 1
Using the above configuration, all the given project requirements were met without issue or significant and complicated configuration.
All data transmission to and from all Splunk instances has been encrypted in a manner which conforms to data encryption standards using Splunk's internal SSL libraries on the forwarder and receiver as well as IPSec between Splunk and enterprise network devices.
All Splunk instances were hardened to comply with hardening standards by reducing network port usage on the forwarders and indexer, running as non-privileged users on the forwarders (Local System) and the indexer (splunk), and requiring authentication on all Splunk instances.
Authentication and authorization within Splunk conforms to existing security policies and password standards, as Active Directory is used to authenticate users and map security groups to internal Splunk roles.
Splunk was configured conforms to existing data security and data asset lifecycle policies by enabling block signing as well as file system monitoring, all of which feed in to Splunk's existing audit facilities.
TBD