A diverse and inclusive Splunk helps us achieve our mission of making data accessible, usable and valuable to everyone, and drives great outcomes for our company, our communities and each other. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most meaningfully to each other’s success. Learn more about Splunk careers and how you can become a part of our journey.

As a Senior DevOps/Observability Engineer, you will be responsible for implementing observability use cases within Splunk’s corporate IT division. You will leverage your knowledge of logs, metrics, and traces to bring infrastructure monitoring (IM) and application performance monitoring (APM) to Splunk’s various IT services. You will help define strategy, standards and best practices that will leverage DevOps tools to automate implementation as well as on-going support.

As a member of the Splunk@Splunk team, you will have Splunk’s full suite of observability products at your disposal, along with a team of world-class engineers to collaborate with. In addition, you will have the opportunity to share your innovations, insights, and lessons learned with the Splunk community (e.g. via open-source, conference talks, blog posts, etc.).

The Splunk@Splunk team is renowned for excellence in engineering, both inside and outside of Splunk. Last year, we presented five highly-rated sessions at .conf20, published multiple blog articles, and open-sourced several of our innovations via Splunk’s GitHub project, including but not limited to: ansible-role-for-splunk, docker-swarm-splunk-hf, and TA-VaultSync.

Splunk@Splunk is also a strong partner with the Products and Engineering teams, where we’ve contributed to everything from Splunk-built addons (e.g. Zoom, Gmail, Falcon Data Replicator), to the health checks and alerts that ship with Splunk’s Distributed Management Console (DMC), to the feature roadmap for Data Stream Processor (DSP). If you’re looking for a team where you can make a big impact, this is it!

What you'll do: Yeah, I want to and can do that.

  • Collaborate with the team to develop cutting-edge solutions that monitor Splunk’s numerous IT services
  • Seize opportunities to automate the configuration and deployment of various observability tools (e.g. SignalFx, Plumbr, Rigor, OpenTelemetry, Splunk)
  • Instrument infrastructure monitoring, application performance monitoring (APM), and tracing using Splunk Infrastructure Monitoring (SignalFx) combined with Ansible roles and Terraform providers
  • Develop synthetic monitoring checks in Python
  • Implement security best practices in our team’s AWS-based infrastructure, Python code, and various Docker services

Requirements: I’ve already done that or have that.

  • 8+ years working with distributed systems at scale in IT
  • 5+ years APM and infrastructure monitoring experience
  • 5+ years DevOps experience
  • 3+ years experience with configuration management (Ansible) and infrastructure provisioning tools (Terraform, CloudFormation)
  • 3+ years experience with Docker and container orchestration (Kubernetes, ECS, EKS)
  • 5+ years experience in cloud platform administration (preferably AWS)
  • Strong knowledge of IT Security (encryption, firewalls, secure systems design, vulnerability management, secure software design, credential management, proxies, WAFs, etc.)
  • Deep understanding of IT fundamentals, including Linux and Microsoft operating systems, networking, security, cloud, and database technologies
  • Excellent communication skills, both verbal and written; able to explain complex technical topics to both internal and external customers with ease
  • Fully proficient in git and version control systems (GitLab, BitBucket, GitHub)
  • Passionate about IT, technology, and following the latest industry trends

Preferred knowledge and experience: These are a huge plus.

  • Experience developing in Golang or writing custom Terraform providers
  • Experience developing custom Ansible modules
  • Published open-source projects on GitHub
  • Splunk experience a plus, but not required
  • Certifications in AWS, Docker, Ansible, and/or Security (e.g. CISSP, CEH, Security+, etc.)

Education: Got it.

  • Bachelor’s degree or higher in Computer Science, Information Technology, Computer Information Systems, Information Security, or a closely related field

What We Offer You: Wow, I want that.

  • An opportunity to innovate cutting-edge solutions using Splunk software for Splunk itself. Plus, the ability to share those innovations with our customers! How cool is that?
  • A rare opportunity to join one of the best Splunk engineering teams in the world with a stable, collaborative, and supportive work environment that encourages work-life balance.
  • A constant stream of new things for you to learn. We're always expanding into new areas, bringing in open source projects and contributing back, and exploring new technologies.
  • A set of extraordinarily hardworking, innovative, open, fun and dedicated peers, all the way from engineering and QA to product management and customer support.
  • Growth and mentorship. We believe in growing engineers through ownership and leadership opportunities. We also believe mentors help both sides of the equation.

We don't expect people to work 12-hour days. We want you to have a successful time outside of work too. Want to work from home sometimes? No problem. We trust our colleagues to be responsible with their time and dedication and believe that balance helps cultivate an extraordinary environment

This isn’t a job – it’s a life changer – are you ready?

Splunk has been named one of San Francisco Bay Area’s “Best Places to Work” by the San Francisco Business Times, ten years in a row. We offer a highly competitive compensation package and a plethora of benefits.

Splunk is proud to be an equal opportunity workplace and is an affirmative action employer. We value diversity at our company. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or any other applicable legally protected characteristics in the location in which the candidate is applying. For job positions in San Francisco, CA, and other locations where required, we will consider for employment qualified applicants with arrest and conviction records.

Splunk's Hiring Practices
Splunk turns machine data into answers. Organizations use market-leading Splunk solutions with machine learning to solve their toughest IT, Internet of Things and security challenges.

Individuals seeking employment at Splunk are considered without regards to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition (except where physical fitness is a valid occupational qualification), genetic information, veteran status, or any other consideration made unlawful by federal, state or local laws. Click here to review the US Department of Labor’s EEO is The Law notice. Please click here to review Splunk’s Affirmative Action Policy Statement.

Splunk also has policies in place to protect the personal information candidates disclose to us as part of the application process. Please click here to review Splunk’s Career Site Privacy Policy.

Splunk does not discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Please click here to review Splunk’s Pay Transparency Nondiscrimination Provision. 
For job positions in San Francisco, CA, and other locations where required, we will consider for employment qualified applicants with arrest and conviction records.

Splunk is also committed to providing access to all individuals who are seeking information from our website. Any individual using assistive technology (such as a screen reader, Braille reader, etc.) who experiences difficulty accessing information on any part of Splunk’s website should send comments to accessiblecareers@splunk.com. Please include the nature of the accessibility problem and your e-mail or contact address. If the accessibility problem involves a particular page, the message should include the URL of that page.

Splunk doesn't accept unsolicited agency resumes and won't pay fees to any third-party agency or firm that doesn't have a signed agreement with Splunk.

To check on your application click here.
Find out what makes Splunk such a great place to work
Our Values

Splunkers are encouraged and empowered to be Innovative, Passionate, Disruptive, Open and Fun.
Learn More

Our Locations

From San Francisco to Shanghai, Splunkers work in 25+ offices across the globe.
Learn More

University Recruiting Program

Intern with people you want to hang out with, even outside the office.
Learn More

Our Blog

Hear from Splunkers on the latest.
Learn More

Diversity & Inclusion

Culture of Inclusion: Splunkers Share Their Stories
Learn More


Follow Splunk on LinkedIn for job announcements, company news, and more.
Learn More