Skip to main content

Principal Software Engineer - Backend/Analytics (Observability)

Join us as we pursue our disruptive new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly to each other’s success.

As applications and systems become more sophisticated and user experience is at high stake, observability - the ability to supervise and understand the systems and how they impact users, becomes one of the biggest challenges for engineering teams. We are building world-class tools to help engineers deliver better, faster, and more reliable applications.

We are looking for a Principal Software Engineer to join our Splunk Observability teams. As a technical leader in the team, you will closely collaborate with Architects and Product Managers to design very large scale distributed systems, streaming data pipelines, and sophisticated algorithms that power the Splunk Observability products.


  • You are a backend software generalist with interest or experience in algorithm development. You will run foundational experiments to benchmark and tweak for scalability and performance and build in fault-tolerance.
  • You will reason about trade-offs (timeliness/accuracy/resource consumption) of various algorithmic solutions and to highlight the difference in product outcomes.
  • You will participate in the design, implementation, maintenance, and evolution of Splunk Observability’s backend systems.
  • You will deploy, operate, and scale Splunk Observability’s backend services in production environments and help improve engineering quality and operation perfection for the whole team.
  • With your domain expertise, you will make recommendations to influence the roadmap. This role is pivotal to generate ideas and share what is possible with streaming algorithms.


  • Several years of experience building large-scale backend systems.
  • Proficiency in Java 8 (or above) and familiar with Java development environment
  • Understanding of analytical/statistical algorithms in computer science. Experience with approximate data structures is a plus.
  • The ability to adapt quickly and solve problems through ambiguity.
  • Strong communication skills, both verbal and written.
  • Leadership and mentoring skills and collaboration with engineers and non-engineers of various skill levels.
  • Prior experience with Jupyter, NumPy, or visualization libraries like matplotlib, seaborn or similar Python libraries is helpful but not required
  • Understanding of data science workflow and/or experience collaborating with data scientists helpful but not required
  • It would help, but not required, if you have experience with these systems that are the foundation of Splunk Observability’s backend systems - Kubernetes/Docker, Kafka, Cassandra, Druid, Presto, AWS, and GCP.

We value diversity at our company. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or any other applicable legally protected characteristics in the location in which the candidate is applying.

For job positions in San Francisco, CA, and other locations where required, we will consider for employment qualified applicants with arrest and conviction records.

Splunk's Hiring Practices
Splunk turns machine data into answers. Organizations use market-leading Splunk solutions with machine learning to solve their toughest IT, Internet of Things and security challenges.
Individuals seeking employment at Splunk are considered without regards to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition (except where physical fitness is a valid occupational qualification), genetic information, veteran status, or any other consideration made unlawful by federal, state or local laws. Click here to review the US Department of Labor’s EEO is The Law notice. Please click here to review Splunk’s Affirmative Action Policy Statement.
Splunk also has policies in place to protect the personal information candidates disclose to us as part of the application process. Please click here to review Splunk’s Career Site Privacy Policy.

Splunk does not discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Please click here to review Splunk’s Pay Transparency Nondiscrimination Provision.

Splunk is also committed to providing access to all individuals who are seeking information from our website. Any individual using assistive technology (such as a screen reader, Braille reader, etc.) who experiences difficulty accessing information on any part of Splunk’s website should send comments to Please include the nature of the accessibility problem and your e-mail or contact address. If the accessibility problem involves a particular page, the message should include the URL of that page.

Splunk doesn't accept unsolicited agency resumes and won't pay fees to any third-party agency or firm that doesn't have a signed agreement with Splunk.

To check on your application click here.


Find out what makes Splunk such a great place to work

box1 box1
Our Values

We are Splunk. How? Well we're passionate about customer success, driven by data, real and respectful, serious about fun, powered by our Million Data Points, and are all in this together.

Learn More
box2 box2
Our Locations

From San Francisco to Shanghai, Splunkers work in 25+ offices across the globe.

Learn More
box3 box3
Early Talent Program

Intern with people you want to hang out with, even outside the office.

Learn More
box3 box3

Our Blog

Hear from Splunkers on the latest.

Learn More
box2 box2
Diversity & Inclusion

Culture of Inclusion: Splunkers Share Their Stories

Learn More
box1 box1

Follow Splunk on LinkedIn for job announcements, company news, and more.

Learn More