Skip to main content
false
Engineering

Principal Site Reliability Engineer

  • - No Remote

Job Description

Join us as we pursue our ground-breaking vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we are committed to our work, customers, having fun, and most significantly to each other’s success.

The Splunk Observability Cloud provides full-fidelity monitoring and fixing across infrastructure, applications, and user interfaces, in real-time and at any scale, to help our customers keep their services reliable, innovate faster, and deliver great customer experiences. Infrastructure Software Engineers at Splunk are cloud-native systems engineers who use infrastructure-as-code, microservices, automation, and efficient design to build, operate, and scale our products.

Role

You will help us run one of the largest and most sophisticated cloud-scale, bigdata, and microservices platforms in the world. You will be responsible for enabling developers to operate highly available, scalable, and cost-efficient applications with low operational burden by handling and improving the reliability and resiliency of SRE-managed services and infrastructure. You thrive on automation, infrastructure-as-code, reliability engineering, and getting rid of tedious, manual tasks.

You will:

  • Set technical direction, author design docs, and get consensus from internal and external partners.
  • Develop new processes to make the team more efficient and effective.
  • Collaborate with other team leaders to orchestrate large system changes.
  • Spend a significant amount of time on technical leadership activities in addition to hands-on technical work
  • Design new services, tools, and monitoring to be implemented by the entire team.
  • Analyze the tradeoffs of the proposed design and make recommendations based on these tradeoffs.
  • Mentor new engineers to achieve more than they thought possible. You enjoy making other teams successful and are fulfilled through the success of others.

Work on reliability projects, including:

  • HA, Business Continuity Planning, disaster recovery, backup/restore, RTO, RPO
  • Chaos engineering
  • Application uptime and performance
  • Capacity management & planning
  • SLIs, SLOs, error budgets, and monitoring dashboards
  • Responsible for deployment and operations of large-scale distributed data stores and streaming services
  • Establishing design patterns for monitoring and benchmarking
  • Establishing and documenting production run books and guidelines for developers
  • Tooling, toil reduction, runbooks & automation to handle production environments
  • Incident management and improving MTTD/MTTR for services
  • Cloud cost optimization

Qualifications

Must-Have

  • 7+ years of SRE experience in handling large-scale cloud-native microservices platforms.
  • 3+ years of strong hands-on experience deploying, handling, and monitoring large-scale Kubernetes clusters in the public cloud specifically AWS or GCP
  • Experience with infrastructure automation and scripting using Python and/or bash scripting.
  • Strong hands-on experience in monitoring tools such as Splunk, Prometheus, Grafana, ELK stack, etc. in order to build observability for large-scale microservices deployments.
  • Experience with deployment, operations and performance management of one or more of the following large-scale clusters such as Cassandra, Kafka, Elastic Search, MongoDB, ZooKeeper, Redis, etc.
  • Experience leading large-scale technical initiatives across multiple teams.
  • Excellent problem-solving, triaging, and debugging skills in large-scale distributed systems

Preferred

  • AWS Solutions Architect certification preferred.
  • Confluent Certified Administrator for Apache Kafka and/or Apache Cassandra Administrator Associate certifications are preferred
  • Experience with Infrastructure-as-Code using Terraform, CloudFormation, Google Deployment Manager, Pulumi, Packer, ARM, etc.
  • Experience with CI/CD frameworks and Pipeline-as-Code such as Jenkins, Spinnaker, Gitlab, Argo, Artifactory, etc.
  • Experience with one or more security/compliance frameworks such as SOC2, PCI, and/or FedRAMP.
  • Proven skills to effectively work across teams and functions to influence the design, operations, and deployment of highly available software.

Bachelors/Masters in Computer Science, Engineering, or related technical field, or equivalent practical experience.

We value diversity, equity, and inclusion at Splunk and are an equal employment opportunity employer. Qualified applicants receive consideration for employment without regard to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition, genetic information, veteran status, or any other consideration made unlawful by federal, state, or local laws. We consider qualified applicants with criminal histories, consistent with legal requirements.

 

 

Note:

Base Pay Range

India

Base Pay: INR 4,800,000.00 - 6,600,000.00 per year

Splunk provides flexibility and choice in the working arrangement for most roles, including remote and/or in-office roles. We have a market-based pay structure which varies by location. Please note that the base pay range is a guideline and for candidates who receive an offer, the base pay will vary based on factors such as work location as set out above, as well as the knowledge, skills and experience of the candidate. In addition to base pay, this role is eligible for incentive compensation and may be eligible for equity or long-term cash awards.

Benefits are an important part of Splunk's Total Rewards package. This role is eligible for a comprehensive, competitive benefits package which may include healthcare and retirement plans, paid time off, wellbeing expense reimbursement, and much more! Learn more about our comprehensive benefits and wellbeing offering at https://splunkbenefits.com.



Splunk's Hiring Practices
Splunk turns machine data into answers. Organizations use market-leading Splunk solutions with machine learning to solve their toughest IT, Internet of Things and security challenges.

We value diversity, equity, and inclusion at Splunk and are an equal employment opportunity employer. Qualified applicants receive consideration for employment without regard to race, religion, color, national origin, ancestry, sex, gender, gender identity, gender expression, sexual orientation, marital status, age, physical or mental disability or medical condition, genetic information, veteran status, or any other consideration made unlawful by federal, state, or local laws. We consider qualified applicants with criminal histories, consistent with legal requirements. Click here to review the US Department of Labor’s EEO is The Law notice. Please click here to review Splunk’s Equal Employment Opportunity Policy Statement. If you need assistance or an accommodation to apply or during the hiring process, please let us know by completing our Accommodation Request form.

Splunk also has policies in place to protect the personal information candidates disclose to us as part of the application process. Please click here to review Splunk’s Career Site Privacy Policy.

Splunk does not discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Please click here to review Splunk’s Pay Transparency Nondiscrimination Provision.

Splunk is committed to the health and safety of our employees and customers. We comply with local, state/territory, and federal regulations to prevent the spread of COVID-19 in the countries in which we operate. Splunk provides reasonable medical, religious, or other legally required accommodations for eligible employees.

Splunk is also committed to providing access to all individuals who are seeking information from our website. Any individual using assistive technology (such as a screen reader, Braille reader, etc.) who experiences difficulty accessing information on any part of Splunk’s website should send comments to accessiblecareers@splunk.com. Please include the nature of the accessibility problem and your e-mail or contact address. If the accessibility problem involves a particular page, the message should include the URL of that page.

Splunk doesn't accept unsolicited agency resumes and won't pay fees to any third-party agency or firm that doesn't have a signed agreement with Splunk.

DIVE DEEPER

Find out what makes Splunk such a great place to work

box1 box1
Our Values

Splunkers are encouraged and empowered to be Innovative, passionate, disruptive, open and fun.

Learn More
box2 box2
Benefits and Wellbeing

Our benefits are designed to support your physical, financial, emotional and mental wellbeing.

Explore Splunk Benefits
box3 box3
Early Talent Program

Intern with people you want to hang out with, even outside the office.

Learn More
box3 box3

Our Blog

Hear from Splunkers on the latest.

Read the Blog
box2 box2
Diversity, Equity, Inclusion & Belonging

Learn about Splunk’s commitment to creating a culture of belonging.

See Our Approach
box1 box1
LinkedIn

Follow Splunk on LinkedIn for job announcements, company news, and more.

Follow Us