Mark43 and SignalFx: Monitoring for When Real-Time Means Real Lives

Mark43 provides a cloud-native platform for dispatch, records, evidence, and analytics. Its software allows law enforcement officials to perform their critical jobs efficiently and seamlessly, so they can focus more on the communities they serve.

The criticality of real time is nothing new to Kevin Heins, DevOps Technical Lead at Mark43. When your company builds software that improves public safety, downtime is a deal breaker—both for your customers and in the tools that keep your systems available and performant. I recently had the chance to sit down with Kevin to talk about his important role and how his team helps Mark43 achieve its mission of empowering communities and their governments with new technologies that improve the safety and quality of life for all.

Can you tell us about your role at Mark43?
I’m the technical lead on the infrastructure team at Mark43. As we bring the benefits of the cloud to our customers, my team’s primary responsibility is to ensure our applications are running every minute, all day, every day so officers in the field have continuous access to the tools they need in real time. We also handle the rolling out of our infrastructure, deployment pipelines, and our code build.

Aside from the cloud, what other architectural investments have you made to best serve your customers?
We’ve designed each of our products as a set of microservices, so we can easily scale up and down to meet the needs of our customers. This means that as more police officers log in and more requests come through an application, we get an increased number of hosts servicing the application, so it stays performant for those officers.

How were you monitoring your environment previously?
We had two monitoring tools we were using before SignalFx came on board, neither of which met the high standards we required for our customers. One of the tools had consistent stability issues and the other was prohibitively expensive without meeting all of our monitoring needs. Across both tools, we saw a lot of unnecessary alerting noise. Alerts would misfire, waking the whole team up in the middle of the night when they didn’t have to be—not a recipe for happy engineers!

Having multiple tools also fragmented our metrics, so engineers didn’t always know which platform to go to when there was a problem. This slowed time to resolution and ate up valuable development time.

What are some of the results you’ve seen since deploying SignalFx?
SignalFx has allowed us to funnel all of our metrics into one platform, giving the team a single pane of glass for monitoring the health of all of our services. Since we started using SignalFx, we’re alerted to issues in seconds compared to a 10-minute latency with our previous solution. This allows us to get in and fix issues before they have a downstream effect for the officers using the platform.

Additionally, with SignalFx Microservices APM™, we can see all the traces and spans for an API call and fix underperforming APIs, which helps us improve performance and reduce the amount of stress on the production platform overall. Performance improvements that previously took a full team almost an hour can be done by one engineer in minutes.

"Real-time monitoring from SignalFx lets us support police officers by ensuring they don’t have to worry about their tools being down when they’re responding to a call in the field. It helps us keep safe the officers and the communities they serve."
- Kevin Heins, Devops Technical Lead, Mark43

What are the SignalFx features that have been most impactful for you and the team?
There are a few key features from SignalFx that provided immediate value, especially in improving how our developers spend their time:

  • The third party integrations from SignalFx work right out of the box. SignalFx supports the services we use—like Kafka, NGINX, and HAProxy—so we were able to spin up very quickly, get the visibility we needed, and start working on other issues. We don’t want developers toiling away configuring infrastructure or monitoring. The built-in third party integrations from SignalFx allowed us to set it up once and go.
  • SignalFx’s custom dashboards have allowed us to give each team at Mark43 a tailored view of the metrics most relevant to them. We have views that are per team and per app, so each development team can monitor and maintain their particular piece of the Mark43 platform, as well as dashboards with business KPIs displayed on monitors across the office. This real-time data provides our executives and the whole company with visibility into the performance of our applications.
  • Fast and accurate alerting was really important to the team—too many alerts and you’re not sure which ones are most important and require attention. With SignalFx, we’re able to fine tune the alerts and send them to the right team, based on which application or host the alert is coming from. We don’t want to have too many cooks in the kitchen, which can lead to diagnosing the wrong issue and wasted time. Smarter alerting from SignalFx helps us ensure the right alert goes to the right person so they can immediately begin to remediate the issue.

You’re using AWS GovCloud. How did this impact your decision?
We’re heavily invested in the cloud, and have deployed all of our microservices on AWS GovCloud, an isolated AWS region used for highly-sensitive workloads that require increased regulatory and compliance requirements. A lot of the monitoring solutions on the market don’t support integrations with GovCloud. SignalFx does and has been a great partner in monitoring GovCloud.

What’s next for your team?
We want to make sure we stay flexible. We’re evaluating technologies like containerization, Docker, and Kubernetes as well as extending our offering to other cloud providers based on customer demand. We want to make sure our monitoring solution doesn’t lock us in as our prior monitoring tools did. SignalFx doesn’t do that. They’ve been a great partner to us, which gives us the confidence that we can continue to grow and evolve our architecture.

What’s the most valuable insight you get from SignalFx?
The real-time analytics SignalFx provides on our platform have been invaluable. The insights they provide allow us to respond to issues as quickly as they appear, before a downstream effect impacts our customers. When an officer responds to a call in the field, lives are on the line. Seconds are critical. Ultimately, having real-time insight into our platform helps us ensure that our customers don’t have to worry about their tools being down. That helps to keep the officers and their communities safe, which is our ultimate goal at Mark43.

"SignalFx has allowed us to funnel all of our metrics into one platform, giving the team a single pane of glass for monitoring the health of our services in real time. This visibility allows us to get in and fix issues before they have a downstream effect for the officers using the Mark43 platform."
- Kevin Heins, Devops Technical Lead, Mark43

Nick Wood

Posted by



Mark43 and SignalFx: Monitoring for When Real-Time Means Real Lives

Show All Tags
Show Less Tags

Join the Discussion