Observability - From Fire Fighting to Smoke Detection

How the nightmare of losing weeks of data transformed our teams way of working

With the benefits of continuous delivery and microservices comes the challenge of distributed systems. We can no longer just deploy into production and assume it works.

The Importance of Observability – From Fire Fighting to Smoke Detection

How the nightmare of losing weeks of data transformed our teams way of working

In the past we tried to test as much as possible up front and reacted with customer issues as they arose. Traditionally we didn’t have a focus on observability, now we have the ability to monitor and detect problems as soon as they occur in production.

With the benefits of continuous delivery and microservices comes the challenge of distributed systems. We can no longer just deploy into production and assume it works. We need to monitor in production and see how the customer uses our features.

In this story I’ll recount how our team went from a system of firefighting customer issues to a situation where we were able to detect issues in production before the customer was aware and resolve them quickly. How we implemented Monitoring, Logging and Alerting tools to help combat this. Now the whole team is focused on observability during design and planning instead of trying to retrofit monitoring tools as an afterthought once features have been deployed.

I’ll also discuss how I brought the learnings from this experience over to a brand new team. How as a tester I was able to influence the team into thinking about observability from the offset without having to go through the pain of getting hit with a major customer impacting issue first.


More Related Sessions


120-minute Workshop

Tuesday, June 07, 2022 10:30 a.m. – 12:30 p.m. River Equipment required

45-minute Keynote

Wednesday, June 08, 2022 1:30 p.m. – 2:15 p.m. Grand Ballroom 1-4

45-minute Keynote

Thursday, June 09, 2022 1:30 p.m. – 2:15 p.m. Grand Ballroom 1-4

Combo-Session: 30-minute Talk & 75-minute Workshop

Thursday, June 09, 2022 2:30 p.m. – 4:45 p.m. Chicago Ballroom G