Visualising Production Data For The Team – Retrospective 2

So progress on this mini project has faltered a bit – hence the reason I haven’t been going on about it.

My use of Splunk has diminished after I found out that we don’t send the Apache logs to Splunk. Conversations about this are still ongoing & not just from me.

Also, I’m still struggling to trace a transaction from top to bottom & back again. Its difficult to get a true reflection of the Customer experience other than use the site myself.

We now have a bank of monitors in the office showing a plethora of statistics, including server monitoring, Gomez stats, DB performance & Google Analytics. Unfortunately these are at the other end of the office from me, so I’m not using them as much as I’d like.

We had great fun visualising customer behaviour on Cheltenham & Aintree race days (2 of the biggest in the horse racing calendar)

We used glTail – it was great to see the difference in the types & amount of requests before during & after each race.

Watching Google Analytics after the race was exciting to see how high the numbers would go as punters hit the site, checking their balance to see if their bet had settled. It was almost as exciting as watching the people who were watching the numbers…

With regards to New Relic – I’ve been using it locally when we are releasing to Production. I can see as each box is removed from the pool ready for the new release. Its great to be checking the release before you get the nod the release to each box is complete.

The New Relic graph I use during a release to Production is now being used by the release manager – she finds it most useful as well!

Exciting news (this post has been in development for a while…) We are using Graylog2 for our Apache logs which really resurrected my interest in using monitoring tools to help when we release a new version of the website into Production & hopefully free up my time to test further upstream where I can add more value.

Next step – choose the graphs, dials & stats from Splunk, New Relic & Graylog2 to create the dashboard which will help the team understand the customers experience & the behaviour of the website in real-time.