The 3am problem
It was Friday evening, and I was on-call.
One of our central software components had a nasty bug. At random times, especially in the middle of the night, it would just stop consuming Pub/Sub messages. There was no fix ready, and the o...