"Environmental Issue"

I really like a post I read recently from Paul Osman called Production Oriented Development.

He touches on a lot of topics near and dear to my heart, but the section on non-production environments stood out:

Environments like staging or pre-prod are a lie. When you’re starting, they make a little sense, but as you grow, changes happen more frequently and you experience drift. Also, by definition, your non-prod environments aren’t getting traffic, which makes them fundamentally different. The amount of effort required to maintain non-prod environments grows very quickly. You’ll never prioritize work on non-prod like you will on prod, because customers don’t directly touch non-prod. Eventually, you’ll be scrambling to keep this popsicle sticks and duct tape environment up and running so you can test changes in it, lying to yourself, pretending it bears any resemblance to production.

I've heard the heartbreaking phrase "environmental issue" too many times in my career. You know, like when a developer spends days investigating a bug reported by QA only for the team to decide it was due to a configuration drift between a lower environment and production.

With modern infrastructure-as-code tools like Puppet and Terraform, at least there's a chance of preventing the pitfalls with using non-production environments. Even then, only prod is prod. ¯\_(ツ)_/¯