Escaping the Bikeshed

I wrote in 2017 a post called Don't Trap Your Clients in the Bikeshed. That post was about avoiding the trap of seeking feedback on trivial decisions from clients before they're necessary.

Sometimes in software development a group of stakeholders will become very suddenly interested in a particular aspect of the software, and intense debate will ensue with a flurry of changes discussed in an area that everyone has an opinion about. In these occasions, as an engineer, your clients have put you in the bikeshed.

What do you do as an engineer when you find yourself in the bikeshed?

Slow Down

The pressure to immediately address every opinion can be daunting. Try not to get caught up in the whirlwind. Let the group debate while you maintain a healthy distance.

Pause for Documentation

Gently remind people that in order to get features into software, they need to write down clearly what they want. They have a responsibility to be clear about what they're asking for. That's the bargain.

Invoke the Process

QAs need to know what to test. A larger audience needs to know what changes are making it into the software. Customers need to know what's on the horizon. Releases need to be organized. Changes in one part of the codebase impact other areas.

Your engineering team has an established process for making and releasing your software. In the fervor to change the bikeshed, interested parties get excited to see their opinions realized. How long does it take to paint the thing blue? It's clearly not blue right now, and I want it to be blue.

Remind people why the process exists, and the drawbacks of making changes to a software system that don't follow the same process as other changes.

Expose the Bones

Sometimes it's really an issue of transparency. Non-engineers get frustrated that they don't understand why a feature is behaving the way it is. It can help to "expose the bones" as it were. Make a report that anyone can see that shows key metrics (how many requests, how long is it taking, who's using it, etc.). It can help to surface the internals of a feature. Make a page that shows diagnostic information about the input to a feature, how the feature calculated a result, and what the "raw" result is.

Remove Engineering From the Loop

Ultimately the goal is to remove engineering from the debate. If there's an intense debate about the color of the bikeshed, engineering can build a configuration option into the software where non-engineers can change the color at will. Try to distill the points of contention down to self-service features that don't require engineers to make code changes to the system.

Make Your Thing Work Like the Thing Everyone Knows

Users build up expectations about how applications should work based on the applications they're already familiar with.

If you're including a search box in your application, it should work as much as possible like Google. Even if you have some clever idea about how it should work, if that way makes it act differently from Google, that's probably enough of a reason alone not to do it. We'll make the terms they typed in automatically match an exact phrase because our users would probably like that! Nope! They won't because Google doesn't do that. 

How will we let users know that there's fresh content for them to see under a tab of our application? Hmmm...got it! Let's underline the tab title! Wrong! You put a little dot to the side of the tab title, because that's what every app your users are familiar with does.

An invaluable skill for product owners, product managers, business analysts, etc. is that they are widely familiar with popular software and applications. And more importantly they are able to map requirements for the product they manage onto the conventions of other existing software.

Please, by all means, if you're a startup building a general purpose search engine from scratch, go ahead and wildly violate the conventions of Google search. That's literally why you exist. But if you're building a search box for your forklift parts website, make it work as much like Google search as you possibly can.

If you're building a feature for your accounting application where a user can send another user a message, this is not the time to innovate. Just make it work as much like Slack, Instagram, Twitter DMs, etc. 

I feel like well-meaning product managers sometimes misunderstand what innovation means in software. Innovation in the realm of what I would call commodity features is not good for your users. Features that are present in many applications are not the place to innovate unless that feature is at the heart of the value proposition of your application--if your product is that feature.

Wow...if you're violating users' expectations about a commodity feature in your application, you better have a damn good reason. Everyone else: make your thing work like the thing everyone already knows.

Tell Us Why We’re Doing This

I’m constantly shocked that business people don’t make much of an effort to communicate to engineers the impact of their work. It’s par for the course that the engineers don’t know how many users their product has, how many clients they have, how much revenue the product makes, etc. It’s so common in fact that I wonder sometimes if it’s intentional.

There is an Agile concept known as the information radiator. Some companies will put dedicated big screen TVs throughout their offices that show key metrics at a glance. If you have a remote-first culture, a television isn't going to do much good, but a web-based dashboard placed somewhere the whole team goes every day (like your issue tracker) is a great substitute. 

These ideas are more about passive awareness, but I think the next step up is active discussion. If you're doing some kind of regular all-hands meetings, like a retrospective, that's a perfect time to pull up that dashboard as a team and discuss it together.

How do we know we did a good job over the last sprint? Yeah, we marked all of our tickets as "Done", but so what? How do we know that our customers are happy with our work? Company leaders always want to know how to get their employees more "engaged" in their work. Show your team how their day-to-day tasks impact real end users. Working a backlog of items sprint after sprint is so far removed from the impact of the work. Why are we doing these things?

How many users did we add over the last two weeks? How many new customers came on board? How many usages of [NEW FEATURE X] did we have? 

I feel like I'll be banging this drum for the rest of my career: Engineers are not robots. We want to know the high level goals of our work and how our work impacts real people.

Please, tell us why we're doing this.

Slow Is a Superpower

Every company needs people who can work quickly. Stuff happens. Production goes down. We found a showstopper bug right before a big release. So-and-so called in sick--can you finish their thing that was due today?

Some engineers distinguish themselves by how much chaos you can throw at them. The plate-spinners. The late night heroes. Someone has to save the day. 

But it's also possible to distinguish oneself--and build a reputation--as a slow and methodical engineer. Someone needs to take really deep, nasty problems, and figure them out once and for all.

Someone needs to do the work where the attention to detail required is so tedious and annoying to mere mortals, that only a select few are steadfast enough to see it through to completion.

After five different engineers have spent a couple hours each on that bug, and only emerging with theories about what might be wrong, someone needs to spend a week going all the way to the bottom of the rabbit hole, and emerging with the rabbit in hand.

After generations of engineers have struggled to set up Project X locally on their machines, relying on hearsay and ancient scrolls to get to barely functional, and then moving on and never thinking about it again, we need a hero who goes through the process from scratch, writes down every damn thing that goes wrong, every caveat, every blind alley, records the verbal legends, takes screenshots marked up with the important bits circled, and meticulously documents in a format so easy-to-understand and beautiful, that the next generation of engineers will never again waste another moment setting up Product X in a breezy afternoon with all of their questions anticipated and answered before they even have a chance to ask them.

Who will think through the edge cases, draw the diagrams, note the long-term implications, ask the hard questions (and answer them), write the comments, edit for clarity, and study every changed line in a 57-file merge request?

Slow is a superpower. Not everyone can do it.

The Experience to Say "I Don't Know"

One of the difficulties with managing a software project is simply getting people to be honest about the progress they're making and the difficulties they're facing.

I think one of the signs of experience in engineers is that they're not afraid to admit when they don't know something or are having a hard time with a bit of work. I know that when I was a junior engineer it was hard for me to feel okay about not doing something perfectly, and I didn't want anyone to see me struggle with anything.

There's a sense of security that comes with experience. You know enough to know that you're never going to know everything. Just surviving in the industry as a working software engineer for several years, experiencing the inevitable ups and downs, and then simply continuing on. You realize that feeling out of your depth is a common feeling in a line of work that is constantly changing.

So I think it's important to encourage engineers and other technical folks, especially more junior folks, to raise their hand high early when they're bogging down. For most teams that are using an Agile methodology, any kind of daily stand-up is a great time to check in on progress. But of course, we can't help each other out if we aren't honest with each other about our progress.

Say these things as early as possible:

• I have too much work assigned to me this sprint, and I don't feel confident I can get it all done on time.
• I know less about this area of the project than Person X, and I think it will take me longer to do it than they would take.
• I've run into some unforeseen difficulty with Feature X, so I think it will be necessary to wrap up the part now that I understand, and tackle X again in a future sprint.

It's okay to say, "I don't know." It's okay to say, "I'm struggling." We should all say them more often! In fact, the health of a software project depends on it.

Contextualizing the 1-on-1

It's pretty common for engineering managers to have regularly scheduled 1-on-1 meetings with their engineers. Topics often include things like goal setting, problems the engineer needs help with, or a chance to air grievances that the engineer doesn't feel comfortable bringing up at other times.

Personally, I don't find myself coming to 1-on-1s with topics that I was holding onto specifically for that meeting. If I have an issue that needs attention, I'm not going to wait for a monthly meeting to bring it up.

My favorite topic for a 1-on-1 is everything that is happening at the scope that is beyond our day-to-day interactions. How is hiring going? So-and-so left 3 months ago--are we hiring a replacement?

What did you think about that announcement from the town hall? How is the work of our team being perceived by the broader organization? How is customer X liking feature Y that we launched recently?

How do you feel the team is executing toward our strategic goals for this quarter?

I want to "take the temperature" on a thermometer that I don't get to see often--get the context that I'm not getting on a day-to-day basis.

1-on-1s can be anything. They don't have to be a one-way venting session. They can just as easily go entirely the opposite way where we talk mainly about higher level things.

Clean Pull Requests

Pull requests can be a pain to review, but there are things submitters can do to make them easier to manage. I have some opinions about what makes a pull request "clean".

Describe What You Did

First off, pull requests typically have some kind of description field that can by default be left blank. The description is super important! Use the description field on the pull request to describe the theme of the PR. Obviously, it's ideal to have it linked to a ticket in your issue tracker.

I'm a fan of opportunistic refactoring, which I'd describe as cleaning up a bit when you're in the codebase doing something unrelated. This can get gnarly, though, in that when it comes time to submit a PR, your PR is intermingling different contexts in a way that is confusing for reviewers. In these cases, you can certainly split the work across two or more different PRs, but that can be a pain if you've already pushed a bunch of intermingled commits to one branch. In those cases, I think it's fine to keep everything in one PR, but it's incumbent upon the submitter to explain where the ticket ends and the "cleaning up" begins.

Every Line Matters

There's nothing worse than going to review someone's PR and it feels like someone dumped a "pile of changes" on you with little or no explanation to why they were made. I might be particularly fussy, but for my own PRs, I won't submit them for review until I've looked at every single line of code that I changed, and ensured that I could explain why I made it if asked. And it's not because I expect to be asked about every line I changed--in fact that would be kind of annoying, but because I often discover something that I didn't mean to change, or I had an idea for a quick refactoring like a variable rename to make the code clearer, but I forgot to do it.

One of my pet peeves as a reviewer is when a PR comes in that touches several files, but some of the files are only changed by a trivial whitespace change--like someone had been working in a file and then backed their changes out manually, resulting in maybe just an extra line break left in the file. And it's not so much that now the change list of files in the PR includes an extra file I have to look at, but because it makes me nervous that the submitter didn't pay close attention to what they had changed in the codebase before submitting the PR. It makes me wonder what other changes they included that they didn't intend to introduce into the production codebase.

It's so easy to leave in fragments of various code experiments from when you were in the phase of figuring out an approach, and then not realizing that those fragments are about to escape your feature branch and enter the official history of our production codebase.

The Clean Pull Request™

If I could sum up what I want from submitters in a Clean Pull Request™ it would be:

1. Make sure to examine each line changed in each file and ensure it's something you want to include in the production codebase.
2. Provide context for all changes included, whether that is via a linked issue/ticket that explains the task at hand, or via a written description on the PR itself that provides the context.

Keep it clean, and we'll get through code reviews faster with fewer mistakes.

Lunch & Learns

A lot of companies have a regular practice of "Lunch & Learns"—a series where engineers within the company give a presentation to their coworkers on some technical topic in which they have an interest.

I myself have given presentations at several companies where they did this, and have sat through many presentations by other engineers. Over time, I've developed a taste for what works and what doesn't.

Set the Stage

Start with the basics: Who are you? What do you work on in the company? Why is this topic important to you?

I've watched too many of these where someone starts the presentation by screen-sharing what seems like a random code file in Visual Studio (or their code editor of choice), and just starts talking in minute detail about lines of code in the file. No context at all for: what am I looking at, what is this, why is this significant, what is this application, what does this application do, why would I care, what am I supposed to do with this information, etc.

Go Beyond Generic Content

I've sat through too many "Introduction to X" presentations. They always have generic names like "React.js" or "Test-Driven Development". Now, there's nothing wrong with introductory content, but here's the thing: the audience knows how to use YouTube. If they want a generic introduction on some technical topic, there are undoubtedly free videos available that are more authoritative and higher quality.

If the content of the lunch & learn is completely generic in a way that it could be given to any group at any company without context, then there's really no good reason to be gathering a captive audience at one particular company to watch it.

Take advantage of the unique experience you have with the topic as it pertains to the projects and domain of the internal audience. Make the generic specific.

Make It Practical

What value is your audience going to get out of this presentation? Here are some lunch & learns I like to see:

• This Is How We Used X to Do Y in Application Z
• How to Replace X in Your Project with Y
• Postmortems / Lessons Learned from Project X

One of the best tech leads I've ever worked with once gave me some advice about technical presentations that I've never forgotten. He encouraged me to think about "presentation to Production". It's one thing to give an interesting presentation, or pick an interesting topic, but the next level is to think about "How are we going to put this technique/language/framework/practice into Production?"

Include the Thing

I literally wrote a blog post a couple years ago called Include the Thing about effective written communication for engineers. And it's advice that is totally applicable to presentations.

Look, no one wants to watch a presentation where someone is showing a PowerPoint deck and simply reading the slides back to you. But I think some people, especially engineers, go too far in the other direction and eschew any kind of written outline material for a lunch & learn.

It's great to "show the code", but it's next level to leave a written artifact (PowerPoint deck, Wiki page, etc.) with links to the specific internal code repositories (GitLab, Bitbucket, whatever) and individual files that you were looking at in your local code editor during your screen-share.

For lunch & learns at my current company, we do the trifecta: 1) wiki page including… 2) embedded PowerPoint deck and… 3) embedded screencast video of the presentation. Anyone coming along later who couldn't attend the lunch & learn live, or anyone who wants to refer back to the content has all the context they need.

The Yin and Yang of Technical Personalities

Categorize this post as "advice to my younger self." As a developer early in my career I would get very annoyed when coworkers and people in management at my various jobs in the software industry were not approaching the creation of software in the way that I did. Or did not have the same philosophy as mine about how this work should be done.

I'd assume that people who didn't think like me were doing things the "wrong" way, so they must be less talented, less thoughtful, less conscientious. They must not have read the books that I had read, didn't read blogs about the industry, and must not "care" like I did.

It took many experiences across different jobs of working with people who I respected, and who I could tell were very smart, but did not work the way I did, for me to finally understand that there was no "right" way to do things.

There are spectrums of personality traits in the software industry that can be summarized as a pair of opposites. Some of them might be:

Dreamers ↔ Pragmatists

Some like to think/talk about an ideal vision of a system, and some like to think/talk about the system as it is today.

Big Picture ↔ Detail-Oriented

Some take a high level view, some take a low level view. Some like to zoom out, some like to zoom in.

Move Fast & Break Things ↔ Slow & Methodical

Some value speed of change, and some value steady progress and stability.

Optimists ↔ Pessimists

Some proceed as if the best case will happen, some proceed as if the worst case will happen.

Answerers ↔ Questioners

There are so many ways to do anything in software. There are flaws in any solution, and people need to be watching for them and raising them. But at the end of the day, someone has to be willing to pick a direction and go with it.

I've tended to be toward the right side of these spectra, and would get very frustrated with people who were on the other side. What it took me years to understand was that you cannot have a successful software project where everyone is on one side or everyone is on the other side. Like so many areas of life, balance and diversity are essential for a good outcome. 

The dreamers balance the pragmatists. The visionaries need boots on the ground. The speedsters need people to check their speed. Forward progress requires leaps of optimism, even when there are a million things that could go wrong. 

To the younger me I'd like to say: different kinds of people exist for a reason, there is no "best" way to accomplish difficult goals, and you don't know everything.

Let Unsustainable Things Fail

I read a post by Max Countryman recently titled Let It Fail (Hacker News discussion), where he talks about the importance of failure in software engineering organizations. He recounts an experience he had where his boss gave him surprising advice to let a legacy system that was not getting enough attention fail visibly rather than paper over the cracks.

I've written before on this blog about how heroics cover up systemic issues. Organizations become dependent on heroics. Heroics beget more heroics. Heroics are normalized. Eventually the heroes burn out.

That one long-serving, loyal employee who had become a silo of critical knowledge up and quit. That manual, error-prone process we had been using to deploy our software for way too long finally caused an outage. We've been rushing code changes right before big releases, and one finally bit us hard.

It's important to talk openly about unsustainable engineering practices and the failure they portend. But issues of technical debt are hard to understand when placed alongside new feature development and other business priorities. The value of under-the-covers work is hard to understand. Sometimes you need to pull the covers away.

Tech's COVID Hangover

So many layoffs! I can't open Hacker News or LinkedIn without seeing constant stories about the next tech company to lay off hundreds or thousands of employees. For a couple years there, the tech industry was hotter than I had ever experienced in my career.

It never quite made sense to me, but I was riding the wave. Well, that wave has now crashed, and I'm left wondering what happened.

Ben Thompson of the Stratechery blog, a frequent guest on the front page of Hacker News, wrote recently about The Four Horsemen of the Tech Recession, and I found the analysis pretty interesting.

In particular, his discussion of the "COVID hangover" caught my attention. Essentially, demand for digital transformation in companies around the world suddenly sped up when the pandemic hit. Planned investments to modernize were all made at once, rather than the slow rollout that they envisioned before the world suddenly changed. Growth in the tech sector is now slowing as the demand was "pulled forward." Thompson explains…

When corporations the world over were forced literally overnight to transition to an entirely new way of working they needed to scale their server capabilities immediately: that was only realistically possible using cloud computing. This in turn likely accelerated investments that companies were planning on making in cloud computing at some point in the future. Now, some aspect of this investment was certainly inefficient, which aligns with both Amazon and Microsoft attributing their cloud slowdowns to companies optimizing their spend; it’s fair to wonder, though, how much of the slowdown in growth is a function of pulling forward demand.

…the COVID pull-forward was massive, but underneath the inevitable hangover there was a meaningful long-term shift to digital broadly.

The layoffs and slowdown in hiring we're seeing now after the massive boom in hiring during the pandemic is…

precisely because these [only-possible-on-the-Internet] jobs — and similarly, many of the COVID-specific workloads like work-from-home and e-commerce — were digital that it is tech that is in a mini-recession even as the so-called “real” economy is doing better than ever.

The level of growth in the tech industry that kicked off during the pandemic was not sustainable, but if it represented a pulling forward of pent-up demand, then it would make sense that there would be a slowdown in spending to follow. Hopefully after the hangover we get back to a healthy level of demand, that while not at the highest of highs, will be sustainable.

The AI Can't Destroy Anything Worth Preserving

The front page of Hacker News has been flooded lately with articles about ChatGPT, a seemingly magical AI tool that can generate realistic paragraphs of text that can seem indistinguishable from human written text.

My favorite article on the topic is ChatGPT Can't Kill Anything Worth Preserving by John Warner, a former English teacher. In that article, Warner is talking specifically about the prospect of students using AI to cheat on writing assignments, as it seems that ChatGPT can often fool the people grading them. He points out that if writing assignments are so tedious, boring, and formulaic that students aren't interested in doing the exercise themselves, and if an AI can write a passible solution, then we have to conclude that the exercise itself is absurd and not worth asking students to do. The AI is just a more convenient and efficient form of cheating than has been available to bored students for years.

Before ChatGPT was the hot topic on Hacker News, there were daily articles about GitHub Copilot and DALL-E 2. 

I've written before on this blog about the disturbing effect of automation in areas of work that one holds dear. In my post Be the Automator, I described the experience of discovering as a junior software engineer that there were tools called "code generators" that could automatically write code for you. I felt an existential threat at the time, but learned over the following years that not having to write tedious and formulaic code myself is a blessing rather than a curse.

My general feeling for many years has been that, fundamentally, the task of automation is about human liberation. It's demeaning to make a human being do something that they don't want to do if a machine can do the job just as well (or better). If the code I'm writing is so trivial that an AI assistant like Copilot can write it for me, then by all means let the machine do it! If English students are writing prose so trivial that ChatGPT can do it for them, then let the machine do it, and give them something better to do.

To paraphrase the title of Warner's article, the AI can't destroy anything worth preserving. If a machine can do it, a machine should do it. We might learn some hard lessons along the way that we weren't as creative as we thought.