Our Allergy to Problems, or How to Easily Over-Engineer Anything
It’s rare to have the opportunity to “design blue sky” or, work on solving a new problem “from scratch”, but those opportunities do come up, more often than one would expect. Whether it’s to refactor some legacy module, or an entire system that’s seen many hands over the years, or writing an entire application from the ground-up. It’s easy to fall into the temptation of wanting to make sure we never have any problems ever again!
So you sit down, you mull over the problems that the existing system has (usually inflexibility, bloat, FUD, etc.) and you make sure you solve for each and every one of those issues. The problem with this is that this is the best way to over-engineer anything.
In general, I’ve found that many engineers (myself included) have developed a full-on allergy to problems. As engineers, its more or less in our nature to want to foresee issues with our particular implementation and to build in solutions to those problems ahead of time. Here’s some reasons why this may not be the best approach.
You don’t know what the most important problems are before you have them
First of all, no matter how convinced you may be, you can never know for 100% certainty what problems you will have in the future.
Secondly, while you can usually guess with some high level of confidence the gamut of problems that your system might have, what you definitely can’t know for certain is which problem will be the most severe or the most crippling.
The order of the problems you solve matters as much as the problems themselves
In general, solving the most severe problem will drastically change the dynamic of all the other problems in your system. For example, imagine you have a multi-device chat application and you want message read status to be updated across devices. Depending on how you choose to solve this problem will most likely affect how you solve the smaller problem of marking messages as read as the user scrolls them into view.
Design your system so that you have problems
There’s a reason why the MVP approach to product design has garnered so much attention. It specifically dictates that you don’t design a product that solves all problems for all people (because you’ll end up with a product that solves no problems for no one). This same approach should be taken for software engineering (architecture and design).
An allergy to problems
As software engineers, we’re very allergic to having problems. If we can foresee an issue, we’re sure to start thinking of a solution to it. I urge you, however, to allow yourself (or your system, your process, etc) to actually have that problem. Because experiencing the problem first-hand crystalizes what a proper (and simple!) solution would look like. Rather, if you try to short-circuit this experience by solving for the problem before you have it, you’re sure to end up with over-engineered solutions to a whole slew of problems that you don’t really have.
