Why We Over-Engineer Software (and How to Break the Habit)

<p>With ready access to public cloud computing, container orchestrators, and microservices architecture, it has become trivial to create distributed systems of nearly limitless scale and complexity. While all of these tools have their purpose, it&rsquo;s important for engineers to carefully consider when and whether to use them, especially in smaller organizations. Making the wrong choices can make you less agile, less financially sound, and less successful. This article explores some potential causes and proposes some remedies for the common anti-pattern of unnecessary complexity.</p> <h1>The Problem</h1> <h2>We Begin With Complexity</h2> <p>When we interview for software engineering jobs, we must navigate a synthetic, challenging, and often stressful interview process to prove to potential employers that we are qualified to continue doing the things we&rsquo;ve already been doing for the past several years. During this process, we may be asked to solve multiple&nbsp;medium&nbsp;or&nbsp;hard&nbsp;algorithmic questions in a short time in what feels more like a spectator sport than coding. We are usually required to design a novel system in under an hour whose requirements we must first extract from the interviewer. None of these exercises is much like what we do in our actual jobs, but are instead challenges designed to produce varying degrees of &ldquo;signal&rdquo; that hiring committees can use to level and differentiate candidates.</p> <p>The&nbsp;system design interview, in particular, is a construct where candidates are asked to demonstrate wide and deep understanding of systems at a scale you might find in the very largest companies. We use concepts like&nbsp;microservices,&nbsp;consistent hashing,&nbsp;event busses,&nbsp;service mesh,&nbsp;protocol buffers,&nbsp;WebSockets,&nbsp;Kubernetes,&nbsp;API gateways,&nbsp;data pipelines,&nbsp;data lakes, and other technology buzzwords to show that we know what they are and how they are used. This gives potential employers confidence that we are up to date on all of the latest techniques and tools and can do what might be asked of us.</p> <p><a href="https://betterprogramming.pub/why-we-over-engineer-software-and-how-to-break-the-habit-72a32b62e3e2">Visit Now</a></p>