EP2: Inefficiencies of Software Development

Why is software development so slow and costly?

There is always a stage during the lifetime of a software system where development efficiency decreases significantly. Everything feels slow and costly. Teams that previously delivered features within a couple of weeks start requesting months to complete newly prioritized features. Tension builds between business and development.

The typical management’s reaction is to introduce more process and red tape. Everyone wants to have a clearer picture of what is happening. Trust is eroded.

This story does not have a happy ending. It turns out all measures to address the problem end up amplifying it rather than solving it. Teams turn defensive and senior engineers start searching for greener pastures. 😢

Why does this happen?

Same job, different estimate

If you asked me how long I need to implement a typical CRUD API, I would tell you 1 to 4 hours. I can back it with evidence because I did this multiple times in my own free time. I like to challenge myself and some of the systems I build for fun are not trivial at all.

However, if you commissioned to me a similar job for which you would pay me a sum, I would tell you I need at least 3 days. I need to make sure everything is done in the most complete and elegant way possible. I will also invest more time in documenting the code, since I need to hand it off.

It might sound incredible but if you asked me to estimate how long it takes an entire agile team to perform the same job I would answer 1 sprint or 2 weeks!

How is that even possible?

The are multiple angles to this but one that is rarely explored is risk appetite.

Nothing ventured, nothing gained

When I work on a side project, I represent business, architecture, and development. The three things are perfectly aligned in one person: me! I am in charge, and I feel entitled to take calculated risks to get to the finish line as quickly as possible. As you can imagine, this maximizes my agility. Feedback loops between business and development happen in my own brain. No chance you can beat me! 💪

In the scenario I am commissioned the same job, I do not represent business anymore. I am not in the position of taking calculated risks because it is not my business on the line.

It becomes more complicated if I oversee a development team while employed full-time in a company. Now the company’s success is in my own interest. However, I might not be the person developing the feature requested. My team could also be dependent on other factors such as integration with other teams or infrastructure. There might be several layers between business and me that make it impractical or impossible to propose options to get to the finish line quicker.

What we can derive from this reasoning is that risk appetite is inversely proportional to the distance between business owners and developers. In simpler words, the larger the distance between business owners and developers, the lower is the appetite for risk.

That is a big problem in a hyper competitive market since we must take some risks to be able to beat the competition. We have seen this very recently in our industry when ChatGPT disrupted the AI market. Microsoft and Google had to cover the gap as quickly as they could. That meant they shipped products which were not tested as much as they typically do. However, getting quick to the market was imperative and they just steamed ahead with them.

I had to say it was really entertaining to see these two giants panicking about it 😂

Agile to the rescue?

There is widespread consensus that to improve agility in software development we need to create cross-functional teams that are responsible for a particular feature or area of your system. However, despite many businesses embraced this idea, not all of them have seen improvements in their delivery timelines.

Sometimes the improvement is temporary. There is an initial boost, and everything seems to be working great, but fast forward a few months and they are back to a crawling pace.

This scenario is very telling because usually what changed during those months is scale. They probably started with one or two teams trying out the new approach. Business owners are very present in the early stages by either participating in first person or by appointing key decision makers as product owners for those teams.

Eventually the approach is applied to the entire company and distance between business and development increases again by appointing product owners or managers with no real decision-making power.

Risk appetite plummets again! Everyone is more concerned about doing their portion of the job (allow me to say it… cover their 🍑), rather than getting to the market rapidly.

What do we do now?

Trust is the key

If the company scale allows it, owners and executives need to be more present with the development teams. They can even setup camp in the office like Elon Musk did at Twitter HQ (joking… don’t do that please🙏).

Otherwise, they need to effectively delegate their decisional power to those they elect to represent them. In a nutshell, they need to trust!

Agile is like a car and trust is its fuel

It doesn’t matter if you bought a shiny Ferrari capable of impressive top speeds. If you don’t fill it with gas and drive it, it’s going to collect dust in the garage.

Keep also in mind that the faster the car, the more fuel it consumes. Faster teams capable of delivering features at a very high pace will require even more trust from business to express their full potential. There is no point in hiring the best developers if their output is hindered by micromanagement and mistrust.

I know you are now feeling my advice is very banal and generic. Unfortunately, it’s the real key to faster delivery times. There is no way around it. You need to trust me! (see what I did there? 😂)

Joking aside, we had recently a marvelous example of the power of trust showcased by Meta and the delivery of their Threads app which grew to 100 million users in 4 days and 6 hours. Let me quote a portion of their article titled “Threads: The inside story of Meta’s newest social app”.

Threads was developed in an environment more akin to a startup. Creating a new app with such a small team meant assembling a group with a high level of trust – where everyone was aligned toward a singular goal and there was close alignment with our leadership, like the Head of Instagram, Adam Mosseri.

My job is done for today. Hope I gave you something to think about!

Till next time 👋

Latest Video

Don’t forget to have a look at my YouTube channel as well!

Last week we discussed some issues created by overusing APIs for intra-system integration in a microservices architecture.