How to Advance Your Software Architecture

Content by Software Testing News

How can you advance your software architecture? The first lines of CircleCI’s codebase were written nine years ago. Hindsight exposes some interesting themes that can be instructive in identifying approaches to dealing with change.

While they aren’t all international, it doesn’t make them any less valuable.

Three themes are: deferring the need to handle change, thinking like a product manager, and keeping your head up.

Change is a constant in technology

Change might be one of the only constants engineers will deal with throughout their careers. It certainly drives many of our most interesting challenges.

The market is changing, our business is evolving, our customer base is growing, and our team is scaling. We need to build a solution that meets today’s needs but sets us up for the demands of tomorrow. How do you design systems in a manner that can adapt and change to things that don’t even exist yet?

Outside of the youngest projects, it’s unlikely that you can describe your architecture with any simple one-liner. It’s more likely just “your architecture”. If you’ve been working on it for a significant time, you may have applied multiple patterns to solve specific problems. There’s probably a monolith somewhere, some number of microservices, a few events, and a serverless element or two. Additionally, you might be in the middle of transitioning some of those pieces from one pattern to another.

Search Vacancies

Design architecture to solve problems

As your product evolves, you should consider whether your needs are changing. You can use many approaches to design software and systems to be more resilient to change. Not having to do complex refactoring down the road leads us to want to create systems with low cost of change. On the other hand, while generalised abstractions and reusable components are at the core of reducing the impact of change, they are hard to get right in the first place.

It’s hard to predict the future

At the same time, the technical landscape is changing, and your business and your organisation are evolving. These both have a significant impact on your architectural decisions. The specific types of change that you are undergoing will influence the software architecture best suited to absorb those changes.

In the early days of a project or company, it’s possible to make sweeping fundamental shifts in the objective. Extreme, but not uncommon, examples include Tiny Speck becoming Slack and Odeo becoming Twitter.

Once you find product/market fit, your priorities will shift towards supporting your rapid growth. With system scale as a driver of change, the ability to respond to change ties to the independence of systems based on their operational characteristics. After the scaling phase, you’ll have a larger organisation and more teams. If you’re lucky, you’ll be back to sustainable product evolution. Then the cost of change will be proportional to the amount of cross-team coordination required to make a change,

Defer response to change

When an individual or team has a problem, they start with a simple directive – let’s build to solve that problem. What’s the most effective architecture we can use to create a viable product at the lowest cost? This is generally true whether it’s a startup or a huge multinational corporation. A new project will be built on a relatively new system. It will unlikely have a product-market fit because the company won’t fully understand its business domain yet. Additionally, they won’t know where the primary sources of change will come from.

In this situation, designing for change becomes a lot harder. If it’s more expensive to make things resilient to change, and you can’t tell which parts of your system will change, how do you decide where to make that investment?

As counterintuitive as it may sound, the answer is probably nowhere. At least not yet. Instead, watch change closely and build in a way that minimises the cost of being wrong.

In “97 Things Every Software Architect Should Know”, Kevlin Henney describes a critical approach to thinking about uncertainty:

“The presence of two options indicates that you need to consider uncertainty in the design. Use uncertainty as a driver to determine where you can defer commitment to details and where you can partition and abstract to reduce the significance of design decisions. If you hardwire the first thing that comes to mind, you’re more likely to be stuck with it – incidental decisions become significant, and the softness of the software hardens.”

This framing is excellent for explicit decisions, but what if you don’t know you’re making a choice?

Often, the choice isn’t visible yet but will reveal itself later. In these situations, the answer is not to overgeneralise, building abstractions everywhere just in case. Instead, keep things as simple as possible so you can understand them later if you have to make a change.

Simplicity makes it easier to adapt to change

The CircleCI application was a monolith that took a customer’s build with its data and pushed it into one of several LXC containers. Every container spun up was instantiated from the same image that contained everything anyone would want in their test environment. In hindsight, this sounds like a terrible idea, but at the time, it was fantastic. It was simple to maintain and support the needs of our early customers, many of whom were building Rails monoliths.

As time passed and the customer base grew, so did the diversity of their needs in test environments. As a result, upgraded versions of underlying databases, novel new development frameworks, and even new operating system versions became necessary.

The original container management in CircleCI wasn’t designed in a way that allowed us to adapt quickly to these changing needs. So when we set out to solve these problems, we knew where to splice in a new approach, and it was minimal work to enable that splicing. We also didn’t have to unravel a poor generalisation that didn’t support our new problem.

It’s important to note that while seemingly simple, we were five years into successful growth as a company meeting the needs of our customers on that simple system before our first customers tested its replacement. In those five years, Docker was created, as was HashiCorp’s Nomad. Those tools eliminated vast portions of the work necessary to get the flexible and scalable environments we support for customers today.

Also, as we retooled the system to adapt to the changing market, we were in a position to ask, “How do we do this in a way that better positions us for incremental change?” It would not be easy to overstate how much value five years of experience provide when designing a solution.

It sounds weird and grossly unfair to say this, but most of the time, we don’t even know that we’re making a monumental design decision because the alternate path hasn’t shown up yet. So how do you guard against this?

Defer, defer, defer

The correct solutions have a habit of revealing themselves if you can find a way to wait long enough. Technology gains traction or dies. If you chose a container orchestration engine in 2016, there’s a roughly 20% chance that you would have selected Kubernetes. By the beginning of 2018, almost 80% of companies were switching. Those are not great odds.

So deferring can be good. However, deferring to the point of creating a crisis is not.

Think like a product manager

Your architecture is a leaky abstraction. While CircleCI is probably an extreme example of this case due to our customers’ access to systems in our platform, there is always some implication for customers of the decisions you make in defining your architecture. Recognising this impact and discussing approaches and alternatives coherently is a considerable asset an engineer can bring to their PM and team.

While we were witnessing all of this change in how our customers were building software, one thing we ignored for too long was the rise of Docker. Docker didn’t even exist when CircleCI started in 2011. Then by 2014, it was everywhere. Many of our customers had started building Docker images as part of their build to prepare for deployment.

Docker was initially built on top of LXC, which meant in those early days, we could support the use of Docker commands to develop images and push them to repositories inside one of our LXC containers. In 2014, Docker launched something called libcontainer, and they pulled apart the access to the underlying system and created execution drivers, which soon led to the deprecation of the LXC driver. Disappointing, but it still worked. Then Docker deleted all LXC support.

Always think about users.

As an engineer, an architect, or a leader in technology, you need to think about the product’s direction and work with your product managers to ensure everyone understands the implications of technical choices.

It’s essential to separate this discussion from technology investments. There is a place for this type of investment, and an argument that framing these investments in the same way you prepare product investments makes it much easier to make tradeoff decisions. However, identifying the directly customer-visible feature impact of architectural choices differs from the more commonly discussed implications of cost, performance, security, etc.

Building a better understanding of the relationship between your architecture and the value achieved by your customers will put you in a position to make more informed decisions about what and how to build as your business evolves. Then you can focus on getting ahead of that evolution.

Keep your head up

In software, we have a terrible tendency to make the fundamentals harder than they should be by adopting new technology that we think will be game-changing but will have a modest upside, if any. The downside is often unbounded as we get our teams up the learning curve, find untested edge cases during production incidents, and invent our own “best practices.” A world where Stack Overflow has no answers.

A time simplicity would have helped

No company has ever won because of an amazingly novel or esoteric technology choice. However, there is an endless list of companies that have won because they can move quickly and with agility, adapting to change as it happens. Technology should be an accelerator helping us meet the needs of our customers. Well-understood, production-tested tools are far more likely to fit that bill. As my colleague, Bear says, “If a tool isn’t helping, it’s not a tool; it’s a chore. Drop it.”

Conclusion

Unforeseen change is top-of-mind for so many of us, and it’s more evident than ever that, in business, nobody can predict the future. So it’s a safe bet that folks at GM didn’t start the year thinking about how to use their factories to make ventilators.

Preparing to adapt in the face of change requires thinking about change as a driver in everything you do. Watch how your market is changing and reflect on how that impacts your architecture so that you can make targeted, incremental improvements in its adaptability. This will help keep you focused on evolution without wasting time or money on areas that won’t need it. Black Swan events are so unusual that studying the specifics can be of low value, but they remind us how far reality can be from our plans.