Scalability as an Architectural Concern (Part 1)

This article is part of a series (authored by Murat Erder, Eoin Woods and me) that provides practical advice and guidance on how to leverage the Continuous Architecture approach. Today, we’ll start discussing scalability, by providing a definition, discussing its importance, its relationship with other quality attributes and the forces affecting it.

Scalability as a Quality Attribute

Scalability has not been traditionally at the top of the list of quality attributes for an architecture, but this view of scalability has changed during the last few years, perhaps due to incidents that may have been related to scalability issues. As an example, the year 2020 was one of dramatic volatility for the stock market. As concerns about the impact of COVID-19 sent the market plunging and then monetary policy and the promise of a vaccine sent them soaring back up, retail investors and traders scrambled to put in trades to limit loss or in hopes of profit. But on the days when the market was the most volatile, the days when there was the most to lose or gain, the days when activity was the highest, many retail investors found that they were unable to enter trades or even see their portfolios online[1]. Some were unable to log in, some who logged in were unable to enter trades in a timely manner, and some found that their accounts were showing up with an erroneous zero balance. As events with similar negative effects may occur in the future for all types of systems, software architects need to focus on key quality attributes such as scalability and resilience when architecting new systems or enhancing existing ones.

Another reason why the perception of scalability by IT professionals has changed could be because large Internet-based companies such as Google, Amazon, and Netflix have been focusing more on scalability. Software architects and engineers realize that treating scalability as an afterthought may have serious consequences should an application workload suddenly exceed expectations. When this happens, they are forced to make decisions rapidly, often with insufficient data. It is much better to seriously consider scalability from the start. But what exactly do we mean by scalability in architecture terms, and what is its importance?

What is Scalability & Why is it Important?

Scalability can be defined as the property of a system to handle an increased (or decreased) workload by increasing (or decreasing) the cost of the system. The cost relationship may be flat, may have steps, may be linear, or may be exponential. In this definition, increased workload could include higher transaction volumes or a greater number of users. A system is usually defined as a combination of software and computing infrastructure and may include the people required to operate the software and associated infrastructure.

To understand the importance of scalability, consider companies such as Google, Amazon, Facebook, and Netflix. They handle massive volumes of requests every second and are able to successfully manage extremely large datasets in a distributed environment. Even better, their sites have the ability to rapidly scale without disruption in response to increased usage due to expected events such as the holiday season or unexpected events such as health incidents (e.g., the COVID-19 pandemic) and natural disasters. Although you probably don’t need that amount of scalability, it is very likely that there are critical scalability levels needed to support your business (also see the last paragraph in this section).

Testing for scalability is generally difficult for any system, as the resources needed may be available only in the production environment, and generating a synthetic workload may require significant resources and careful design. This is especially difficult for sites that need to handle both high and low traffic volumes without precise volume estimates. Yet companies such as Google, Amazon, Facebook, and Netflix excel at achieving scalability in very dynamic environments, and this is a critical factor behind their success.

One word of caution here: software architects and engineers need to remember that scalability tactics used at those companies do not necessarily apply to other companies that operate at a different scale. They need to be careful about too quickly reusing scalability tactics in their own environments without thoroughly understanding the implications of the tactics they are considering and capturing their requirements and assumptions explicitly as a series of architectural decisions (see our “Essential Activities of Software Architecture” article).

Scalability Requirements & Cost Consideration

Scalability requirements are notoriously difficult to elicit. Stakeholders may not always be a good source for scalability requirements, but those requirements can often be inferred from business goals. For example, in order to achieve a target revenue goal, a certain number of transactions would need to be conducted, assuming a particular conversion rate. Each of these estimated factors would have its own statistical distribution, which might tell you that you need 10 times more standby capacity than average to be able to handle loads at the 99^th percentile.

Most quality attributes requirements have an impact on cost. For example, design choices made to satisfy scalability, performance, availability and usability requirements will affect the cost of the solution. Cost effectiveness is not commonly included in the list of quality attribute requirements for a system, yet it is almost always a factor that must be considered. All clients are cost sensitive, regardless of the business domain.  The diagram below illustrates the relationships between some of the key quality attributes for many of today’s systems.

Developing an executable architecture quickly and then evolving it are essential for modern applications that need to rapidly deliver software systems to meet ever-changing business requirements. However, you do not want to paint yourself into a corner by underestimating the economic impact of scalability. Not planning for scalability up front and hoping that your platform will be able to survive a sudden load increase by relying on a relatively simple mechanism such as a load balancer and leveraging an ever-increasing amount of cloud-based infrastructure may turn out to be an expensive proposition. This strategy may not even work in the long run.

Architectural decisions for scalability could have a significant impact on deployment costs and may require tradeoffs between scalability and other attributes, such as performance. For example, an architect may need to sacrifice some performance initially by implementing asynchronous interfaces between some services to ensure that the software system can cope with greater loads at some point in the future. Some architectural decisions for scalability may also increase the cost of the system by including additional components such as a message bus to effectively implement inter-service asynchronous interfaces. Scalability is clearly a key driver of most modern software architectures.

Forces Affecting Scalability

One way to look at scalability is to use a supply-and-demand model frame of reference. Demand-related forces cause increases in workload, such as increases in number of users, user sessions, transactions and events. Supply-related forces relate to the number and organization of resources that are needed to meet increases in demand, such as the way technical components (e.g., memory, persistence, events, messaging, data) are handled. Supply-related forces also relate to the effectiveness of processes associated with the management and operation of the technical component as well as the architecture of the software.

Architectural decisions determine how these forces interact. Scaling is optimized when there is a balance between these forces. When demand-related forces overwhelm supply-related forces, the system performs poorly or fails, which has a cost in terms of customer experience and potentially market value. When supply-related forces overwhelm demand, the organization has overbought and needlessly increased its costs, although using a commercial cloud may minimize this issue.

The next few articles in this series will continue to discuss the architectural aspects of scalability, such as the impact of cloud computing on scalability and some of the architectural tactics available to deal with scalability requirements, so we hope that you find those articles interesting and you will keep on reading them!

For more information on our book, “Continuous Architecture in Practice”, which discusses scalability in much more detail, please visit our website at continuousarchitecture.com.

---

[1] See for example https://www.thinkadvisor.com/2020/11/09/trading-outages-at-schwab-e-trade-td-ameritrade-other-platforms-irk-investors/