What Are Distributed Systems?

How does a distributed system work?

Distributed systems have evolved over time, but today’s most common implementations are largely designed to operate via the internet and, more specifically, the cloud. A distributed system begins with a task, such as rendering a video to create a finished product ready for release.

The web application, or distributed applications, managing this task — like a video editor on a client computer — splits the job into pieces. In this simple example, the algorithm gives one frame of the video to each of a dozen different computers (or nodes) to complete the rendering. Once the frame is complete, the managing application gives the node a new frame to work on. This process continues until the video is finished and all the pieces are put back together.

A system like this doesn’t have to stop at just 12 nodes — the job may be distributed among hundreds or even thousands of nodes, turning a task that might have taken days for a single computer to complete into one that is finished in a matter of minutes.

What are the types of distributed systems?

There are many models and architectures of distributed systems in use today.

• Client-server systems, the most traditional and simple type of distributed system, involve a multitude of networked computers that interact with a central server for data storage, processing or other common goal.
• Cell phone networks are an advanced type of distributed system that share workloads among handsets, switching systems and internet-based devices.
• Peer-to-peer networks, in which workloads are distributed among hundreds or thousands of computers all running the same software, are another example of a distributed system architecture.
• The most common forms of distributed systems in the enterprise today are those that operate over the web, handing off workloads to dozens of cloud-based virtual server instances that are created as needed, then terminated when the task is complete.

What are key characteristics of a distributed system?

Distributed systems are commonly defined by the following key characteristics and features:

• Scalability: The ability to grow as the size of the workload increases is an essential feature of distributed systems, accomplished by adding additional processing units or nodes to the network as needed.
• Concurrency: Distributed system components run simultaneously. They’re also characterized by the lack of a “global clock,” when tasks occur out of sequence and at different rates.
• Availability/fault tolerance: If one node fails, the remaining nodes can continue to operate without disrupting the overall computation.
• Transparency: An external programmer or end user sees a distributed system as a single computational unit rather than as its underlying parts, allowing users to interact with a single logical device rather than being concerned with the system’s architecture.
• Heterogeneity: In most distributed systems, the nodes and components are often asynchronous, with different hardware, middleware, software and operating systems. This allows the distributed systems to be extended with the addition of new components.
• Replication: Distributed systems enable shared information and messaging, ensuring consistency between redundant resources, such as software or hardware components, improving fault tolerance, reliability and accessibility.

What is distributed tracing?

Distributed tracing, sometimes called distributed request tracing, is a method for monitoring applications — typically those built on a microservices architecture — which are commonly deployed on distributed systems. Distributed tracing is essentially a form of distributed computing in that it’s commonly used to monitor the operations of applications running on distributed systems.

In software development and operations, tracing is used to follow the course of a transaction as it travels through an application — an online credit card transaction as it winds its way from a customer’s initial purchase to the verification and approval process to the completion of the transaction, for example. A tracing system monitors this process step by step, helping a developer to uncover bugs, bottlenecks, latency or other problems with the application.

Distributed tracing is necessary because of the considerable complexity of modern software architectures. A distributed tracing system is designed to operate on a distributed services infrastructure, where it can track multiple applications and processes simultaneously across numerous concurrent nodes and computing environments.

Without distributed tracing, an application built on a microservices architecture and running on a system as large and complex as a globally distributed system environment would be impossible to monitor effectively.

(Learn about best practices for distributed tracing.)

What are patterns in a distributed system?

A software design pattern is a programming language defined as an ideal solution to a contextualized programming problem. Patterns are reusable solutions to common problems that represent the best practices available at the time, and while they don’t provide finished code, they provide replication capabilities and offer guidance on how to solve a certain issue or implement a needed feature.

When thinking about the challenges of a distributed computing platform, the trick is to break it down into a series of interconnected patterns; simplifying the system into smaller, more manageable and more easily understood components helps abstract a complicated architecture. Patterns are commonly used to describe distributed systems, such as command and query responsibility segregation (CQRS) and two-phase commit (2PC). Different combinations of patterns are used to design distributed systems, and each approach has unique benefits and drawbacks.

What are the benefits of distributed systems?

Distributed systems offer a number of advantages over monolithic, or single, systems, including:

• Greater flexibility: It is easier to add computing power as the need for services grows. In most cases today, you can add servers to a distributed system on the fly.
• Reliability: A well-designed distributed system can withstand failures in one or more of its nodes without severely impacting performance. In a monolithic system, the entire application goes down if the server goes down.
• Enhanced speed: Heavy traffic can bog down single servers when traffic gets heavy, impacting performance for everyone. The scalability of distributed databases and other distributed systems makes them easier to maintain and also sustain high-performance levels.
• Geo-distribution: Distributed content delivery is both intuitive for any internet user, and vital for global organizations.

What are some challenges of distributed systems?

Distributed systems are considerably more complex than monolithic computing environments, and raise a number of challenges around design, operations and maintenance. These include:

• Increased opportunities for failure: The more systems added to a computing environment, the more opportunity there is for failure. If a system is not carefully designed and a single node crashes, the entire system can go down. While distributed systems are designed to be fault tolerant, that fault tolerance isn’t automatic or foolproof. 
• Synchronization process challenges: Distributed systems work without a global clock, requiring careful programming to ensure that processes are properly synchronized to avoid transmission delays that result in errors and data corruption. In a complex system — such as a multiplayer video game — synchronization can be challenging, especially on a public network that carries data traffic.
• Imperfect scalability: Doubling the number of nodes in a distributed system doesn’t necessarily double performance. Architecting an effective distributed system that maximizes scalability is a complex undertaking that needs to take into account load balancing, bandwidth management and other issues.
• More complex security: Managing a large number of nodes in a heterogeneous or globally distributed environment creates numerous security challenges. A single weak link in a file system or larger distributed system network can expose the entire system to attack.
• Increased complexity: Distributed systems are more complex to design, manage and understand than traditional computing environments.

What are the risks of distributed systems?

The challenges of distributed systems as outlined above create a number of correlating risks. These include:

• Security: Distributed systems are as vulnerable to attack as any other system, but their distributed nature creates a much larger attack surface that exposes organizations to threats.
• Risk of network failure: Distributed systems are beholden to public networks in order to transmit and receive data. If one segment of the internet becomes unavailable or overloaded, distributed system performance may decline.
• Governance and control issues: Distributed systems lack the governability of monolithic, single-server-based systems, creating auditing and adherence issues around global privacy laws such as GDPR. Globally distributed environments can impose barriers to providing certain levels of assurance and impair visibility into where data resides.
• Cost control: Unlike centralized systems, the scalability of distributed systems allows administrators to easily add additional capacity as needed, which can also increase costs. Pricing for cloud-based distributed computing systems are based on usage (such as the number of memory resources and CPU power consumed over time). If demand suddenly spikes, organizations can face a massive bill.

How do you apply access control in distributed systems?

Administrators use a variety of approaches to manage access control in distributed computing environments, ranging from traditional access control lists (ACLs) to role-based access control (RBAC).

One of the most promising access control mechanisms for distributed systems is attribute-based access control (ABAC), which controls access to objects and processes using rules that include information about the user, the action requested and the environment of that request. Administrators can also refine these types of roles to restrict access to certain times of day or certain locations.

How are distributed systems used?

Distributed systems are used when a workload is too great for a single computer or device to handle. They’re also helpful in situations when the workload is subject to change, such as e-commerce traffic on Cyber Monday. Today, virtually every internet-connected web application that exists is built on top of some form of distributed system.

Some of the most common examples of distributed systems:

• Telecommunications networks (including cellular networks and the fabric of the internet)
• Graphical and video-rendering systems
• Scientific computing, such as protein folding and genetic research
• Airline and hotel reservation systems
• Multiuser video conferencing systems
• Cryptocurrency processing systems (e.g. Bitcoin)
• Peer-to-peer file-sharing systems (e.g. BitTorrent)
• Distributed community compute systems (e.g. Folding@Home)
• Multiplayer video games
• Global, distributed retailers and supply chain management (e.g. Amazon)

What are different types of distributed deployments?

Distributed deployments can range from tiny, single department deployments on local area networks to large-scale, global deployments. In addition to their size and overall complexity, organizations can consider deployments based on:

• The size and capacity of their computer network
• The amount of data they’ll consume
• How frequently they run processes and whether they'll be scheduled or ad hoc
• The number of users accessing the system
• Capacity of their data center 
• The necessary data fidelity and availability requirements

Based on these considerations, distributed deployments are categorized as departmental, small enterprise, medium enterprise or large enterprise.

While there are no official taxonomies delineating what separates a medium enterprise from a large enterprise, these categories represent a starting point for planning the needed resources to implement a distributed computing system. Distributed systems can also evolve over time, transitioning from departmental to small enterprise as the enterprise grows and expands.

Why do we need distributed systems now?

Modern computing wouldn’t be possible without distributed systems. They’re essential to the operations of wireless networks, cloud computing services and the internet. If distributed systems didn’t exist, neither would any of these technologies.

But do we still need distributed systems for enterprise-level jobs that don’t have the complexity of an entire telecommunications network? In most cases, the answer is yes. Distributed systems provide scalability and improved performance in ways that monolithic systems can’t, and because they can draw on the capabilities of other computing devices and processes, distributed systems can offer features that would be difficult or impossible to develop on a single system.

This includes things like performing an off-site server and application backup — if the master catalog doesn’t see the segment bits it needs for a restore, it can ask the other off-site node or nodes to send the segments. Virtually everything you do now with a computing device takes advantage of the power of distributed systems, whether that’s sending an email, playing a game or reading this article on the web.

Distributed systems are well-positioned to dominate computing as we know it for the foreseeable future, and almost any type of application or service will incorporate some form of distributed computing. The need for always-on, available-anywhere computing is driving this trend, particularly as users increasingly turn to mobile devices for daily tasks. Looking ahead, distributed systems are certain to cement their importance in global computing as enterprise developers increasingly rely on distributed tools to streamline development, deploy systems and infrastructure, facilitate operations and manage applications.