System Design for Beginners: How Real Applications Scale in Production

A developer-friendly walkthrough of DNS, load balancers, API gateways, caching, queues, database replicas, and the decisions that connect them

Most developers learn system design too late.

We build APIs, connect a database, deploy the application, and assume the job is finished. It works perfectly with ten users. Maybe even a hundred.

Then traffic increases.

Requests become slower. The database starts struggling. A third-party API goes down. One expensive endpoint gets called thousands of times. Eventually, the application stops responding.

That is when system design stops looking like interview theory.

It becomes debugging.

The useful way to learn system design is not by memorising dozens of components. Start with one server, introduce one real problem at a time, and add infrastructure only when that problem demands it.

Start With the Smallest Working System

A basic web application needs two things:

• A client making requests
• A server processing them

The server is simply a machine running your application. It has CPU, memory, storage, and an IP address through which clients can reach it.

The problem is that nobody wants to remember an address such as:

10.2.3.4

We prefer:

myapp.com

DNS connects those two worlds. It resolves a human-readable domain into the IP address required by the network.

User enters myapp.com
 ↓
DNS returns the destination address
 ↓
Browser sends the request to the server

For a small application, this architecture may be enough.

The trouble begins when success arrives.

The First Scaling Mistake: Buying a Bigger Machine

Suppose the server has two CPUs and 4 GB of RAM. As traffic grows, memory fills up and requests compete for processing time.

The obvious response is to upgrade the server:

2 CPUs, 4 GB RAM
 ↓
16 CPUs, 64 GB RAM

This is vertical scaling.

It is simple because the application still runs on one machine. There is no traffic distribution, service discovery, or coordination between replicas.

But there are two limitations:

1. A machine can only become so large.
2. Increasing resources may require restarting it.

At first, I assumed vertical scaling was the cleanest answer because it avoided architectural complexity. The part I overlooked was traffic behaviour. Real traffic is rarely constant.

A commerce application may remain quiet during the afternoon and receive a sudden spike when a sale begins. Keeping an oversized server running all month means paying for capacity that sits idle most of the time.

Horizontal Scaling Changes the Problem

Instead of making one server larger, we can run multiple copies of the application.

Server 1
Server 2
Server 3

This is horizontal scaling.

New servers can be added when traffic rises and removed when demand falls. One server can fail while the others continue handling requests.

But now every server has its own address.

Which one should the client call?

That is the problem solved by a load balancer.

The DNS record points to the load balancer. The load balancer then distributes requests among healthy application instances.

A simple strategy is round-robin:

class RoundRobinBalancer {
 constructor(servers) {
 this.servers = servers;
 this.nextIndex = 0;
 }
getNextServer() {
 const server = this.servers[this.nextIndex];
 this.nextIndex = (this.nextIndex + 1) % this.servers.length;
 return server;
 }
}
const balancer = new RoundRobinBalancer([
 "server-1",
 "server-2",
 "server-3"
]);
console.log(balancer.getNextServer()); // server-1
console.log(balancer.getNextServer()); // server-2

This example explains the idea, not a production load balancer. Real systems also consider server health, active connections, latency, and capacity.

The important lesson is that adding servers is only half the solution. Traffic must know how to reach them.

Load Balancer and API Gateway Are Not the Same Thing

This distinction confused me when I first studied microservices.

Imagine an application containing:

• Authentication service
• Order service
• Payment service
• Product service

Each service may have several replicas.

An API gateway decides which service should receive the request:

/auth/* → Authentication Service
/orders/* → Order Service
/payments/* → Payment Service

The service’s load balancer then decides which replica should process it.

Client
 ↓
API Gateway
 ↓
Order Service Load Balancer
 ↓
Order Server 1, 2, or 3

The gateway handles routing across different services. The load balancer distributes traffic across equivalent instances of the same service.

That small distinction makes a large architecture much easier to understand.

Slow Work Should Leave the Request Path

Consider a payment endpoint that must also send an email.

A beginner implementation may do this:

app.post("/payments", async (req, res) => {
 const payment = await processPayment(req.body);
 await sendConfirmationEmail(payment);
 res.status(201).json(payment);
});

The payment response now depends on the email provider.

If the email API takes four seconds, the user waits four extra seconds. If it fails, a successful payment may appear unsuccessful.

A better design places an event in a queue:

app.post("/payments", async (req, res) => {
 const payment = await processPayment(req.body);
 await emailQueue.send({
 type: "PAYMENT_CONFIRMED",
 paymentId: payment.id,
 email: payment.customerEmail
 });
 res.status(201).json(payment);
});

A separate worker processes it:

async function processEmailJobs() {
 while (true) {
 const job = await emailQueue.receive();
 try {
 await sendConfirmationEmail(job);
 await emailQueue.acknowledge(job);
 } catch (error) {
 await emailQueue.retryOrMoveToDeadLetterQueue(job);
 }
 }
}

Now the payment API finishes quickly, while retries and email failures remain isolated.

The surprising payoff is that queues are not only performance tools. They are also failure boundaries.

One Event, Several Consumers

A completed payment may trigger:

• Customer email
• Vendor notification
• SMS
• Analytics update
• Inventory processing

A single queue usually gives one message to one consumer. When several independent services need the same event, a publish-subscribe model works better.

Payment Completed
 ↓
 Topic
 ↙ ↓ ↘
Email SMS Analytics
Queue Queue Queue

Giving each subscriber its own queue creates a fan-out architecture. Every service receives the event, but each still gets queue-based retries and failure handling.

Protect the Database Before Scaling It

Application servers are easy to replicate. The database usually becomes the harder bottleneck.

Two practical improvements help:

1. Cache repeated reads

async function getProduct(productId) {
 const key = `product:${productId}`;
 const cachedProduct = await redis.get(key);
 if (cachedProduct) {
 return JSON.parse(cachedProduct);
 }
 const product = await database.products.findById(productId);
 await redis.set(key, JSON.stringify(product), {
 EX: 300
 });
 return product;
}

This cache-aside pattern reduces repeated database queries.

The catch is stale data. A five-minute cache improves speed, but recently updated products may temporarily return an older value.

2. Separate critical and non-critical reads

Writes go to the primary database. Reporting and analytics queries can use read replicas when a small replication delay is acceptable.

Not every read requires perfectly current data. Recognising that difference can remove significant load from the primary database.

Rate Limiting Is Part of Capacity Planning

A scalable system should not accept unlimited work.

Even legitimate users can overwhelm an expensive endpoint. Bots and denial-of-service attempts make the problem worse.

A basic Express limiter might look like this:

import rateLimit from "express-rate-limit";
const apiLimiter = rateLimit({
 windowMs: 60 * 1000,
 limit: 100,
 standardHeaders: true,
 legacyHeaders: false
});
app.use("/api", apiLimiter);

This allows 100 requests per minute per identified client.

Production systems may use token-bucket or leaky-bucket algorithms with a shared store such as Redis. Otherwise, every application replica maintains a different counter and the limit becomes unreliable.

Reflection: Scaling Is Mostly About Removing Dependencies

After understanding these components, I stopped viewing system design as a collection of cloud-service names.

Each component removes a specific dependency:

• DNS removes the need to remember server addresses.
• A load balancer removes dependence on one application instance.
• An API gateway removes routing responsibility from clients.
• A queue removes slow work from the synchronous request.
• Caching removes repeated pressure from the database.
• Read replicas remove analytical load from the primary node.
• Rate limiting prevents demand from exceeding controlled capacity.

That was the useful shift for me.

A scalable architecture is not created by adding every available component. It is created by finding the next bottleneck, understanding its failure mode, and adding the smallest mechanism that solves it.

Final Takeaways

Start simple, but know where the design will break.

Most importantly, do not add infrastructure because it appears in architecture diagrams.

Add it when you can clearly name the problem it solves.

From Dev Simplified

• 👏 Enjoyed the article? Don’t forget to leave a clap.
• 💬 Have thoughts or questions? Share them in the comments.
• ✍️ Want to write for Dev Simplified? Drop a personal note on any Dev Simplified story with your draft link.

Before you go

• Please take a moment to like the post and follow the writer!
• Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute here