Embeddings, RAG, AI Agents, and MCP: How the Modern AI Stack Actually Works

A practical mental model for understanding how AI applications search private data, retrieve context, make decisions, and interact with external tools

Thumbnail Image: Embeddings, RAG, AI Agents, and MCP: How the Modern AI Stack Actually Works

Most developers can call an LLM API within an afternoon.

The harder part begins when someone asks the application a question the model cannot answer:

“What action items are still pending from last week’s design review?”

The model was never trained on your meeting notes. It cannot inspect your database by itself. It does not automatically know which document matters, and it cannot send an email unless your application explicitly gives it that ability.

This is why production AI applications rarely consist of a single LLM call.

They are systems made from several connected pieces:

  • Embeddings represent meaning.
  • Vector databases retrieve related information.
  • RAG gives the model relevant context.
  • Agents decide which actions to perform.
  • MCP provides a consistent way to expose tools and data.

Once I understood how these parts connect, AI architecture stopped feeling mysterious. It started looking like regular software engineering — with a probabilistic component in the middle.

It Starts With a Problem Keywords Cannot Solve

Suppose your application stores this task:

Prepare the quarterly project update for the engineering team.

Later, a user searches:

Draft the technical progress summary.

A traditional keyword search may struggle because the phrases contain different words.

Semantically, however, they are closely related.

This is where embeddings help.

An embedding model converts content into a numerical vector:

const embedding = await embeddingModel.embed(
 "Prepare the quarterly project update"
);
console.log(embedding);
// [0.018, -0.224, 0.731, ...]

The individual numbers are not useful to a developer reading them. Their position relative to other vectors is what matters.

Texts with similar meanings usually appear closer together in the embedding space.

Image- 3D projections of HD vectors

Embeddings can represent more than text. Depending on the model, they can also represent code, images, audio, and other structured content.

The important lesson is simple:

Embeddings do not give the model memory. They make semantic comparison possible.

Where Do All Those Vectors Go?

Generating embeddings is only half the job. Your application also needs somewhere to store and search them.

That is the role of a vector database or vector-search system.

Each stored record usually contains:

  • The embedding
  • The original text or a reference to it
  • Metadata such as document ID, user ID, date, or category

A simplified record might look like this:

await vectorStore.insert({
 id: "meeting-note-42",
 values: noteEmbedding,
 metadata: {
 source: "design-review",
 date: "2026-06-24",
 text: "The mobile navigation needs another accessibility review."
 }
});

When the user submits a question, the application embeds the question and searches for nearby vectors:

const queryVector = await embeddingModel.embed(
 "What is still pending from the design review?"
);
const matches = await vectorStore.search({
 vector: queryVector,
 topK: 5,
 filter: { source: "design-review" }
});

This is semantic search. The wording can change while the intent remains similar.

Popular options include Pinecone, Weaviate, Milvus, and FAISS. They differ in hosting model, filtering support, scalability, operational complexity, and cost.

Image: Vector DB options with strength and tradeoffs

RAG Is a Pipeline, Not a Database

At first, I assumed the LLM somehow searched the vector database itself.

It does not.

Your application performs the retrieval and places the results inside the model’s prompt. This pattern is called retrieval-augmented generation, or RAG.

A minimal version looks like this:

const context = matches
 .map(match => match.metadata.text)
 .join("\n\n");
 const response = await llm.generate(`
 Answer only from the supplied context.
 Context:
 ${context}
 Question:
 What tasks remain from the design review?
`);

RAG helps the model answer questions using private or recently updated information.

But there is a catch.

Retrieval does not guarantee correctness.

Poor document chunking, irrelevant matches, missing metadata filters, or vague prompts can still produce bad answers. RAG reduces unsupported guessing; it does not eliminate it.

A production RAG pipeline often needs:

  1. Document parsing
  2. Chunking
  3. Embedding generation
  4. Metadata storage
  5. Similarity search
  6. Optional reranking
  7. Prompt construction
  8. Source citation

Most tutorials explain only steps three and five. In practice, chunking and retrieval quality often affect the result more than changing the LLM.

Agents Add Decisions and Actions

RAG answers questions. An agent can decide what should happen next.

Consider this request:

“Find my three highest-priority tasks and email them to the engineering team.”

The system may need to:

  1. Retrieve today’s tasks.
  2. Rank them.
  3. Draft a concise summary.
  4. Ask for approval, depending on the workflow.
  5. Call an email tool.
  6. Confirm whether the operation succeeded.

That is orchestration.

const tools = {
 searchTasks,
 sendEmail,
 getCalendarEvents
};
const result = await agent.run({
 request: userMessage,
 tools
});

ReAct-style loops allow an agent to alternate between reasoning, tool use, observing results, and choosing another step.

The surprising part is that more autonomy is not always better.

For predictable workflows — such as generating a report every Friday — a normal application pipeline is often safer and easier to debug. Agents are most useful when the sequence of actions depends on information discovered during execution.

MCP Standardizes the Tool Boundary

Tool integrations quickly become repetitive.

One connector expects JSON in one format. Another has different authentication. A third uses its own discovery mechanism.

Model Context Protocol, or MCP, aims to provide a standard interface through which AI applications can discover and use tools, resources, and prompts.

Instead of tightly coupling every AI application to every service, developers can expose capabilities through MCP servers.

The benefit is not that MCP makes an agent intelligent. It makes integration boundaries more consistent.

Security still remains your responsibility:

  • Authenticate users.
  • Authorize every operation.
  • Validate tool arguments.
  • Limit accessible resources.
  • Log sensitive actions.
  • Require approval before destructive operations.

A universal connector without strong permission controls only creates a universal security problem.

A Lesson I Learned From Looking at the Whole Stack

The biggest shift was realizing that the LLM is not the entire application.

The model generates and reasons. The surrounding system handles retrieval, permissions, state, observability, tool execution, fallbacks, and failure recovery.

That also changes how you debug AI applications.

When an answer is wrong, the model may not be the cause. The query embedding may be weak. The wrong chunks may have been retrieved. A metadata filter may be missing. The tool could have returned stale information.

AI debugging is often pipeline debugging.

Final Takeaways

Modern AI systems become easier to understand when each component has a clear responsibility:

  • Embeddings convert meaning into comparable numerical representations.
  • Vector databases retrieve semantically related information.
  • RAG places retrieved information into the model’s context.
  • Agents choose and execute actions.
  • MCP standardizes how tools and resources are exposed.

Do not begin by combining every component.

Start with the problem.

Use semantic search when keywords fail. Add RAG when the model needs private knowledge. Introduce tools when the system must take action. Use agentic orchestration only when the workflow genuinely requires dynamic decisions.

The real skill is not knowing every AI term.

It is knowing which parts your application actually needs — and which parts would only make it harder to operate.

From Dev Simplified

  • 👏 Enjoyed the article? Don’t forget to leave a clap.
  • 💬 Have thoughts or questions? Share them in the comments.
  • ✍️ Want to write for Dev Simplified? Drop a personal note on any Dev Simplified story with your draft link.