I Accidentally Built a Banking Chatbot Anyone Could Hack

A step-by-step teardown from Demo to Production: Architecting a Real Agentic AI System

Here’s a number that made me sit up straight the first time I saw it: 4.2 lakh calls in a single month, just to a bank’s customer support line.

Not complaints. Not fraud reports.

Just people asking

What’s my balance
Why was I charged
My checkbook is finished, please send a new one.

Average handle time on a toll-free call? About 4 minutes. And here’s the part that stings the most— the bank pays the phone bill for every single one of those calls, even though the customer pays nothing.

Do that math over a year and you’re looking at a number with a lot of zeros in it, for information that’s already sitting inside the bank’s own net banking app.

So why do people keep calling instead of just logging in?

Because net banking has 14+ screens, and most people don’t want to hunt through a menu tree to find one number.

They want to ask. In plain language.

Like they’re talking to a person.

That’s the exact problem I got handed: Build a chatbot that can answer these questions conversationally, without making the bank’s security team have a heart attack and bank having to pay a long bill.

I want to walk you through how that build actually went — not the polished final architecture diagram you’d see in a conference talk, but every dumb mistake, security hole, and “oh no” moment that happened on the way there. Because honestly, that’s where the real learning is.

If you’ve ever wondered what separates “I built an AI agent this weekend” from “this agent is now live for millions of banking customers,” stick around. It’s a longer list than you’d think, and most of it has nothing to do with prompting.

Stage 1: The chatbot that couldn’t do anything

First version was embarrassingly simple. A UI where a user types a question, an API on the backend catches it, passes it to an agent, the agent talks to an LLM, and whatever comes back gets shown on screen.

I typed: “What is my account balance?”

The bot replied: “Sorry, I don’t have access to your bank accounts or account balances.”

Which, fair. I hadn’t connected it to anything. It was a chatbot with a personality and zero knowledge — basically a very polite wall.

Question for you: how many “AI chatbot” demos have you seen that stop exactly here, dressed up with a nice UI, and get called “production ready”? I’ve seen more than I’d like to admit.

Stage 2: Giving it tools (and immediately regretting it)

The bank already had internal APIs for everything — balance inquiry, transaction history, statement requests, address changes. Net banking itself is basically a UI wrapped around these APIs. So step two was obvious: wire the agent up to them.

This turns your setup into what people call a tool-based agent. The flow looks roughly like this:

User: "What's my balance?"
   │
   ▼
Agent → sends question + list of available tools → LLM
   │
   ▼
LLM → "use the balance_inquiry tool" → Agent
   │
   ▼
Agent calls balance_inquiry API → gets result
   │
   ▼
Agent sends result back to LLM → LLM writes a natural reply
   │
   ▼
"Your account balance is ₹24,345"

In code, the tool definition is nothing fancy:

tools = [
    {
        "name": "balance_inquiry",
        "description": "Fetch the account balance for a given customer ID",
        "parameters": {"customer_id": "string"}
    },
    {
        "name": "transaction_history",
        "description": "Fetch the last N transactions for a customer",
        "parameters": {"customer_id": "string", "count": "int"}
    },
    # ...and so on
]

response = llm.chat(
    messages=conversation_history,
    tools=tools
)

if response.tool_call:
    result = call_bank_api(response.tool_call.name, response.tool_call.args)

Worked great with 6 tools. Then I sat down and mapped out what a real banking chatbot needs — loan queries, card blocking, dispute filing, standing instructions, KYC updates — and landed somewhere around 30–40 tools.

At that point the LLM starts second-guessing itself. Should this go to “update_address” or “service_request_generic”?

Is a checkbook request a “service” tool or an “account” tool? I watched the model pick the wrong tool for questions I’d have thought were obvious.

This is what people mean by tool overload — pile enough options in front of any decision-maker, human or LLM, and accuracy drops.

Stage 3: Breaking the agent into specialists

The fix felt obvious overall: stop making one agent know everything. I split it into three specialist agents:

Accounts agent — balance, statements
Transaction agent — transaction history, disputes
Service agent — checkbooks, address changes, card requests

Each one only sees the tools relevant to its job. Ask “what were my last 5 transactions” and routing correctly sends that to the transaction agent, which now has a much smaller, much cleaner decision space.

This is basically the same lesson every engineering team relearns eventually: a service that does one thing well beats a service that tries to do everything. Turns out it’s true for agents too.

But then I asked it a two-part question, and everything fell apart again.

Stage 4: When one question needs two agents

“What’s my balance, and can you also get me my last five transactions?”

No single specialist agent can answer that alone. The accounts agent doesn’t know transactions, and vice versa. So who decides that this question actually needs both of them, in what order, and how to merge the answers?

That’s where a coordinator agent comes in — a layer above the specialists whose only job is planning:

plan = coordinator_llm.plan(user_question)
# plan = [
#   {"agent": "accounts_agent", "action": "get_balance"},
#   {"agent": "transaction_agent", "action": "get_last_n_transactions", "n": 5}
# ]

results = []
for step in plan:
    results.append(dispatch_to_agent(step))
final_answer = llm.summarize(results)

The coordinator asks the LLM to break the question into a sequence of steps, fires off each specialist agent, waits on the results, and then asks the LLM one more time to stitch it all into a single, readable answer.

Have you ever built something that worked fine for single-step requests and then completely face-planted the moment a user asked a compound question? I feel like this happens in basically every chatbot project — the multi-intent query is where the “it works on my machine” demo dies.

Stage 5: Ripping the API logic out of my agents

Here’s a mistake I almost shipped: baking all the API-calling logic — auth headers, retries, error handling, response parsing — directly into each agent’s codebase.

Three tools on the service agent meant three separate integration headaches living inside my agent’s business logic.

That’s backwards. An agent’s job is reasoning — figuring out which tool to call and why. It shouldn’t also be the one wrestling with an API’s quirks.

So I pulled all of that out into separate MCP (Model Context Protocol) servers — one for accounts, one for transactions, one for service requests. Each MCP server owns its own API contracts, error handling, and response shaping. The agent just talks to the MCP server in a standard way and stops caring about the mess underneath.

The difference is basically the same reason you don’t write raw SQL inline in every controller — you put a data layer in between so the messy stuff has one home.

Stage 6: The security hole that should scare you

I want to pause here because this is the part that genuinely worried me when I saw it happen.

I opened a fresh chat and asked: “What’s my account balance?” It asked for a customer ID. I gave it one — any ID I happened to know — and it happily told me that account’s balance.

Then I opened another fresh chat, gave it a different customer ID, and it told me that person’s balance too.

Read that again. The bot would tell anyone anyone’s balance, as long as they could type in a customer ID. No login. No verification. Just “trust me, this is my ID.”

I’d built a fully functional information-leaking machine with a friendly conversational interface.

Stage 7: Authentication — knowing who’s asking

The fix is to stop asking the user who they are and instead know who they are. Every serious organization already has an identity provider (IdP) for employee and customer logins. So the flow changes:

User opens the chatbot
Gets redirected to the bank’s identity provider
Logs in with username/password (or MFA)
Gets redirected back, now carrying an auth token
Every request to the backend includes that token

The chatbot never has to ask “what’s your customer ID?” again — that information rides along silently with every request through token-based auth.

@app.route("/chat", methods=["POST"])
def chat():
    token = request.headers.get("Authorization")
    customer_id = verify_token(token)  # raises if invalid/expired
    question = request.json["message"]
    return coordinator_agent.handle(customer_id, question)

Notice: none of this is “AI engineering.” It’s the same authentication process every web app has needed since before anyone said the word “agent.”

Stage 8: Authentication isn’t authorization

Here’s a scenario that trips people up constantly, myself included the first time: two customers, same request — “can you increase my credit limit to ₹5 lakhs?”

Customer A: gets asked for an OTP, confirms it, limit increased. Customer B: gets told, “Sorry, I don’t have permission to do that for you.”

Both were logged in. Both were authenticated. So why the different outcome?

Because authentication tells you who someone is; authorization tells you what they’re allowed to do. Say the bank’s policy is that only “privileged” tier customers can self-serve a credit limit increase. Before the service agent even calls that tool, there’s a permission check:

def increase_credit_limit(customer_id, new_limit):
    tier = get_customer_tier(customer_id)
    if tier != "privileged":
        raise PermissionError("Not authorized for self-service credit limit change")
    # proceed with OTP flow...

Genuine question — how many of you have seen a system nail authentication but completely forget fine-grained authorization, and only find out when something embarrassing happens in prod? I’d bet it’s more common than anyone wants to admit.

Stage 9: The agent with amnesia

Next weird bug. A customer asks: “What was my last transaction?” Gets an answer — a debit to Amazon. Then follows up: “Was that the one I flagged as suspicious last week?”

The bot responds: “I don’t have any record of a transaction being flagged. Could you clarify which transaction you mean?”

Two things wrong here. First, it genuinely has no memory of anything flagged before (fair, that data might live elsewhere — separate problem). Second, and worse: it’s asking “which transaction” when the answer was literally in the previous message.

The root cause is simple once you know it: LLMs are stateless. Every call is a blank slate unless you explicitly hand it the conversation history. So the fix is a session store — a place to keep each customer’s conversation history and any shared state between agents (so if the transaction agent figures something out, the accounts agent can see it too).

session = session_store.get(customer_id)
session.history.append({"role": "user", "content": question})
response = llm.chat(messages=session.history, tools=tools)
session.history.append({"role": "assistant", "content": response.text})
session_store.save(customer_id, session)

Not glamorous. Just a database table, really. But without it, your “smart” assistant has the memory of a goldfish.

Stage 10: Please don’t send credit card numbers to a third-party API

This one made me genuinely uncomfortable. A customer typed their full credit card number into the chat while asking for a limit increase. That message — card number and all — was about to get sent straight to a third-party LLM API.

Think about what that means. If you’re calling out to an external provider, you’re handing them your customer’s sensitive financial data on every single request that happens to contain it. That’s not a hypothetical compliance problem, that’s an actual one.

Two changes here:

PII redaction before anything leaves your system:

def redact_pii(text):
    text = re.sub(r"\b\d{16}\b", "[CARD_NUMBER_REDACTED]", text)
    text = re.sub(r"\b\d{10}\b", "[PHONE_REDACTED]", text)
    # ...more patterns as needed
    return text

safe_message = redact_pii(user_message)
llm_response = external_llm.chat(safe_message)

Self-hosted LLM for general reasoning — an open-weight model running inside the bank’s own environment, so most traffic never leaves the secure boundary at all. The third-party model gets reserved for the cases that genuinely need heavier reasoning, and even then, only after redaction.

Worth asking yourself if you’re building anything similar: do you actually know what’s inside every prompt your app sends to an external API? Because I didn’t, until I went looking.

Stage 11: The regression nobody saw coming

A developer updates a system prompt. Old version: “before submitting a checkbook request, always confirm the delivery address with the customer.” New version: “submit the checkbook request using the customer’s registered details.”

Small change. Feels harmless. Except now, if a customer’s registered address is outdated, their checkbook just goes to the wrong place — silently, with no confirmation step to catch it.

Nobody wrote a failing unit test for this, because you can’t really unit-test “did the model behave the same way in spirit.” The output is natural language and it’s non-deterministic by nature. What you actually need is an evaluation suite — a curated set of test conversations (a “golden dataset”) covering edge cases you’ve thought through in advance, run against the system whenever anything changes, checking whether behavior drifted.

Have you ever shipped a “tiny prompt tweak” that quietly broke something three steps downstream? If you’ve worked with LLMs for more than a few weeks, I’d guess the answer is yes.

Stage 12: “My balance is wrong” — and you have no idea why

A customer disputes their balance: the bot said ₹18,200, their actual net banking shows ₹52,340. Support escalates it to engineering. You go digging through logs and find… almost nothing.

A timestamp for when the request came in, a timestamp for when the response went out. No record of what tool got called, what arguments were passed, which agent handled it, or what the model actually reasoned through.

You cannot debug what you cannot see. This is where observability stops being optional. For a normal service you’d track CPU, memory, error rates. For an agentic system you additionally need to capture:

logger.log_event(
    trace_id=trace_id,
    customer_id=customer_id,
    agent="accounts_agent",
    tool_called="balance_inquiry",
    tool_input={"customer_id": customer_id},
    tool_output=result,
    llm_prompt=prompt_sent,
    llm_response=raw_response
)

Every hop — coordinator to sub-agent, sub-agent to tool, tool to LLM — needs to be traceable. Otherwise “why did the bot say the wrong balance” becomes an unanswerable question, and that’s a genuinely bad place to be with a live banking product.

Stage 13: Nobody talks about the bill until it’s huge

Every layer of this system talks to an LLM. Every one of those calls costs money, and unlike a fixed server bill, this cost scales directly with how much people use your product — which is exactly the thing you want to grow. Without cost tracking per request, per agent, per customer segment, you find out you have a cost problem the same way you find out about a leak in your roof: after the damage is done.

Anyone here inherited an AI feature and only discovered the token bill during a budget review? That conversation is never fun.

Stage 14: The last mile — infrastructure and edge security

Everything up to this point was internal design. Getting this in front of real users adds a layer of pure infrastructure concerns: a Web Application Firewall to catch abuse patterns, rate limiting so one user can’t hammer your API and take the whole thing down, and an API gateway that enforces authentication before a request even reaches your backend. None of this is glamorous, none of it is “AI,” and all of it is non-negotiable if you’re putting this in front of real customers.

What I actually took away from building this

Looking back at the whole thing, there’s a pattern worth naming: about half of what made this system production-ready had nothing to do with AI at all. Authentication, authorization, rate limiting, session storage, logging — that’s just software engineering, the same stuff every serious backend needed a decade before “agent” was a buzzword.

The genuinely AI-specific parts were narrower than I expected going in: deciding how to split responsibilities across agents so the model doesn’t choke on too many tools, designing an eval suite for something that’s non-deterministic by nature, redacting PII before it touches a model you don’t control, and building observability that understands reasoning steps, not just requests and responses.

If there’s one thing I’d want someone earlier in their AI engineering journey to take from this: don’t let the “agent” abstraction convince you that regular engineering discipline doesn’t apply anymore. It applies more, not less, because now you’ve also got a non-deterministic component sitting in the middle of your request path.

So here’s what I’m actually curious about — if you’ve built something similar, what broke first for you? Was it security, like mine? Was it the memory problem? Or something I haven’t even hit yet? Drop it in the comments, I’d genuinely like to know where everyone else’s system fell over first.

From Tech By Neha Gupta

👏 Enjoyed the article? Don’t forget to leave a clap.
💬 Have thoughts or questions? Share them in the comments.