Skip to content
Shekhar Kumar

All projects

Work · Shipped

Generative AI platform

Model routing, tool calling, a custom MCP server, and RAG over vector databases.

Role
Architect
Organisation
Pure Billion Technologies
Updated

TL;DR

  1. A production-grade LangChain platform with model routing across multiple fine-tuned models.
  2. Categorised prompt orchestration, conversation context management and tool calling, including a custom MCP server for real-time web search.
  3. RAG pipelines over vector databases improved response relevance and accuracy by 25%.
response relevance (RAG)
+25%

Life of a request

Illustrative

  1. Client

    user prompt

  2. Model router

    picks a model

  3. Orchestrator

    prompt + context

  4. MCP tools

    web search

  5. Retrieval

    vector DB

  6. Answer

    streamed back

step 1 of 6

A question arrives

The client sends the user's message with the conversation id, so earlier turns can be pulled into context.

{ "conversation": "c_19", "message": "What changed in the refund policy this month?" }
Step through one request as it moves through the platform. The trace snippets are illustrative, written to show each stage's job; they are not production logs.

The problem

Different kinds of questions need different models, prompts and context. A single general prompt to a single model left answers generic and, for anything time-sensitive, out of date.

The platform needed to send each request to the right model with the right context, let models call tools for fresh information, and ground answers in the company's own data.

My role · Architected the platform and its retrieval pipelines.

How it works

  1. Model routing

    Each request is classified and routed to one of several fine-tuned models.

  2. Prompt orchestration

    Categorised prompt templates combine with managed conversation context for that model.

  3. Tool calling over MCP

    Models call tools through a custom MCP server, including real-time web search, with room to plug in new tools.

  4. Retrieval-augmented generation

    Answers are grounded in relevant chunks retrieved from vector databases before the response streams back.

Key decisions

  • Route across multiple fine-tuned models

    instead of: One general-purpose model for everything

    Matching request categories to specialised models and prompts keeps answers on-topic for each kind of question.

  • A custom MCP server for tools

    instead of: Hard-coding each tool into the application

    MCP gives tools a standard interface, so new integrations plug in without changing the orchestration code.

  • Ground answers with RAG

    instead of: Relying on what the model already knows

    Retrieval over the company's own data is what moved response relevance and accuracy by 25%.

Outcomes

  • +25% response relevance and accuracy from the RAG pipelines.
  • Extensible tool integration through MCP, starting with real-time web search.

Stack

  • LangChain
  • MCP
  • Vector databases
  • RAG

Built at Pure Billion Technologies; client details abstracted.