Building a Simple Chatbot Using an LLM API

Building a working chatbot using an LLM API takes surprisingly little code — the real engineering effort in a production chatbot goes into context management and guardrails, not the basic conversational loop itself.

import anthropic

client = anthropic.Anthropic(api_key="your-api-key")
conversation_history = []

def chat(user_message):
    conversation_history.append({"role": "user", "content": user_message})

    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        system="You are a helpful customer support assistant for a software company.",
        messages=conversation_history,
    )

    assistant_reply = response.content[0].text
    conversation_history.append({"role": "assistant", "content": assistant_reply})
    return assistant_reply

print(chat("How do I reset my password?"))
print(chat("What if I don't have access to my email anymore?"))  # remembers prior context

What Turns This into a Genuinely Production-Ready Chatbot

• Context window management — trimming or summarizing older conversation turns before they exceed the model’s context limit
• Grounding with real, current data — retrieval (RAG) so answers reflect your actual, current product/policy details, not just the model’s general pretrained knowledge
• Guardrails — explicit handling for off-topic requests, and clear escalation to a human for genuinely sensitive or high-stakes situations
• Cost management — genuinely monitoring and controlling token usage, since a naive conversation history keeps growing (and costing more) with every single turn