Building an AI Chatbot with LangChain and OpenAI API

Why LangChain

LangChain provides ready-made abstractions for conversation memory, tool calling, and chaining multiple LLM calls together, which saves significant boilerplate compared to hand-rolling everything against the raw API.

Installing Dependencies

pip install langchain langchain-openai

A Basic Conversational Chatbot

from langchain_openai import ChatOpenAI
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

llm = ChatOpenAI(model="gpt-4o", temperature=0.7)
memory = ConversationBufferMemory()

conversation = ConversationChain(llm=llm, memory=memory)

response = conversation.predict(input="Hi, I'm building a support bot.")
print(response)

response = conversation.predict(input="What did I just tell you I'm building?")
print(response)

Adding Tool Calling

Real chatbots often need to look things up rather than just chat. Here’s how to give the model access to a simple tool:

from langchain.tools import tool
from langchain.agents import create_tool_calling_agent, AgentExecutor

@tool
def get_order_status(order_id: str) -> str:
    """Look up the status of an order by ID."""
    return f"Order {order_id} is currently in transit."

agent = create_tool_calling_agent(llm, [get_order_status], prompt)
executor = AgentExecutor(agent=agent, tools=[get_order_status])

result = executor.invoke({"input": "Where is order 4821?"})
print(result["output"])

Managing Conversation Length

Unbounded memory eventually blows past the context window. Use ConversationSummaryBufferMemory to automatically summarize older turns while keeping recent messages verbatim.

Adding Guardrails

• Constrain the system prompt to the bot’s actual scope of responsibility.
• Validate tool inputs before executing them — never let the model call sensitive tools unchecked.
• Log every conversation for later review and fine-tuning candidate collection.

Deploying the Chatbot

Wrap the chain in a FastAPI endpoint, stream responses using server-sent events for a responsive UI, and keep memory scoped per user session rather than globally.

Conclusion

LangChain removes most of the plumbing around memory and tool orchestration, letting you focus on the actual conversation design. Start with a simple buffer memory, add tools incrementally, and invest in logging early — it’s your best source of data for improving the bot over time.