Creating a Generative AI Chatbot in Just Two Hours with AWS
Written on
Chapter 1: Introduction to Generative AI Chatbots
In recent times, leveraging generative AI for business applications has become increasingly vital. One of the most effective implementations is using a large language model (LLM) chatbot as an intuitive user-facing knowledge base.
Imagine your colleagues engaging with an intelligent chatbot that provides information on policies and procedures, instead of sifting through countless pages in traditional document repositories like Confluence or SharePoint.
Humans thrive on conversation. It's how we share ideas and form connections, and in this context, it’s also a natural way to learn. Thus, the concept of interacting with data through dialogue is incredibly appealing, and no technology has demonstrated this potential better than the rise of LLMs and chatbots.
"Conversation is an exchange of thoughts that leaves all parties a grain wiser." — Alfred Whitney
However, the prospect of creating an LLM chatbot can appear overwhelming, especially for those not well-versed in machine learning concepts. Terms like embeddings, cosine similarity, and vector databases can seem daunting.
Rest assured, these are crucial elements in developing and maintaining large language models, particularly in the realm of retrieval-augmented generation (RAG). While in-depth explorations of these topics can be found elsewhere, this article aims to simplify the process. We will focus on employing AWS Bedrock—a fully managed generative AI service—and a touch of Python to quickly build a basic LLM-backed chatbot.
What You'll Need
- An AWS Account with access to AWS Bedrock and the Claude model family (details to follow).
- An S3 bucket for storing your knowledge data.
- A Streamlit account (don’t worry, we’ll explain this).
- A GitHub account to host your code.
- Basic Python programming skills (it's simpler than it sounds!).
- One or more PDF or text files that will serve as your knowledge base.
Note: The code we will utilize is derived from the official AWS tutorial for establishing a Bedrock Knowledge Base. However, we will take a slightly different approach by creating a front-end application hosted on Streamlit.io and implementing a basic API integration to separate our backend from the frontend.
Are you ready? Let’s dive in!
Section 1.1: Setting Up Your Knowledge Base
"Human behavior flows from three main sources: desire, emotion, and knowledge." — Plato
Our journey begins at AWS Bedrock, Amazon’s fully managed generative AI service. Here, you can configure ML models for text or image generation and utilize playgrounds for testing. AWS Bedrock serves as the foundation of our generative AI project.
Access Bedrock through the AWS console, remembering that it's currently available in select regions: Europe (Frankfurt), US West (Oregon), Asia Pacific (Tokyo), Asia Pacific (Singapore), and US East (N. Virginia).
For this tutorial, we will be working in the us-west-2 (Oregon) region. As with all generative AI projects, we need to base our work on a suitable LLM. AWS offers various open-source model families, but we will use the Claude model from Anthropic, as it integrates seamlessly with the Bedrock Knowledge Base service for staging our information.
Let's verify access to the Claude v2.1 model in your Bedrock console. If you find it unavailable, follow the instructions to request access.
The foundation model allows our chatbot to generate human-like responses. However, our chatbot also needs to be knowledgeable. To achieve this, we will create an AWS Bedrock Knowledge Base.
Think of the Claude model as a friendly, polite individual lacking knowledge. The Knowledge Base acts as a reference book that Claude can memorize to provide accurate answers.
Setting Up the Knowledge Base
For our demonstration, we'll use a PDF e-book on Kubernetes. Our goal is to develop a Kubernetes chatbot to assist users in understanding the fundamentals of container orchestration. If Kubernetes isn't your area of interest, you can choose any material suitable for your Knowledge Base.
Place the documents into the S3 bucket created for this demo, such as a bucket named 'kubernetes-knowledge'.
Next, let’s open the AWS Bedrock console. First, ensure that the Claude v2.1 model is enabled. If not, follow the on-screen prompts to enable it.
Select "Knowledge Base" under the Orchestration submenu. You will be guided through a multi-step setup process, starting with indicating your S3 URI (file or folder). After that, you’ll select the embedding model—let's choose Amazon's "Titan" embedding model for this demo.
Embeddings convert text from your files into numerical representations that machine learning can utilize. They describe not only individual words but also their interrelationships.
One of my favorite parts of this process involves the vector database. If embeddings are the secret ingredient in large language models, the vector database is the pot where everything is mixed together. Fortunately, AWS Bedrock simplifies this setup with a one-click configuration.
You can opt for an existing vector store, or, as we will, allow Bedrock to establish an AWS OpenSearch serverless service tailored for our data.
Once you review all your settings and click "Create," your Knowledge Base will be ready in a few minutes.
You can test your setup using the console's integrated chat interface. Select your Knowledge Base and click "Test Knowledge Base" to initiate a chat.
Look at that! We now have a functional chat interface for our data. However, it's currently limited to the AWS Console. How do we make it accessible to a wider audience? This is where Streamlit comes into play.
Section 1.2: Introduction to Streamlit
If you're not frequently building machine learning applications, you might be wondering, "What is Streamlit?" Streamlit is an open-source Python library adored by data scientists and machine learning practitioners for its simplicity in creating and sharing data-driven web applications.
With Streamlit, you can develop web applications entirely in Python, eliminating the need to work with HTML, CSS, JavaScript, or other front-end languages. Streamlit abstracts much of that complexity, allowing you to create interactive dashboards and chat interfaces with ease.
This library's simplicity and rapid deployment capabilities make it the ideal choice for our "chatbot in 2 hours" initiative.
AWS has published an excellent repository that guides you through setting up a knowledge base-backed application. While this tutorial can be followed, my approach involves deploying all resources to the cloud instead of running the Streamlit app locally, enhancing its functionality.
The Lambda Function
As good software developers, we start by sketching out our architecture. The beauty of our setup is that the backend Lambda service has already been configured in previous steps.
Our focus now shifts to the Streamlit application and a Lambda function that will serve as the backend, interfacing with the AWS services we are utilizing.
The Lambda function's role is straightforward. It will receive user queries from the Streamlit front end and relay them to the Bedrock service for responses.
To make the Lambda accessible from our Streamlit application, we will utilize a feature called a function URL. This creates a simplified API that can be accessed via a URL through standard HTTP requests.
Here’s a look at the code that will make this work:
import os
import boto3
import json
boto3_session = boto3.session.Session()
region = boto3_session.region_name
# Create a boto3 Bedrock client
bedrock_agent_runtime_client = boto3.client('bedrock-agent-runtime')
# Get knowledge base ID from environment variable
kb_id = os.environ.get("KNOWLEDGE_BASE_ID")
# Declare model ID for calling RetrieveAndGenerate API
model_id = "anthropic.claude-v2:1"
model_arn = f'arn:aws:bedrock:{region}::foundation-model/{model_id}'
def retrieveAndGenerate(input, kbId, model_arn, sessionId=None):
if sessionId != "None":
return bedrock_agent_runtime_client.retrieve_and_generate(
input={
'text': input},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': kbId,
'modelArn': model_arn
}
},
sessionId=sessionId
)
else:
return bedrock_agent_runtime_client.retrieve_and_generate(
input={
'text': input},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': kbId,
'modelArn': model_arn
}
}
)
def lambda_handler(event, context):
try:
body = json.loads(event['body'])
query = body["question"]
session_id = body["sessionid"]
response = retrieveAndGenerate(query, kb_id, model_arn, session_id)
generated_text = response['output']['text']
return {
'statusCode': 200,
'body': {"question": query.strip(), "answer": generated_text.strip()}
}
except Exception as e:
return {
'statusCode': 400,
'body': json.dumps(str(e))
}
Let's break down the key components of our Lambda function. We begin by setting environment variables, including the Knowledge Base ID and the model ID we selected.
The Lambda function will accept an event containing the user's question, which will be sent from the Streamlit front end. This payload will include the question and a session ID to maintain conversation context.
The function will pass the query to the Bedrock service, which will return a coherent response.
Next, we will look at the Serverless Application Model (SAM) template we’ll use for deployment:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Backend lambda stack for communicating with Bedrock
Parameters:
KnowledgeBaseId:
Type: String
Description: The ID of the knowledge base.
Resources:
KubeKbFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: kuberknowitall-bedrock-handler
CodeUri:
Handler: handler.lambda_handler
Runtime: python3.12
Timeout: 25
MemorySize: 1024
Policies:
AWSLambdaBasicExecutionRole
Version: '2012-10-17'
Statement:
Effect: Allow
Action:
- bedrock:RetrieveAndGenerate
Resource: !Sub "arn:aws:bedrock:${AWS::Region}:${AWS::AccountId}:knowledge-base/${KnowledgeBaseId}"
Environment:
Variables:
KNOWLEDGE_BASE_ID: !Ref KnowledgeBaseIdFunctionUrlConfig:
AuthType: NONE
Cors:
AllowOrigins:
This template is straightforward, defining a single Lambda function configured for Python with specific permissions. The FunctionUrlConfig section allows for public access, but make sure to set up authentication for production use.
To deploy this Lambda function, we can use the 'sam sync' command, which allows for easy deployment and automatic updates.
Now that we have established our backend and AI services, it’s time to focus on building our front end using Streamlit.
Chapter 2: Building the Streamlit Application
Let's create a simple front end for our chat experience using Streamlit. Here’s the main Python file for the application:
import streamlit as st
import requests
import json
import os
st.set_page_config(page_title="Kuberknowitall")
st.title("Kuberknowitall")
# Sidebar for sample questions
st.sidebar.title("Sample questions to get started")
st.sidebar.markdown("""
- Show me a simple K8s yaml file
""")
# Use a hardcoded session ID
sessionId = "None"
# Initialize chat history
if "messages" not in st.session_state:
st.session_state.messages = []
# Display chat messages from history
for message in st.session_state.messages:
with st.chat_message(message["role"]):
st.markdown(message["content"])
# React to user input
if prompt := st.chat_input("How can I help you?"):
question = prompt
st.chat_message("user").markdown(question)
# Prepare the payload for the HTTP request
payload = {"question": prompt, "sessionid": st.session_state['sessionId']}
# Specify the function URL
function_url = os.environ.get('FUNCTION_URL')
with st.spinner('Kuberknowitall is thinking..shhhh'):
response = requests.post(function_url, json=payload)
if response.status_code == 200:
result = response.json()
answer = result['answer']
sessionId = result.get('sessionId', 'None')
st.session_state['sessionId'] = sessionId
st.session_state.messages.append({"role": "user", "content": question})
with st.chat_message("assistant"):
st.markdown(answer)
st.session_state.messages.append({"role": "assistant", "content": answer})
else:
st.error(response.text)
This code sets up the Streamlit app, laying out the HTML, images, and initializing the chat history. It captures user input, sends it to our Lambda function, and displays the response.
To host this application, upload your front-end code to a GitHub repository. Then, sign in to Streamlit.io and create a new app, linking it to your GitHub repository.
Before deploying, ensure you set an environment variable called FUNCTION_URL that points to your Lambda URL.
Once deployed, your application will be accessible at a designated Streamlit domain. Notably, integrating Streamlit with your GitHub repository enables automatic updates when you merge changes.
After this deployment, you’ll have a live Kubernetes expert chatbot ready for interaction.
Nonetheless, keep a few considerations in mind:
- The Knowledge Base service is strict and sticks closely to source material, making it less flexible with unrelated queries.
- Security considerations for backend authentication and authorization should not be overlooked.
- While great for text-based answers, the response format may require enhancement for structured data types like JSON or tables.
In conclusion, we’ve successfully created a compelling knowledge-based chatbot in around two hours. AWS Bedrock offers a powerful suite of generative AI services, and what we've accomplished here barely scratches the surface of its potential.
I hope you found this guide helpful as we built our Kubernetes know-it-all chatbot together. Please feel free to engage by leaving feedback or comments. Until next time!
The first video titled "Build a Chatbot in 20 min with AWS Bedrock" provides a concise overview of building an AI chatbot using AWS Bedrock.
The second video titled "Build a CHATBOT with AWS Amazon Bedrock- Llama 2, Langchain and Streamlit [HANDS-ON]" offers a hands-on tutorial for constructing a chatbot using AWS and related technologies.