Day 4: RAG: Answer Questions from Your Own Documents | Build an AI Chatbot

What RAG Is

RAG works in two steps: first, find the relevant parts of your documents that match the user's question (retrieval). Second, include those parts in the prompt and ask Claude to answer based on them (generation). The simplest implementation uses text search rather than vector embeddings — good enough for most use cases.

Terminal — Install Dependencies

npm install natural

rag.js — Document Store and Retrieval

// Simple keyword-based retrieval
class DocumentStore {
  constructor() {
    this.chunks = [];
  }
  
  addDocument(text, metadata = {}) {
    // Split into ~500 character chunks with overlap
    const words = text.split(/\s+/);
    const chunkSize = 80; // words
    const overlap = 20;
    
    for (let i = 0; i < words.length; i += (chunkSize - overlap)) {
      const chunk = words.slice(i, i + chunkSize).join(' ');
      if (chunk.trim()) {
        this.chunks.push({ text: chunk, metadata });
      }
    }
  }
  
  retrieve(query, topK = 3) {
    const queryWords = new Set(
      query.toLowerCase().split(/\W+/).filter(w => w.length > 3)
    );
    
    const scored = this.chunks.map(chunk => {
      const chunkWords = new Set(
        chunk.text.toLowerCase().split(/\W+/).filter(w => w.length > 3)
      );
      let score = 0;
      for (const word of queryWords) {
        if (chunkWords.has(word)) score++;
      }
      return { ...chunk, score };
    });
    
    return scored
      .sort((a, b) => b.score - a.score)
      .slice(0, topK)
      .filter(c => c.score > 0);
  }
}

module.exports = DocumentStore;

server.js — Add RAG to the /chat Route

const DocumentStore = require('./rag');
const docStore = new DocumentStore();

// Load your documents at startup
docStore.addDocument(`
  Your company knowledge base text goes here.
  This can be product documentation, FAQs,
  policies, or any text you want the bot to
  know about. Add as much as you need.
`, { source: 'knowledge-base' });

// Update the /chat route to use RAG
app.post('/chat', async (req, res) => {
  const { message, sessionId } = req.body;
  
  if (!conversations[sessionId]) {
    conversations[sessionId] = [];
  }
  
  // Retrieve relevant chunks
  const chunks = docStore.retrieve(message, 3);
  const context = chunks.length > 0
    ? 'Relevant information:
' + chunks.map(c => c.text).join('

')
    : '';
  
  conversations[sessionId].push({
    role: 'user',
    content: message
  });
  
  const systemPrompt = `You are a helpful assistant.
${context ? context + '

Use the above information to answer if relevant.' : ''}
If you don't know something, say so honestly.`;
  
  const response = await client.messages.create({
    model: 'claude-opus-4-5',
    max_tokens: 1024,
    system: systemPrompt,
    messages: conversations[sessionId]
  });
  
  const reply = response.content[0].text;
  conversations[sessionId].push({ role: 'assistant', content: reply });
  
  res.json({ reply, sourcesFound: chunks.length });
});

Day 4 Exercise

Build RAG for Your Use Case

Add rag.js to your project.
Update server.js to import DocumentStore and add the RAG system prompt.
Add some real content to docStore.addDocument() — use an actual FAQ, product description, or policy document.
Ask questions that are in your documents. Verify the chatbot answers from the content you provided.
Ask questions NOT in your documents. Verify it says it does not know rather than hallucinating.

Day 4 Summary

RAG = retrieve relevant text + generate answer based on that text. Two steps, one prompt.
Simple keyword retrieval works for most use cases — no vector database required to start.
The system prompt dynamically includes retrieved context only when relevant.
Grounded chatbots say "I don't know" instead of hallucinating — this builds user trust.