Day 3: Streaming Chat Responses — React + AI

How Streaming Works

Instead of waiting for the complete response, the server streams tokens as they're generated. The browser receives a stream of server-sent events (SSE) and updates the UI incrementally.

The flow: user sends message → server starts streaming Claude's response → browser receives chunks → React appends each chunk to the message in state → full response appears token by token.

Server-Side Streaming

javascript — server.js

app.post('/api/chat/stream', async (req, res) => {
  const { messages } = req.body;

  // Set SSE headers
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  try {
    const stream = client.messages.stream({
      model: 'claude-3-haiku-20240307',
      max_tokens: 1024,
      messages: messages,
    });

    // Send each text delta as an SSE event
    stream.on('text', (text) => {
      res.write(`data: ${JSON.stringify({ text })}\n\n`);
    });

    stream.on('end', () => {
      res.write('data: [DONE]\n\n');
      res.end();
    });

    stream.on('error', (err) => {
      res.write(`data: ${JSON.stringify({ error: err.message })}\n\n`);
      res.end();
    });
  } catch (err) {
    res.write(`data: ${JSON.stringify({ error: err.message })}\n\n`);
    res.end();
  }
});

Reading the Stream in React

jsx

async function sendMessageStream(userText) {
  const userMsg = { id: Date.now(), role: 'user', content: userText };
  setMessages(prev => [...prev, userMsg]);
  setIsLoading(true);

  // Add a placeholder for the streaming response
  const aiId = Date.now() + 1;
  setMessages(prev => [...prev, { id: aiId, role: 'assistant', content: '' }]);

  try {
    const response = await fetch('/api/chat/stream', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages: [...messages, userMsg] })
    });

    const reader = response.body.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const lines = chunk.split('\n').filter(l => l.startsWith('data: '));

      for (const line of lines) {
        const data = line.slice(6); // remove "data: "
        if (data === '[DONE]') break;

        try {
          const { text } = JSON.parse(data);
          if (text) {
            // Append the chunk to the last message
            setMessages(prev =>
              prev.map(msg =>
                msg.id === aiId
                  ? { ...msg, content: msg.content + text }
                  : msg
              )
            );
          }
        } catch {}
      }
    }
  } finally {
    setIsLoading(false);
  }
}

💡

The key pattern: Create the assistant message with empty content before the stream starts. Then append each chunk to it. This gives React something to update rather than inserting a new message on every chunk.

Auto-Scroll to Latest Message

jsx

import { useRef, useEffect } from 'react';

function ChatMessages({ messages, isLoading }) {
  const bottomRef = useRef(null);

  useEffect(() => {
    bottomRef.current?.scrollIntoView({ behavior: 'smooth' });
  }, [messages]);  // runs whenever messages update

  return (
    
      {messages.map(msg => )}
      {isLoading && }
        {/* invisible element at the bottom */}
    
  );
}

Markdown Rendering

Claude's responses often include markdown — bold, code blocks, lists. Render it properly:

bash

npm install react-markdown

jsx

import ReactMarkdown from 'react-markdown';

function MessageCard({ role, content }) {
  return (
    
      {role === 'assistant'
        ? {content}
        : {content}
      }
    
  );
}

📝 Exercise

Implement Streaming Responses

Add the /api/chat/stream endpoint to your Express server.
Update App.jsx to call the streaming endpoint instead of the regular one.
Add a placeholder message with empty content before the stream starts.
Read the stream and append each chunk to the placeholder message.
Add the auto-scroll hook using useRef and useEffect.
Install react-markdown and render Claude's responses as markdown.

Lesson Summary

Streaming sends SSE events as tokens are generated. Set Content-Type: text/event-stream on the server.
Create a placeholder message before streaming starts, then append each chunk to it using setMessages with a map.
Use useRef + scrollIntoView to auto-scroll to the latest message.
react-markdown renders Claude's markdown formatting correctly.

Challenge

Add a 'Stop Generation' button that appears during streaming. When clicked, cancel the fetch request using AbortController and leave the partial response in the chat.

←

← Previous

Day 2: Calling the Claude API

Day 4: RAG with File Uploads

→

Streaming: Responses That Appear Word by Word

How Streaming Works

Server-Side Streaming

Reading the Stream in React

Auto-Scroll to Latest Message

Markdown Rendering

Lesson Summary