I shipped my first chat UI in 2023 with raw fetch and a hand-rolled EventSource reader. It worked. It was also 180 lines of stateful spaghetti, and I rewrote it three times in two months because every model provider had a slightly different streaming format. So when I tell you the Vercel AI SDK is the rare library worth the space it takes up, I’m saying it as someone who tried doing it the hard way and lost.
This post is what I actually use after switching to AI SDK v5 across three production apps. Not the marketing page. The bits I keep, the bits I avoid, and what I’d tell my past self the day I started.
The thing that finally clicked: useChat is not a UI library
The biggest mental shift moving from v3 or v4 to v5 is that useChat stopped pretending to know what your messages look like. In older versions, the message type was hardcoded to { role, content } and the streaming protocol was baked in. v5 splits this in two: a transport layer that handles streaming over HTTP, and a UI hook layer that gives you messages, status, sendMessage, and the rest.
If you’ve never seen the pain this fixes, here’s roughly what my v3 client looked like:
// 2024 — manual streaming, no SDK
const [messages, setMessages] = useState<Msg[]>([])
async function sendPrompt(text: string) {
const res = await fetch("/api/chat", { method: "POST", body: text })
const reader = res.body!.getReader()
let buf = ""
for (;;) {
const { done, value } = await reader.read()
if (done) break
buf += new TextDecoder().decode(value)
setMessages([...messages, { role: "assistant", content: buf }])
}
}
You can spot at least two bugs from across the room: the stale closure on messages, and the fact that any partial JSON in value will silently corrupt the UI. I fixed those bugs three times in three different apps before I gave up and used the SDK.
Here’s the v5 equivalent:
import { useChat } from "ai/react"
export function Chat() {
const { messages, input, handleInputChange, handleSubmit, status } = useChat({
api: "/api/chat",
})
return (
<form onSubmit={handleSubmit}>
{messages.map(m => <p key={m.id}>{m.role}: {m.content}</p>)}
<input value={input} onChange={handleInputChange} />
<button disabled={status === "streaming"}>send</button>
</form>
)
}
That’s the whole client. The transport, partial-chunk reassembly, and message state are handled. I haven’t read the source, and that’s the point. I used to read the source of every chat library because every chat library was broken.
The server side: streamText is doing more than it looks
The companion API on the server is streamText. Same ergonomic profile: looks tiny, hides a lot. A real route handler in one of my apps looks roughly like this:
// app/api/chat/route.ts
import { streamText } from "ai"
import { openai } from "@ai-sdk/openai"
export async function POST(req: Request) {
const { messages } = await req.json()
const result = streamText({
model: openai("gpt-4o-mini"),
system: "You are a precise technical writer. Avoid hedging.",
messages,
temperature: 0.3,
})
return result.toDataStreamResponse()
}
The thing that took me a while to internalize: toDataStreamResponse returns the stream in the SDK’s own wire format, not raw SSE. That’s what lets useChat reassemble messages, tool calls, and structured data on the client without me writing a parser. The official docs explain the protocol, but you mostly never need to know it.
What I do need to know: switching providers is a one-line change. I went from OpenAI to Anthropic to Mistral on the same route handler in about ninety seconds by swapping the model import. That alone has saved me an embarrassing amount of time during procurement conversations.
Tools are where v5 stops feeling like a chat library
If you’re using the SDK only for chat UIs, you’re getting maybe 30% of the value. The other 70% shows up the first time you give the model a tool to call.
Here’s a real shape I run in production, slightly de-identified:
import { streamText, tool } from "ai"
import { z } from "zod"
const result = streamText({
model: openai("gpt-4o"),
messages,
tools: {
searchInvoices: tool({
description: "Find invoices by customer and date range.",
parameters: z.object({
customerId: z.string(),
from: z.string().datetime(),
to: z.string().datetime(),
}),
execute: async ({ customerId, from, to }) => {
return await db.invoices.find({ customerId, from, to })
},
}),
},
maxSteps: 5,
})
A few things from real use. First, maxSteps matters. Without it, the model can call a tool, decide it wants another result, and just keep going. I capped it at 5 after a debugging session that started at “why is this request 11 seconds” and ended at “the model called the search function nine times in a row.” Second, Zod parameter schemas are not optional in practice; the model will hallucinate field names if you let it. Third, execute runs server-side, so anything in scope is fair game: database clients, internal API keys, all of it. Be careful about what you expose.
The Vercel team’s AI SDK 5 announcement walks through tool patterns in more detail, including multi-step agents. The summary I’d give a new teammate: tools are great, but write them like you’re writing endpoints, because that’s what they are.
What I don’t use
A few SDK features I see other people reach for that I’ve stayed away from.
I don’t use generateUI and the generative UI features for anything customer-facing. It works fine on demos. In production, I want every component the user sees to exist in my code review, and “the model decided to render a date picker” makes that hard. I cover the cases this is meant to solve with my own structured output and a switch statement in React. Less magical, easier to debug.
I don’t use client-side tool calls outside of internal tools. The SDK lets the model call functions that run in the browser, which is a clever idea and a security nightmare for anything user-facing. Internal admin panel, fine. Public app, route everything through the server.
I rarely use useCompletion. It exists for non-chat single-turn completions, and most of the time streamText plus a one-message array works just as well and keeps my code uniform.
Errors and the retry trap
The thing that took me longest to set up well is error handling. The SDK has reasonable defaults (an error object and an onError callback on useChat), but the trap is at a different level. If your model provider returns a 429 mid-stream, you get a partial assistant message in the UI and a stream that just stops. That looks like a normal “model decided to be brief” outcome, not a transient error.
My current pattern is to wrap streamText calls with retry on 429 and 5xx, but only before any tokens have been sent to the client. Once the first token goes out, retrying duplicates content. The SDK gives you the raw provider error via onError, so I tag the response with a request ID and surface a retry button in the UI for any post-first-token failure. Cheaper than trying to be clever about resuming.
I also log every tool call’s input and output to a separate table. If something goes wrong, I can replay the conversation against a fresh model without reproducing user input. The first time a customer asked “what did the AI tell me yesterday at 3pm?” I had no answer. Now I do.
The integration thing nobody warns you about
If you’re using Next.js App Router, the AI SDK route handlers play well with server actions and middleware, but there’s a sharp edge that bit me twice. Streamed responses don’t go through Next.js middleware the way you might expect. The body stream is opaque to most middleware, so per-request rewrites that work for normal API routes silently no-op. I write about this kind of Next.js server-side mistake because I keep making them. If you need auth on your AI routes (you do), check it in the handler, not in middleware.
I cover most of my AI integration work in my portfolio, including a few of the patterns above with the customer-specific bits stripped out.
One concrete thing to try this week
If you’re still on AI SDK v3 or v4, the migration to v5 is genuinely small for most apps. The two changes that catch people:
messagesin the new shape is an array of{ id, role, content, parts? }. If you persist messages, your DB rows probably need an extra column or two. Write a migration before the deploy, not after.- The
useChatcallback names changed in v5.onResponseis gone; useonFinishandonError. If you have logging hooked up to the old names, it’s now silently disabled.
Pull v5 into a branch, run your chat UI, and look at the network tab. The wire format is different and that’s the easiest place to confirm the new transport is actually engaging. If the response shows old-style SSE, you’re still on v4 somewhere.
That’s it. The AI SDK isn’t magic. It’s a thin, well-designed wrapper over a problem space that everyone was solving badly. Use it, read the GitHub source when something gets weird, and don’t reach for the generative UI features until you’ve earned them.
