Building a GPT-3 app with Next.js and streaming (historical)

Note: This post is historical and describes Edge Functions as they existed at the time. For new projects, use Vercel Functions with the Node.js runtime and Fluid compute.

The field of artificial intelligence continues to take the world by storm. Huge strides have been made in text and image generation through tools like ChatGPT, GPT-3, DALL-E, and Stable Diffusion. It’s spawned a wave of exciting AI startups, many of which we’re seeing built with Vercel and Next.js.

One of the most exciting developments in the AI space is GPT-3, a cutting-edge natural language processing model developed by OpenAI. With its ability to understand and generate human-like text, GPT-3 has the potential to disrupt how we perform many of our tasks.

This historical walkthrough uses twitterbio.com to show how streaming changed the user experience for GPT-3 apps. For new projects, build this pattern with Vercel Functions, Fluid compute, and streaming responses.

Copy link to headingThe frontend

The Next.js frontend consists of a few elements:

• A text box for users to copy their current bio or write a few sentences about themselves

• A dropdown where they can select the tone of the bio they want to be generated

• A submit button for generating their bio, which when clicked calls an API route that uses OpenAI’s GPT-3 model and returns two generated bios

• Two containers to display the generated bios after we get them back from the API route

Here’s what the code for our index page looks like. We have a few pieces of state that correspond to the elements mentioned above. We’re also defining a prompt—like ChatGPT, we need to send a prompt to GPT-3 to instruct it to generate the new bios. Finally, we ask GPT-3 to generate two bios clearly labeled (so we can parse them correctly) using the user-provided bio and vibe as context.

1
const prompt = `Generate 2 ${vibe} twitter bios with no hashtags and clearly labeled "1."
2
 and "2.". Make sure each generated bio is at least 14 words and at max 20 words and base 
3
 them on this context: ${bio}`;

The rest of our index page is comprised of the UI elements themselves: our text box, dropdown, submit button, and two containers on the bottom that we display when we get the generated bios. There's also some loading logic for the button to show a loading indicator when clicked.

1
<textarea
2
  value={bio}
3
  onChange={(e) => setBio(e.target.value)}
4
  rows={4}
5
  className="..."
6
  placeholder={"e.g. Senior Engineer @vercel. Tweeting about web dev & AI."}
7
/>
8
<div className="...">
9
  <Image src="/2-black.png" width={30} height={30} alt="1 icon" />
10
  <p className="...">Select your vibe.</p>
11
</div>
12
<div className="block">
13
  <DropDown vibe={vibe} setVibe={(newVibe) => setVibe(newVibe)} />
14
</div>
15
<button className="..." onClick={(e) => generateBio(e)}>
16
  Generate your bio &rarr;
17
</button>
18
<hr className="..." />
19
<div className="...">
20
  {generatedBios && (
21
    <>
22
      <div>
23
        <h2 className="...">Your generated bios</h2>
24
      </div>
25
      <div className="...">
26
        {generatedBios
27
          .substring(generatedBios.indexOf("1") + 3)
28
          .split("2.")
29
          .map((generatedBio: any) => {
30
            return (
31
              <div className="..." key={generatedBio}>
32
                <p>{generatedBio}</p>
33
              </div>
34
            );
35
          })}
36
      </div>
37
    </>
38
  )}
39
</div>

In addition to the UI elements and the loading logic, we have a generateBio function that’s called when the user clicks the submit button. This sends a POST request to our /api/generate API route with the prompt in the body.

1
const generateBio = async (e: any) => {
2
  e.preventDefault();
3
  setGeneratedBios("");
4
  setLoading(true);
5
  const response = await fetch("/api/generate", {
6
    method: "POST",
7
    headers: {
8
      "Content-Type": "application/json",
9
    },
10
    body: JSON.stringify({
11
      prompt,
12
    }),
13
  });
14
  if (!response.ok) {
15
    throw new Error(response.statusText);
16
  }
17
  let answer = await response.json();
18
  setGeneratedBios(answer.choices[0].text);
19
  setLoading(false);
20
};

We get the generated bios back from the API route, save it to the generatedBios state, then display it to the user. Because we asked GPT-3 to return the text in a specific numbered format, we can split it based on the “2.” to show the user the two bios separated nicely into containers as seen below.

Copy link to headingThe backend

A great advantage of using Next.js is being able to handle both our frontend and backend in a single application. We can spin up an API route just by creating a file called generate.ts in our api folder. Let’s take a look at our /api/generate API Route.

We get the prompt from the request body that’s passed in on the frontend, then construct a payload to send to OpenAI. In this payload, we specify some important information like the exact model (GPT-3) and how many tokens we want OpenAI to respond with (a token is approximately 4 characters). In this case, we’re limiting the max tokens because Twitter bios have a character constraint.

After the payload is constructed, we send it in a POST request to OpenAI, await the result to get back the generated bios, then send them back to the client as JSON.

1
export default async function handler(req, res) {
2
  const { prompt } = req.body;
3

4
  const payload = {
5
    model: "text-davinci-003",
6
    prompt,
7
    temperature: 0.7,
8
    top_p: 1,
9
    frequency_penalty: 0,
10
    presence_penalty: 0,
11
    max_tokens: 200,
12
    n: 1,
13
  };
14

15
  const response = await fetch("https://api.openai.com/v1/completions", {
16
    headers: {
17
      "Content-Type": "application/json",
18
      Authorization: `Bearer ${process.env.OPENAI_API_KEY ?? ""}`,
19
    },
20
    method: "POST",
21
    body: JSON.stringify(payload),
22
  });
23

24
  const json = await response.json();
25
  res.status(200).json(json);
26
}

There we have it! We built the first version of our application. Feel free to check out the code and demo for this approach.

Copy link to headingCurrent guidance: use Vercel Functions with streaming

The original version of this tutorial migrated the example to Edge Functions to get streaming and avoid timeouts. For new AI applications on Vercel, use Vercel Functions with the default Node.js runtime and Fluid compute. This gives you streaming support, longer durations, and full Node.js API compatibility.

If you are rebuilding this example today:

• Keep the API route on the Node.js runtime.

• Stream model output from the route to the client.

• Use Fluid compute for long-running model calls and efficient concurrency.

• Choose a function region close to the upstream AI provider when latency matters.

See Vercel Functions and Streaming Functions for current implementation guidance.

Copy link to headingResources

We hope this walkthrough helps you build incredible GPT-3 powered applications. We’ve already seen several sites built with this template such as Rephraser, GenzTranslator, and ChefGPT—some of which have thousands of users. Visit the Twitter Bio site to see everything we talked about in action, check out our other AI templates, or start optimizing prompts across various models with Vercel's AI Playground.