Skip to content

Commit

Permalink
docs: Add streaming AI example. (#51382)
Browse files Browse the repository at this point in the history
  • Loading branch information
leerob committed Jun 16, 2023
1 parent 04cd1fd commit f7533e0
Showing 1 changed file with 54 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,60 @@ export async function GET(request, { params }) {

### Streaming

Streaming is commonly used in combination with Large Language Models (LLMs), such an OpenAI, for AI-generated content. Learn more about the [AI SDK](https://sdk.vercel.ai/docs).

```ts filename="app/api/completion/route.ts" switcher
import { Configuration, OpenAIApi } from 'openai-edge'
import { OpenAIStream, StreamingTextResponse } from 'ai'

const config = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
})
const openai = new OpenAIApi(config)

export const runtime = 'edge'

export async function POST(req: Request) {
const { prompt } = await req.json()
const response = await openai.createCompletion({
model: 'text-davinci-003',
stream: true,
temperature: 0.6,
prompt: 'What is Next.js?',
})

const stream = OpenAIStream(response)
return new StreamingTextResponse(stream)
}
```

```js filename="app/api/completion/route.js" switcher
import { Configuration, OpenAIApi } from 'openai-edge'
import { OpenAIStream, StreamingTextResponse } from 'ai'

const config = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
})
const openai = new OpenAIApi(config)

export const runtime = 'edge'

export async function POST(req) {
const { prompt } = await req.json()
const response = await openai.createCompletion({
model: 'text-davinci-003',
stream: true,
temperature: 0.6,
prompt: 'What is Next.js?',
})

const stream = OpenAIStream(response)
return new StreamingTextResponse(stream)
}
```

These abstractions use the Web APIs to create a stream. You can also use the underlying Web APIs directly.

```ts filename="app/api/route.ts" switcher
// https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#convert_async_iterator_to_stream
function iteratorToStream(iterator: any) {
Expand Down

0 comments on commit f7533e0

Please sign in to comment.