How to call handSubmit() and render returned message concurrently

[TranBaVinhSon]

I have a list of models (gpt4, claude...etc) and I want to get the responses from all submitted models at the same time. Even the backend can stream the response back to frontend but it seems like only the last message was rendered on the frontend and added to messages. I noticed that two chats were created. I also tried to add sleep(1000) after handleSubmit() function but it didn't help.

const [models, setModels] = React.useState<string[]>(["gpt4", "claude"]);
const { messages, input, handleSubmit, setInput, setMessages } = useChat();
//....etc
<form
  onSubmit={async (event) => {
    event.preventDefault();
    for (let i = 0; i < models.length; i++) {
      handleSubmit(event, {
        options: {
          body: {
            model: models[i],
          },
        },
      });
     // add await sleep(1000)
    }
  }}
  className="flex w-full items-center space-x-2"
>

[mcavaliere]

What does your API route look like? Can you share the code?

[mcavaliere]

My initial thought would be that if you're using the pages router and want the same input to go to each model, you could have N different API routes (one for each model/API endpoint), and one useChat hook for each. Then in the form's submit handler call each useChat's handleSubmit in parallel.

[TranBaVinhSon]

@mcavaliere Hi, I'm using one API for all models. Here is the code for backend logic, frontend will send the model in the body request
app/api/chats/route.ts

import {
  StreamingTextResponse,
  streamText,
} from "ai";
import { openai } from "@ai-sdk/openai";

// TODO: IMPORTANT! Set the runtime to edge
export const runtime = "nodejs";

export interface ChatRequestBody {
  model: string;
  messages: any;
}

export async function POST(req: Request) {
  const { messages, model } = (await req.json()) as ChatRequestBody;

  return await generateResponseByModel(
    model,
    messages
  );
}

async function generateResponseByModel(
  model: string,
  messages: any
) {
  let stream: ReadableStream<any>;
  if (model === "gpt-3.5") {
    const result = await streamText({
      model: openai("gpt-3.5-turbo"),
      messages,
    });
    stream = result.toAIStream();
  } else if (model === "gemini") {
  } else if (model === "claude") {
  } else {
    const result = await streamText({
      model: openai("gpt-4o-2024-05-13"),
      messages,
    });
    stream = result.toAIStream();
  }

  return new StreamingTextResponse(stream);
}

[mcavaliere]

@TranBaVinhSon I'd try having one useChat per model then. Each usage of useChat has its own internal state, and looking at the source it does not appear to be made to work with more than one stream at a time.

https://github.com/vercel/ai/blob/main/packages/core/react/use-chat.ts