Any way to integrate with tRPC?

[Bartek532]

Is there any way to integrate tRPC API with hooks like useChat or useCompletion?

[ibenilsen]

I was running into the same issue and just figured out a working solution to this, though there are probably improvements that can be made.

TRPC mutation

    .mutation(async function* ({ input }) {
      try {
        const result = await streamText({
          model: openai("gpt-4o-mini"),
          messages: convertToCoreMessages(
            input.messages as Parameters<typeof convertToCoreMessages>[0],
          ),
        });

        for await (const text of result.fullStream) {
          yield text;
        }
      } catch (error) {
        console.log(error);
      }
    }),

Since TRPC returns an iterator for the SSE batch events, on your route handler where you are parsing the TRPC response you can make it usable by the hook again by converting it to a ReadableStream and then utilizing the LangChainAdapter from "ai".

    const result = await client.chat.mutate({
      messages,
    });

    const stream = new ReadableStream({
      async start(controller) {
        for await (const text of result) {
          if (text?.type === "finish") {
            controller.close();
            break;
          } else if (text?.type === "text-delta") {
            controller.enqueue(text.textDelta);
          }
        }
      },
    });

    return LangChainAdapter.toDataStreamResponse(stream);

Note that it is still required to have a separate handler since the useChat hook requires an api string. I am using NextJS so I just used the built in API but it is also possible to use express alongside TRPC on your server if it comes to that.

You also can iterate over the textStream instead of fullStream to remove some of the extra logic if you don't need the extra data.

[Bartek532]

Thanks! I also came up with something similar; my concern was that I was unable to pass the response from the tRPC route to AI SDK hooks directly. Declaring another API route doesn't seem like a perfect solution for me 😕

[ibenilsen]

Makes sense. From what I can tell this would require a change in TRPC and how they handle transformations.

My next attempt is going to be to use https://github.com/jlalmes/trpc-openapi which allows creation of procedures that behave like rest endpoints and skips the transformers, so in theory it should work but I'll report back after I try.

[Bartek532]

Please, let me know about the results. It's still not the best solution I can imagine, most probably I'll just Vercel AI SDK standalone for the AI and leave tRPC for everything else.