javascript - 如何 stub /模拟来自 ChatGPT 的流式响应？

Question

在我的测试环境中，我尝试拦截我对 ChatGPT 的请求(如下所示)并将其替换为模拟响应。 fetch api 返回 Response.body作为ReadableStream的内容。在我的实现中，我使用 ReadableStream.getReader()它创建一个读取器并将流锁定到它。当“some-condition”为真时，我们等待返回一系列对象的 reader.read() 方法(请参阅下面的 console.log() 屏幕截图)。它们的值被解码，然后传递到解析器，解析器处理最终显示在 UI 中的解析内容。

由于直接与 ChatGPT api 交互会导致不稳定的 e2e 测试，我试图弄清楚如何正确地 stub /模拟来自 /api/chat-stream 的响应是可消耗的并且形状正确，以便在我的集成测试中呈现虚假响应。我已经分享了最终将用于拦截请求的代码片段，但我对如何正确实现它感到困惑。

// this should intercept the request and provide the mock/stub
await page.route(/.*\/api\/chat-stream/, async (route) => {
  await route.fulfill({
    status: 200,
    headers: {
      "some-headers"
    },
    body: someBody,
  });
});

向chatgpt请求

// localhost:3000/api/chat-stream

export async function POST(request: Request) {
  const { messages } = await request.json();

  const completion = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    body: JSON.stringify({
      model: "gpt-3.5-turbo",
      messages: messages,
      stream: true,
    }),
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${process.env.NEXT_PUBLIC_OPENAI_API_KEY}`,
    },
  });

  return new Response(completion.body, {
    // completion.body is a readable stream as detailed above
    // ReadableStream { locked: false, state: 'readable', supportsBYOB: false }
    status: 200,
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
  });
}

触发并处理上述请求的 API 调用

export const handleStreamMessage = async (messages: any) => {
  const response = await fetch("/api/chat-stream", {
    method: "POST",
    body: JSON.stringify({
      messages: messages,
    }),
    headers: {
      "Content-Type": "application/json",
    },
  });

  const reader = response.body?.getReader();
  const decoder = new TextDecoder();

  const onParse: EventSourceParseCallback = (event) => {
    if (event.type === "event") {
      try {
        const data: { choices: { delta: { content: string } }[] } = JSON.parse(
          event.data
        );

        // filter for chatgpt "deltas" with content
        data.choices
          .filter(({ delta }) => !!delta.content)
          .forEach(({ delta }) => {
            // do something in react
            setCurrentMessage((prev) => {
              return `${prev || ""}${delta.content}`;
            });
          });
      } catch (error) {
        console.log("error", error);
      }
    }
  };

  const parser = createParser(onParse);

  if (reader) {
    while (some-condition) {
      const readOperation = await reader.read();
      console.log("readOperation", readOperation);

      const dataString = decoder.decode(readOperation.value);

      if (readOperation.done || dataString.includes("[DONE]")) {
        break;
      }

      parser.feed(dataString);
    }
  }
};

Answer 1

好消息，我解决了我的问题。

我最终做的是复制控制台记录的响应之一(见上文)，解析仅捕获值的对象，将这些值传递到 Uint8Array ，然后创建了一个Buffer从那个来源。然后该缓冲区被传递到我的 route.fulfill 的主体中方法万岁，我现在已经用有效的缓冲区(我从他们那里获取)拦截了 chatgpt api 响应。

// transformation
const rawObject = {
  "0": 100,
  "1": 97,
  "2": 116,
  "3": 97,
  ...
};

const parsedArray = Object.values(rawObject); 
const myBuffer: Buffer = Buffer.from(new Uint8Array(parsedArray));

// playwright route interception
await page.route(/.*\/api\/chat-stream/, async (route) => {
  await route.fulfill({
    status: 200,
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
    body: myBuffer,
  });
});