node.js - 如何在 Google Cloud Function 中通过 HTTP 响应发送大文件

Question

我试过下面的代码，但 Google Cloud Function 的响应限制只有 10M，但我想返回更大的文件:

const csv = json2csv(onlyDataTransactions);
response.setHeader(
  "Content-disposition",
  "attachment; filename=transactions.csv"
);
response.set("Content-Type", "text/csv");
response.status(200).send(csv);

更新:感谢@Andrew，我第一次更新了代码，我强制压缩，因为 firebase 云函数上的压缩中间件甚至依赖于用户代理 header ，我仍在努力寻找其他建议以找到最佳结果，谢谢给所有人。

if (request.headers['content-type'] === 'text/csv') {
    const onlyDataTransactions = transactions.map(transaction => transaction.toCsvRecord());
    const csv = parse(onlyDataTransactions);
    response.setHeader(
       "Content-disposition",
       "attachment; filename=transactions.csv"
    );
    response.set("Content-Type", "text/csv");
    response.set('Content-Encoding', 'gzip');
    const content = await gzip(JSON.stringify(csv));
    response.status(200).send(content);
}

Answer 1

您要传输的 .csv 有多大？

Documentation :

Note: Cloud Functions limits HTTP request body sizes to 10MB, so any request larger than this will be rejected before your function is executed. We recommend uploading large files or files that persist beyond a single request directly to Cloud Storage

基于此，我将通过添加 Content-Encoding 将文件与 gzip 压缩状态相互转换，从而对 .csv 进行一些压缩。如果它是 .csv 文件，您应该获得良好的压缩率。

Content-Type: text/plain
Content-Encoding: gzip

如果您使用的是 Google 云存储 (GCS)，您可以在 GCS 中阅读有关转码的更多信息 documentation .

gzip is a form of data compression: it typically reduces the size of a file. This allows the file to be transferred faster and stored using less space than if it were not compressed. Compressing a file can reduce both cost and transfer time. Transcoding, in Cloud Storage, is the automatic changing of a file's compression before it's served to a requester. When transcoding results in a file becoming gzip-compressed, it can be considered compressive, whereas when the result is a file that is no longer gzip-compressed, it can be considered decompressive. Cloud Storage supports the decompressive form of transcoding.

Mozilla 上也有极好的资源关于设置 header 以压缩和解压缩媒体的值得一读的网站。

如果这对您不起作用，那么您唯一真正的选择是以某种方式分块数据或在文件仍在 GCS 存储桶中时压缩文件。我在 gzip compression levels 上取得了成功过去减少文件大小。如果需要，可以使用另一个 Cloud Function 来完成压缩。