nginx - 使用 Amazon S3 限制 Nginx 的带宽

Question

我在 Amazon S3 上托管了大型下载文件(有些大于 5 GB)。我的主服务器是 Nginx。 Amazon S3 没有公共(public)访问权限。文件通过签名 URL 提供。

使用 Amazon S3 时有没有办法限制带宽？我知道 Amazon S3 上没有选项，但是我们可以使用 Nginx 作为代理并从那里制作它吗？

我正在尝试使用该链接中的示例:

https://coderwall.com/p/rlguog/nginx-as-proxy-for-amazon-s3-public-private-files

此代码块:

location ~* ^/proxy_private_file/(.*) {
  set $s3_bucket        'your_bucket.s3.amazonaws.com';
  set $aws_access_key   'AWSAccessKeyId=YOUR_ONLY_ACCESS_KEY';
  set $url_expires      'Expires=$arg_e';
  set $url_signature    'Signature=$arg_st';
  set $url_full         '$1$aws_access_key&$url_expires&$url_signature';

  proxy_http_version     1.1;
  proxy_set_header       Host $s3_bucket;
  proxy_set_header       Authorization '';
  proxy_hide_header      x-amz-id-2;
  proxy_hide_header      x-amz-request-id;
  proxy_hide_header      Set-Cookie;
  proxy_ignore_headers   "Set-Cookie";
  proxy_buffering        off;
  proxy_intercept_errors on;

  resolver               172.16.0.23 valid=300s;
  resolver_timeout       10s;

  proxy_pass             http://$s3_bucket$url_full;  

}

我不明白的是如何将创建的签名 URL 从 PHP 传递到 Nginx 配置？所以我可以告诉 Nginx 去那个签名的 URL 作为代理。

Answer 1

我找到了解决方案。这里是:

首先在你的 nginx 配置中打开你的 http block 。我们将创建限制每个 IP 连接所需的区域。

limit_conn_zone $binary_remote_addr zone=addr:10m;

现在在/etc/nginx/conf.d/sitename.conf 或任何你定义它的地方打开你的服务器 block 。创建内部位置。我们将 PHP 请求重定向到这里:

location ~* ^/internal_redirect/(.*?)/(.*) {
# Do not allow people to mess with this location directly
# Only internal redirects are allowed
internal;

# Location-specific logging, so we can clearly see which requests
# passing through proxy and what is happening there
access_log /var/log/nginx/internal_redirect.access.log main;
error_log /var/log/nginx/internal_redirect.error.log warn;

# Extract download url from the request
set $download_uri $2;
set $download_host $1;

# Extract the arguments from request.
# That is the Signed URL part that you require to get the file from S3 servers
if ($download_uri ~* "([^/]*$)" ) {
    set  $filename  $1;
}

# Compose download url
set $download_url $download_host/$download_uri?$args;

# Set download request headers
proxy_http_version      1.1;
proxy_set_header        Connection "";
proxy_hide_header       x-amz-id-2;
proxy_hide_header       x-amz-request-id;
proxy_hide_header       Set-Cookie;
proxy_ignore_headers    "Set-Cookie";

# Activate the proxy buffering, without it limiting bandwidth speed in proxy will not work!
proxy_buffering on;

# Buffer 512 KB data
proxy_buffers 32 16k;

proxy_intercept_errors  on;
resolver        8.8.8.8 valid=300s;
resolver_timeout        10s;

# The next two lines could be used if your storage
# backend does not support Content-Disposition
# headers used to specify file name browsers use
# when save content to the disk
proxy_hide_header Content-Disposition;
add_header Content-Disposition 'attachment; filename="$filename"';

# Do not touch local disks when proxying
# content to clients
proxy_max_temp_file_size 0;

# Limit the connection to one per IP address
limit_conn addr 1;

# Limit the bandwidth to 300 kilobytes
proxy_limit_rate 300k;

# Set logging level to info so we can see everything.
# All levels you can set: info | notice | warn | error
limit_conn_log_level info;   

# Finally download the file and send it to client
# Beware that you can shouldn't include "htttp://" or "https://"
# in proxy. Doing that will cause an "invalid port in upstream" error.
proxy_pass $download_url;
}

在 PHP 中画龙点睛，并将您签名的 URL 发送到 Nginx:

header( 'X-Accel-Redirect: ' . '/internal_redirect/' . $YOUR_SIGNED_URL );