nginx - 先通过nginx慢速连接

Question

我对 nginx 有如下问题:
我的网站的峰值请求是 30000req/sec 和 nginx 服务器的最大流量 ~300Mbps，Nginx 是代理 http 服务器。
有时用户通过nginx连接到我系统中的某个服务器，第一次连接非常慢(>10s)连接到服务器，然后连接到服务器下一个连接非常快(<1s)
nginx配置如下:

user  nginx;
worker_processes  auto;
worker_rlimit_nofile 30000;
error_log  /storage/log/vn-fw/nginx/log/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  10240;
    multi_accept on;
    use epoll;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    proxy_pass_header Server;
    sendfile        on;
    keepalive_timeout  15;
    include /etc/nginx/conf.d/*.conf;
    tcp_nodelay on;
    tcp_nopush  on;

    client_body_buffer_size     1k;
    client_header_buffer_size   4k;
    client_max_body_size          5M;
    large_client_header_buffers 4 16k;
    proxy_buffers 8 16k;
    server_tokens off;
    gzip on;
    gzip_min_length 10240;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;
    gzip_disable "MSIE [1-6]\.";
}

和版本内核:

Linux vn-nginx2 2.6.18-348.3.1.el5 #1 SMP Mon Mar 11 19:39:25 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux

和内核版本

net.ipv4.ip_forward = 0

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route = 0

kernel.sysrq = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.shmmax = 68719476736

kernel.shmall = 4294967296

net.ipv4.ip_conntrack_max = 393216

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 86400

net.ipv4.tcp_window_scaling = 0

请帮我解决这个问题。
非常感谢你

Answer 1

根据内核版本，你有 RHEL5，它带有过时的 TCP/IP 配置堆栈和 TCP 初始拥塞窗口设置为 2/3。

您试图通过 net.ipv4.tcp_window_scaling=0 禁用它，这会导致意外行为并且对性能没有帮助。在继续之前打开它。而不是这个，请注意 的值tcp_wmem 和 tcp_rmem .

看透Wikipedia什么是 TCP 拥塞窗口以及它是如何工作的

Slow-start is part of the congestion control strategy used by TCP, the data transmission protocol used by many Internet applications. Slow-start is used in conjunction with other algorithms to avoid sending more data than the network is capable of transmitting, that is, to avoid causing network congestion. The algorithm is specified by RFC 5681.

Slow-start begins initially with a congestion window Size (cwnd) of 1, 2 or 10.1 The value of the Congestion Window will be increased with each acknowledgment received, effectively doubling the window size each round trip time...

如果我们尝试使用 a diagram 进行成像:



就发送的字节而言，我们可以这样显示:



根据内核版本，TCP 拥塞窗口的值为 2/3，在 RHEL 6.2 中默认增加到 10

In Red Hat Enterprise Linux 6.2, the TCP initial congestion window default is now set to 10, according to RFC 5681. Additionally, the initial-window code common to TCP and CCID-2 has been consolidated.

所以问题仍然悬而未决，该怎么办？答案很简单:升级内核(推荐)或调整窗口值:

ip route change default via `ip route| awk '/^def/{print $3}'` dev eth1 initcwnd 10 initrwnd 10

注意正确设置接口(interface)，它可以是 eth0 或不同的。系统重新启动后，该值被重置，因此请确保您在启动时通过 rc.local 设置了它。请注意内核 2.6 的潜在问题。* https://bugs.centos.org/view.php?id=6021

不要忘记窗口会在 控制的特定时间间隔内缩小。 net.ipv4.tcp_slow_start_after_idle 值(value)。

echo "
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_wmem = 4096  87380 16777216
net.ipv4.tcp_rmem = 4096  87380 16777216
" >> /etc/sysctl.conf