python - MemoryError 使用 Nginx、uWSGI 和 web.py

Question

我有一个 Python Apple MDM 服务器，我正在尝试将其从命令行进程运行转换为使用 Nginx 和 uWSGI 托管。我正在使用 web.py 进行路由。一切都在最适当的情况下顺利进行。我已经有了我认为对 Nginx 和 uWSGI 都很好的基本配置，但是今天早上在运行它不到 24 小时后，我在服务器收到的所有其他请求上都收到了这个错误。有足够的可用内存，我使用的是 64 位版本的 python。

编辑 1:SSL 在端口 8080 上通信，因为它前面有一个负载均衡器将 SSL 流量路由到该端口。

EDIT2:我已经将 uWSGI 中的进程设置从 3 更改为 1 以查看是否是问题所在。

EDIT3:这是来自 AWS 的新盒子。安装了我的服务器及其依赖项、nginx 和 uWSGI。我在将 uWSGI 作为服务运行时遇到了麻烦，在我最终获得运行的 pip 版本之前安装和卸载了它的 aptitude 和 pip 版本系统。这就是它现在的运行方式，并且优雅地停止、启动和重新启动。

[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 565, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 77, in reload_mapping
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 581, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 279, in wsgi
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] result = self.handle_with_processors()
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 249, in handle_with_processors
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] return process(self.processors)
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] raise self.internalerror()
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] import debugerror
[ Tue Jun 14 2016 10:29:22 ] MemoryError[ Tue Jun 14 2016 10:29:22 ] 

nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}
#mail {
#       # See sample authentication script at:
#       # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
# 
#       # auth_http localhost/auth.php;
#       # pop3_capabilities "TOP" "USER";
#       # imap_capabilities "IMAP4rev1" "UIDPLUS";
# 
#       server {
#               listen     localhost:110;
#               protocol   pop3;
#               proxy      on;
#       }
# 
#       server {
#               listen     localhost:143;
#               protocol   imap;
#               proxy      on;
#       }
#}

网站启用

server {
    listen 80;

    location ^~ /static/ {
        root /path/to/app/;
    }

    location / {
        include   uwsgi_params;
        uwsgi_pass  127.0.0.1:9000;
    }
}
server {
    listen   8080;

    ssl    on;
    ssl_certificate    /path/to/app/certs/ssl.crt;
    ssl_certificate_key    /path/to/app/certs/ssl.key;

    location ^~ /static/ {
        root /path/to/app/static/;
    }

    location / {
        include   uwsgi_params;
        uwsgi_pass  127.0.0.1:9000;
        uwsgi_param UWSGI_CHDIR /path/to/app/;
        uwsgi_param UWSGI_PYHOME /path/to/app/virtualenv/;
        uwsgi_param UWSGI_SCRIPT server;
    }
}

uWSGI 配置文件

home = /path/to/app/virtualenv/
chdir = /path/to/app/
module = server:application
logto = /var/log/uwsgi/%n.log
rlogdate =  [ %%a %%b %%d %%Y %%H:%%M:%%S ]
logformat = %(method) %(status) %(addr)  %(uri) wid: %(wid) 
log-encoder = format ${strftime:[ %%a %%b %%d %%Y %%H:%%M:%%S ]} ${msgnl}
gid = ubuntu
uid = ubuntu
vhost = false 
socket = 127.0.0.1:9000
master = true
processes = 3
harakiri = 20
limit-as = 128
memory-report
no-orphans

有人告诉我，我有一个相当严重的意外系统管理员综合症案例，因此非常感谢您提供任何帮助/建议。只是为了更好的衡量，这里是 proc/meminfo

的输出

MemTotal:        7657772 kB
MemFree:         5971508 kB
MemAvailable:    7145084 kB
Buffers:          225528 kB
Cached:           900756 kB
SwapCached:            0 kB
Active:          1142912 kB
Inactive:         358288 kB
Active(anon):     380568 kB
Inactive(anon):    11956 kB
Active(file):     762344 kB
Inactive(file):   346332 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                48 kB
Writeback:             0 kB
AnonPages:        375008 kB
Mapped:            55840 kB
Shmem:             17616 kB
Slab:             129300 kB
SReclaimable:     106084 kB
SUnreclaim:        23216 kB
KernelStack:        3120 kB
PageTables:         6672 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     3828884 kB
Committed_AS:     602572 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    143360 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       71680 kB
DirectMap2M:     7923712 kB

Answer 1

你没有在你的 uWSGI .ini 中设置 max-requests。因此，你的 python 解释器永远存在并且可能会积累垃圾。尝试将其设置为某些内容，例如5000，希望泄漏会停止。

如果你以后得到更多的统计结果并且他们反驳了max-requests版本，请分享他们。祝你好运，詹姆斯·菲特。