python - 处理上传的图像 zip 时出现 HTTP 504

Question

我是网络开发的新手，我正在使用 Django 开发一个基本的图片库应用程序(学习练习)。我已经设置好了，所以我可以一次上传一个充满图像的 zip 文件来创建一个新相册。这一切似乎工作正常，但当上传的文件特别大时，我收到 HTTP 504 错误。

我想(如果我错了请纠正我)这个错误意味着我的应用程序太慢而无法返回 HTTP 响应。我猜这是因为解压缩和处理(在数据库中创建 Pic 对象并创建缩略图)所有图像需要很长时间。

有没有办法在后台执行处理的同时返回一个响应(比如对某个中间页面)——也许使用线程？处理这个问题的正确方法是什么？我是时候开始学习 Javascript/AJAX 了吗？

谢谢!

---

模型:

from django.db import models
from blog.models import Post

class Album(models.Model):
    title = models.CharField(max_length=128)
    slug = models.SlugField()
    description = models.TextField()
    parent = models.ForeignKey('self', null=True, blank=True)

    pub = models.BooleanField()
    date_created = models.DateTimeField(auto_now_add=True)
    date_published = models.DateTimeField(null=True, blank=True)
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

class Pic(models.Model):
    image = models.ImageField(upload_to='pics/%Y/%m') 
    title = models.CharField(max_length=128)
    caption = models.TextField(blank=True, null=True)
    albums = models.ManyToManyField('Album', null=True, blank=True)
    posts = models.ManyToManyField(Post, blank=True, null=True)

    date_taken = models.DateTimeField(null=True, blank=True) 
    date_uploaded = models.DateTimeField(auto_now_add=True) 
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

查看:

_{我手动执行此操作是因为我在开始时没有理解 Django 管理员。我认为在这里使用管理员自定义可能会更好。}

def new_album(request):
    if request.method == "POST":
        form = AlbumForm(request.POST, request.FILES)
        if form.is_valid():
            from gallery.pic_handlers import handle_uploaded_album
            pics = handle_uploaded_album(request.FILES['pic_archive'])
            a = form.save()
            a.slug = slugify(a.title)
            a.save()
            for pic in pics:
                pic.albums.add(a)
            return HttpResponseRedirect('/gallery/album/%s/' % a.slug)
    else:
        form = AlbumForm()

    return render_to_response('new_album.html', {
        'form' : form,
    }, context_instance = RequestContext(request))

附加处理:

def handle_uploaded_album(pic_archive):
    destination = open(join(settings.MEDIA_ROOT,pic_archive.name), 'wb+')
    for chunk in pic_archive.chunks():
        destination.write(chunk)
    destination.close()

    today = datetime.date.today()
    save_path = 'pics/{0}/{1:02}/'.format(today.year, today.month)
    tmp_path = 'tmp/'
    z = zipfile.ZipFile(join(settings.MEDIA_ROOT,pic_archive.name), 'r')
    pics = []
    for member in z.namelist():
        if '/' in member or '\\' in member: 
            # don't deal with any directories inside the zip
            # this also solves the '__MACOSX' issue
            continue
        if splitext(member)[1] in IMG_EXT:
            z.extract(member,join(settings.MEDIA_ROOT,tmp_path))
            im = File(open(join(settings.MEDIA_ROOT,tmp_path,member), 'rb'))
            # create a Pic from this file
            pic = Pic()
            pic.title = member
            pic.image.save(
                join(save_path, member),
                im,
                True)
            create_thumbnails(pic)
            im.close()
            # remove extracted images
            remove(join(settings.MEDIA_ROOT,tmp_path,member))

            # TODO: save date taken if available
            pics.append(pic)

    z.close()
    remove(join(settings.MEDIA_ROOT,pic_archive.name))

    return pics

def create_thumbnails(pic):
    fname, ext = splitext(pic.image.path)
    img = Image.open(pic.image.path)

    img.thumbnail((512,512), Image.ANTIALIAS)
    img.save(fname + '_m' + ext)

    img.thumbnail((128,128), Image.ANTIALIAS)
    img.save(fname + '_s' + ext)

Answer 1

像这个处理这样的长任务需要太多时间，你的客户端和/或你的代理将超时 - 这是 504 error你看。

你不应该以这种方式运行长时间的作业!

正如您在最后正确询问的那样，您需要一种方法来分离长时间执行 - 通过异步队列系统，例如 celery .通过这种方式，您可以直接将答案返回给您的客户，而后端则异步运行作业。

您应该查看以下内容之一:

• Django long running asynchronous tasks with threads/processing
• Long running tasks with Django
• Long polling in Django

自 django-celery绝对是最好的选择，下一步是了解它；还有很多SO questions around it

如果您想确定它是处理失败，而不是上传，只需尝试禁用所有处理并立即返回给您的客户。如果仍然失败，您还需要调整您的网络服务器，使其不会超时!