heroku - 从 url 读取 pydub AudioSegment。 BytesIO 仅在 heroku 上返回 "OSError [Errno 2] No such file or directory"；在本地主机上很好

Question

编辑 1 对于任何有相同错误的人:安装 ffmpeg 确实解决了 BytesIO 错误

为仍然愿意提供帮助的任何人编辑 1:我现在的问题是，当我使用 AudioSegment.export("filename.mp3", format="mp3") 时，文件已制作，但大小为 0 字节——详情如下(如“编辑 1")

编辑2:现在所有问题都解决了。

可以使用 BytesIO

将文件作为 AudioSegment 读入

我找到了 buildpacks 以确保在我的应用程序上正确安装了 ffmpeg，并支持导出正确的 mp3 文件

下面回答

原始问题

我让 pydub 在本地工作得很好，可以根据 url 中的参数裁剪特定的 mp3 文件。
(?start_time=3.8&end_time=5.1)

当我运行 foreman start这一切在本地主机上看起来都不错。 html 渲染得很好。
views.py 中的关键行包括使用从 url 读取文件

url = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"
mp3 = urllib.urlopen(url).read() # inspired by http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
original=AudioSegment.from_mp3(BytesIO(mp3))  # AudioSegment.from_mp3 is a pydub command, see http://pydub.com
section = original[start_time_ms:end_time_ms]

这一切都很好......直到我推送到heroku(django应用程序)并在线运行它。
然后当我现在在 herokuapp.com 上加载相同的页面时，我收到了这个错误

OSError at /path/to/page
[Errno 2] No such file or directory
Request Method: GET
Request URL:    http://my.website.com/path/to/page?start_time=3.8&end_time=5
Django Version: 1.6.5
Exception Type: OSError
Exception Value:    
[Errno 2] No such file or directory
Exception Location: /app/.heroku/python/lib/python2.7/subprocess.py in _execute_child, line 1327
Python Executable:  /app/.heroku/python/bin/python
Python Version: 2.7.8
Python Path:    
['/app',
 '/app/.heroku/python/bin',
 '/app/.heroku/python/lib/python2.7/site-packages/setuptools-5.4.1-py2.7.egg',
 '/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.36-py2.7.egg',
 '/app/.heroku/python/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg',
 '/app',
 '/app/.heroku/python/lib/python27.zip',
 '/app/.heroku/python/lib/python2.7',
 '/app/.heroku/python/lib/python2.7/plat-linux2',
 '/app/.heroku/python/lib/python2.7/lib-tk',
 '/app/.heroku/python/lib/python2.7/lib-old',
 '/app/.heroku/python/lib/python2.7/lib-dynload',
 '/app/.heroku/python/lib/python2.7/site-packages',
 '/app/.heroku/python/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']


Traceback:
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  112.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/app/evernote/views.py" in finalize
  105.       original=AudioSegment.from_mp3(BytesIO(mp3))
File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_mp3
  318.         return cls.from_file(file, 'mp3')
File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_file
  302.         retcode = subprocess.call(convertion_command, stderr=open(os.devnull))
File "/app/.heroku/python/lib/python2.7/subprocess.py" in call
  522.     return Popen(*popenargs, **kwargs).wait()
File "/app/.heroku/python/lib/python2.7/subprocess.py" in __init__
  710.                                 errread, errwrite)
File "/app/.heroku/python/lib/python2.7/subprocess.py" in _execute_child
  1327.                 raise child_exception

我已经注释掉了一些原文来说服自己，单行 original=AudioSegment.from_mp3(BytesIO(mp3))是问题出现的地方......但这在本地不是问题

views.py 中的完整功能如下所示:

from django.shortcuts import render, get_object_or_404 
from django.http import HttpResponseRedirect #, Http404, HttpResponse
from django.core.urlresolvers import reverse
from django.views import generic
import pydub
# Maybe only need: 
from pydub import AudioSegment # == see below
from time import gmtime, strftime

import boto
from boto.s3.connection import S3Connection
from boto.s3.key import Key

# http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
import urllib
from io import BytesIO
# import numpy as np
# import scipy.signal as sg
# import pydub # mentioned above already
# import matplotlib.pyplot as plt
# from IPython.display import Audio, display
# import matplotlib as mpl
# %matplotlib inline

import os
# from settings import AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_BUCKET_NAME
AWS_ACCESS_KEY = os.environ.get('AWS_ACCESS_KEY') # there must be a better way?
AWS_SECRET_KEY = os.environ.get('AWS_SECRET_KEY')
AWS_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')

# http://stackoverflow.com/questions/415511/how-to-get-current-time-in-python

boto_conn = S3Connection(AWS_ACCESS_KEY, AWS_SECRET_KEY)
bucket = boto_conn.get_bucket(AWS_BUCKET_NAME)
s3_url_format = 'https://s3.amazonaws.com/shareducate02/{end_path}'

特别是在我访问页面时调用的views.py中的 View :

def finalize(request):

    start_time = request.GET.get('start_time')

    end_time = request.GET.get('end_time')

    original_file = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"


    if start_time:

      # original=AudioSegment.from_mp3(original_file)  #...that didn't work 
      # but this works below:

      # next three uncommented lines from http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
      # python 2.x
      url = original_file
      # req = urllib.Request(url, headers={'User-Agent': ''}) # Note: I commented out this because I got error that "Request" did not exist
      mp3 = urllib.urlopen(url).read()
      # That's for my 2.7

      # If I ever upgrade to python 3.x, would need to change it to:
      # req = urllib.request.Request(url, headers={'User-Agent': ''}) 
      # mp3 = urllib.request.urlopen(req).read()
      # as per instructions on http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb

      original=AudioSegment.from_mp3(BytesIO(mp3))
      # original=AudioSegment.from_mp3("static/givingtree.mp3") # alternative that works locally (on laptop) but no use for heroku

      start_time_ms = int(float(start_time) * 1000)
      if end_time:
        end_time_ms = int(float(end_time) * 1000)
      else:
        end_time_ms = int(float(original.duration_seconds) * 1000)
      duration_ms = end_time_ms - start_time_ms
      # duration = end_time - start_time
      duration = duration_ms/1000

   #   section = original[start_time_ms:end_time_ms]
   #   section_with_fading = section.fade_in(100).fade_out(100)

      clip = "demo-"
      number = strftime("%Y-%m-%d_%H-%M-%S", gmtime())
      clip += number
      clip += ".mp3" 

      # DON'T BOTHER writing locally:
      # clip_with_path = "evernote/static/"+clip
      # section_with_fading.export(clip_with_path, format = "mp3")

   #   tempclip = section_with_fading.export(format = "mp3")

      # commented out while de-bugging, but was working earlier if run on localhost
      # c = boto.connect_s3()
      # b = c.get_bucket(S3_BUCKET_NAME)  # as defined above
      # k = Key(b)
      # k.key=clip
      # # k.set_contents_from_filename(clip_with_path)
      # k.set_contents_from_file(tempclip)
      # k.set_acl('public-read')
      clip_made = True
    else: 
      duration = 0.0
      clip_made = False
      clip = ""
    context = {'original_file':original_file, 'new_file':clip, 'start_time': start_time, 'end_time':end_time, 'duration':duration, 'clip_made':clip_made} 
    return render(request, 'finalize.html' , context) 

有什么建议么？

潜在相关:
我在本地安装了ffmpeg

但由于不了解构建包，无法将其安装到 heroku 上。我刚刚尝试过( http://stackoverflow.com/questions/14407388/how-to-install-ffmpeg-for-a-django-app-on-heroku 和 https://github.com/shunjikonishi/heroku-buildpack-ffmpeg )，但到目前为止 ffmpeg 无法在 heroku 上运行(当我执行“heroku run ffmpeg --version”时，ffmpeg 无法识别)
……你认为这是原因吗？

当我在这里转圈圈时，将不胜感激任何这样的答案:

“我认为 ffmpeg 确实是你的问题。努力解决这个问题，把它安装在 heroku 上”

“实际上，我认为这就是 BytesIO 不适合您的原因:...”

“无论如何，你的方法很糟糕......如果你想读入音频文件以使用 pydub 进行处理，你应该这样做:......”(因为我只是第一次通过 pydub 破解。 ..我的方法可能很差)

编辑 1

ffmpeg 现在已安装(例如，我可以输出 wav 文件)

但是，我无法创建 mp3 文件，仍然......或者更准确地说，我可以，但文件大小为零

(venv-app)moriartymacbookair13:getstartapp macuser$ heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git 
Setting config vars and restarting awe01... done, v93
BUILDPACK_URL: https://github.com/ddollar/heroku-buildpack-multi.git
(venv-app)moriartymacbookair13:getstartapp macuser$ vim .buildpacks 
(venv-app)moriartymacbookair13:getstartapp macuser$ cat .buildpacks 
https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
https://github.com/heroku/heroku-buildpack-python.git
(venv-app)moriartymacbookair13:getstartapp macuser$ git add --all
(venv-app)moriartymacbookair13:getstartapp macuser$ git commit -m "need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file"
[master cd99fef] need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file
 1 file changed, 2 insertions(+), 2 deletions(-)
(venv-app)moriartymacbookair13:getstartapp macuser$ git push heroku master
Fetching repository, done.
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 372 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)

-----> Fetching custom git buildpack... done
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
=====> Detected Framework: ffmpeg
-----> Install ffmpeg
       DOWNLOAD_URL =  http://flect.github.io/heroku-binaries/libs/ffmpeg.tar.gz
       exporting PATH and LIBRARY_PATH
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-python.git
=====> Detected Framework: Python
-----> Installing dependencies with pip
       Cleaning up...

-----> Preparing static assets
       Collectstatic configuration error. To debug, run:
       $ heroku run python ./example/manage.py collectstatic --noinput

Using release configuration from last framework (Python).
-----> Discovering process types
       Procfile declares types -> web

-----> Compressing... done, 198.1MB
-----> Launching... done, v94
       http://[redacted].herokuapp.com/ deployed to Heroku

To git@heroku.com:awe01.git
   78d6b68..cd99fef  master -> master
(venv-app)moriartymacbookair13:getstartapp macuser$ heroku run ffmpeg
Running `ffmpeg` attached to terminal... up, run.6408
ffmpeg version git-2013-06-02-5711e4f Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun  2 2013 07:38:40 with gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1)
  configuration: --enable-shared --disable-asm --prefix=/app/vendor/ffmpeg
  libavutil      52. 34.100 / 52. 34.100
  libavcodec     55. 13.100 / 55. 13.100
  libavformat    55.  8.102 / 55.  8.102
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 74.101 /  3. 74.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
(venv-app)moriartymacbookair13:getstartapp macuser$ heroku run bash
Running `bash` attached to terminal... up, run.9660
~ $ python
Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydub
>>> from pydub import AudioSegment
>>> exit()
~ $ which ffmpeg
/app/vendor/ffmpeg/bin/ffmpeg
~ $ python 

Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydub
>>> from pydub import AudioSegment
>>> AudioSegment.silent(5000).export("/tmp/asdf.mp3", "mp3")
<open file '/tmp/asdf.mp3', mode 'wb+' at 0x7f9a37d44780>
>>> exit ()
~ $ cd /tmp/
/tmp $ ls
asdf.mp3
/tmp $ open asdf.mp3
bash: open: command not found
/tmp $ ls -lah
total 8.0K
drwx------  2 u36483 36483 4.0K 2014-10-22 04:14 .
drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
-rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3

请注意上面 0 的文件大小为 mp3 文件...当我在我的 macbook 上做同样的事情时，文件大小永远不会为零

回到heroku shell:

/tmp $ python
Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydub
>>> from pydub import AudioSegment
>>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg" 
>>> AudioSegment.silence(1200).export("/tmp/herokuSilence.mp3", format="mp3")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'AudioSegment' has no attribute 'silence'
>>> AudioSegment.silent(1200).export("/tmp/herokuSilence.mp3", format="mp3")
<open file '/tmp/herokuSilence.mp3', mode 'wb+' at 0x7fcc2017c780>
>>> exit()
/tmp $ ls
asdf.mp3  herokuSilence.mp3
/tmp $ ls -lah
total 8.0K
drwx------  2 u36483 36483 4.0K 2014-10-22 04:29 .
drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
-rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
-rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3

我第一次意识到我忘记了 pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg"命令，但正如您在上面看到的，文件仍然是零大小

出于绝望，我什至尝试将“.heroku”添加到路径中，以便与您的示例一样逐字记录，但这并没有解决它:

/tmp $ python
Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydub
>>> from pydub import AudioSegment
>>> pydub.AudioSegment.ffmpeg = "/app/.heroku/vendor/ffmpeg/bin/ffmpeg"
>>> AudioSegment.silent(1200).export("/tmp/herokuSilence03.mp3", format="mp3")
<open file '/tmp/herokuSilence03.mp3', mode 'wb+' at 0x7fc92aca7780>
>>> exit()
/tmp $ ls -lah
total 8.0K
drwx------  2 u36483 36483 4.0K 2014-10-22 04:31 .
drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
-rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
-rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
-rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3

最后，我尝试导出一个 .wav 文件来检查 pydub 至少工作正常

/tmp $ python
Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydub
>>> from pydub import AudioSegment
>>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg"
>>> AudioSegment.silent(1300).export("/tmp/heroku_wav_silence01.wav", format="wav")
<open file '/tmp/heroku_wav_silence01.wav', mode 'wb+' at 0x7fa33cbf3780>
>>> exit()
/tmp $ ls
asdf.mp3  herokuSilence03.mp3  herokuSilence.mp3  heroku_wav_silence01.wav
/tmp $ ls -lah
total 40K
drwx------  2 u36483 36483 4.0K 2014-10-22 04:42 .
drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
-rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
-rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
-rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3
-rw-------  1 u36483 36483  29K 2014-10-22 04:42 heroku_wav_silence01.wav
/tmp $ 

至少 .wav 的文件大小不为零，因此 pydub 正在工作

我目前的理论是，要么我仍然没有正确使用 ffmpeg，要么还不够……也许我需要在基本 ffmpeg 之上额外安装一个 mp3。

几个网站提到“libavcodec-extra-53”，但我不知道如何在heroku上安装它，或者检查我是否有它？ https://github.com/jiaaro/pydub/issues/36同样，关于 libmp3lame 的教程似乎是针对笔记本电脑安装而不是在 heroku 上安装的，所以我很茫然 http://superuser.com/questions/196857/how-to-install-libmp3lame-for-ffmpeg
如果相关，我的 requirements.txt 中也有 youtube-dl ...这也适用于我的 macbook 本地，但是当我在 heroku shell 中运行它时失败:

~/ytdl $ youtube-dl --restrict-filenames -x --audio-format mp3 n2anDgdUHic
[youtube] Setting language
[youtube] Confirming age
[youtube] n2anDgdUHic: Downloading webpage
[youtube] n2anDgdUHic: Downloading video info webpage
[youtube] n2anDgdUHic: Extracting video information
[download] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.m4a
[download] 100% of 5.92MiB in 00:00
[ffmpeg] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.mp3
ERROR: audio conversion failed: Unknown encoder 'libmp3lame'
~/ytdl $ 

信息链接是它太具体说明了 mp3 故障，所以也许这两个问题是相关的。

编辑 2

看答案，所有问题都解决了

Answer 1

所有问题都解决了，谢谢

我现在可以使用 BytesIO 从 url 读取 AudioSegments。
我现在可以在处理后导出 mp3 或 wav。

使用此处推荐的包解决了 ffmpeg 问题:http://blog.pogoapp.com/youtube-mp3-with-node-js-and-ffmpeg/ (用我的语言“python”替换“nodejs”)
那里推荐的 ffmpeg 包(https://github.com/jayzes/heroku-buildpack-ffmpeg)已经包含了我需要的蹩脚支持
出于某种原因，https://github.com/integricho/heroku-buildpack-python-ffmpeg没有为我完成这项工作

我还必须在 requirements.txt 中添加“ffprobe”，以允许 youtube-dl 正常运行(我在这里提到，因为它之前也提示过缺失...添加 ffprobe 是让它工作的第二步)

我的答案的完整记录在这里:https://github.com/rg3/youtube-dl/issues/302#issuecomment-60146845