python - 如何使用 pyglet 播放流式音频？

Question

这个问题的目的是试图弄清楚如何使用 pyglet 播放流式音频。首先是确保您能够使用 pyglet 播放 mp3 文件，这就是第一个片段的目的:

import sys
import inspect
import requests

import pyglet
from pyglet.media import *

pyglet.lib.load_library('avbin')
pyglet.have_avbin = True


def url_to_filename(url):
    return url.split('/')[-1]


def download_file(url, filename=None):
    filename = filename or url_to_filename(url)

    with open(filename, "wb") as f:
        print("Downloading %s" % filename)
        response = requests.get(url, stream=True)
        total_length = response.headers.get('content-length')

        if total_length is None:
            f.write(response.content)
        else:
            dl = 0
            total_length = int(total_length)
            for data in response.iter_content(chunk_size=4096):
                dl += len(data)
                f.write(data)
                done = int(50 * dl / total_length)
                sys.stdout.write("\r[%s%s]" % ('=' * done, ' ' * (50 - done)))
                sys.stdout.flush()


url = "https://freemusicarchive.org/file/music/ccCommunity/DASK/Abiogenesis/DASK_-_08_-_Protocell.mp3"
filename = "mcve.mp3"
download_file(url, filename)

music = pyglet.media.load(filename)
music.play()
pyglet.app.run()

如果您已经安装了库 pip install pyglet requests 并且还安装了 AVBin此时您应该可以在下载 mp3 后收听它。

一旦我们达到了这一点，我想弄清楚如何以与使用 pyglet+requests 的大多数现有网络视频/音频播放器类似的方式播放和缓冲文件。这意味着无需等到文件完全下载就可以播放文件。

看完pyglet媒体docs你可以看到有可用的这些类:

media
    sources
        base
            AudioData
            AudioFormat
            Source
            SourceGroup
            SourceInfo
            StaticSource
            StreamingSource
            VideoFormat
    player
        Player
        PlayerGroup

我看到还有另一个类似的 SO 问题，但他们没有得到妥善解决，他们的内容没有提供很多相关细节:

• Play streaming audio using pyglet
• How can I play audio stream without saving it into the file with pyglet?

这就是为什么我创建了一个新问题。你如何使用 pyglet 播放流式音频？您能否提供一个使用上述 mcve 作为基础的小示例？

Answer 1

假设您不想导入新包来为您执行此操作 - 这可以通过一些努力来完成。

首先，让我们转到 Pyglet 源代码并查看 media/__init__.py 中的 media.load。

"""Load a Source from a file.

All decoders that are registered for the filename extension are tried.
If none succeed, the exception from the first decoder is raised.
You can also specifically pass a decoder to use.

:Parameters:
    `filename` : str
        Used to guess the media format, and to load the file if `file` is
        unspecified.
    `file` : file-like object or None
        Source of media data in any supported format.
    `streaming` : bool
        If `False`, a :class:`StaticSource` will be returned; otherwise
        (default) a :class:`~pyglet.media.StreamingSource` is created.
    `decoder` : MediaDecoder or None
        A specific decoder you wish to use, rather than relying on
        automatic detection. If specified, no other decoders are tried.

:rtype: StreamingSource or Source
"""
if decoder:
    return decoder.decode(file, filename, streaming)
else:
    first_exception = None
    for decoder in get_decoders(filename):
        try:
            loaded_source = decoder.decode(file, filename, streaming)
            return loaded_source
        except MediaDecodeException as e:
            if not first_exception or first_exception.exception_priority < e.exception_priority:
                first_exception = e

    # TODO: Review this:
    # The FFmpeg codec attempts to decode anything, so this codepath won't be reached.
    if not first_exception:
        raise MediaDecodeException('No decoders are available for this media format.')
    raise first_exception


add_default_media_codecs()

这里的关键行是 loaded_source = decoder.decode(...)。从本质上讲，为了加载音频，Pyglet 获取一个文件并将其拖到媒体解码器(例如 FFMPEG)，然后媒体解码器返回一个“帧”或数据包的列表，Pyglet 可以使用内置的 Player 类。如果音频格式是压缩的(例如 mp3 或 aac)，Pyglet 将使用外部库(目前仅支持 AVBin)将其转换为原始的解压缩音频。你可能已经知道其中的一些了。

因此，如果我们想了解如何将字节流而不是文件填充到 Pyglet 的音频引擎中，我们需要查看其中一个解码器。对于此示例，让我们使用 FFMPEG，因为它最容易访问。

在media/codecs/ffmpeg.py中:

class FFmpegDecoder(object):

def get_file_extensions(self):
    return ['.mp3', '.ogg']

def decode(self, file, filename, streaming):
    if streaming:
        return FFmpegSource(filename, file)
    else:
        return StaticSource(FFmpegSource(filename, file))

它继承自的“对象”是 MediaDecoder，位于 media/codecs/__init__.py 中。回到 media/__init__.py 中的 load 函数，你会看到 pyglet 会根据文件扩展名选择一个 MediaDecoder，然后返回它的 decode 以文件为参数的函数，以数据包流的形式获取音频。该数据包流是一个 Source 对象；每个解码器都有自己的风格，以 StaticSource 或 StreamingSource 的形式出现。前者用于将音频存储在内存中，后者用于立即播放。 FFmpeg 的解码器只支持 StreamingSource。

我们可以看到FFMPEG的是FFmpegSource，同样位于media/codecs/ffmpeg.py。我们发现了这个巨人:

class FFmpegSource(StreamingSource):
# Max increase/decrease of original sample size
SAMPLE_CORRECTION_PERCENT_MAX = 10

def __init__(self, filename, file=None):
    if file is not None:
        raise NotImplementedError('Loading from file stream is not supported')

    self._file = ffmpeg_open_filename(asbytes_filename(filename))
    if not self._file:
        raise FFmpegException('Could not open "{0}"'.format(filename))

    self._video_stream = None
    self._video_stream_index = None
    self._audio_stream = None
    self._audio_stream_index = None
    self._audio_format = None

    self.img_convert_ctx = POINTER(SwsContext)()
    self.audio_convert_ctx = POINTER(SwrContext)()

    file_info = ffmpeg_file_info(self._file)

    self.info = SourceInfo()
    self.info.title = file_info.title
    self.info.author = file_info.author
    self.info.copyright = file_info.copyright
    self.info.comment = file_info.comment
    self.info.album = file_info.album
    self.info.year = file_info.year
    self.info.track = file_info.track
    self.info.genre = file_info.genre

    # Pick the first video and audio streams found, ignore others.
    for i in range(file_info.n_streams):
        info = ffmpeg_stream_info(self._file, i)

        if isinstance(info, StreamVideoInfo) and self._video_stream is None:

            stream = ffmpeg_open_stream(self._file, i)

            self.video_format = VideoFormat(
                width=info.width,
                height=info.height)
            if info.sample_aspect_num != 0:
                self.video_format.sample_aspect = (
                    float(info.sample_aspect_num) /
                    info.sample_aspect_den)
            self.video_format.frame_rate = (
                float(info.frame_rate_num) /
                info.frame_rate_den)
            self._video_stream = stream
            self._video_stream_index = i

        elif (isinstance(info, StreamAudioInfo) and
                      info.sample_bits in (8, 16) and
                      self._audio_stream is None):

            stream = ffmpeg_open_stream(self._file, i)

            self.audio_format = AudioFormat(
                channels=min(2, info.channels),
                sample_size=info.sample_bits,
                sample_rate=info.sample_rate)
            self._audio_stream = stream
            self._audio_stream_index = i

            channel_input = avutil.av_get_default_channel_layout(info.channels)
            channels_out = min(2, info.channels)
            channel_output = avutil.av_get_default_channel_layout(channels_out)

            sample_rate = stream.codec_context.contents.sample_rate
            sample_format = stream.codec_context.contents.sample_fmt
            if sample_format in (AV_SAMPLE_FMT_U8, AV_SAMPLE_FMT_U8P):
                self.tgt_format = AV_SAMPLE_FMT_U8
            elif sample_format in (AV_SAMPLE_FMT_S16, AV_SAMPLE_FMT_S16P):
                self.tgt_format = AV_SAMPLE_FMT_S16
            elif sample_format in (AV_SAMPLE_FMT_S32, AV_SAMPLE_FMT_S32P):
                self.tgt_format = AV_SAMPLE_FMT_S32
            elif sample_format in (AV_SAMPLE_FMT_FLT, AV_SAMPLE_FMT_FLTP):
                self.tgt_format = AV_SAMPLE_FMT_S16
            else:
                raise FFmpegException('Audio format not supported.')

            self.audio_convert_ctx = swresample.swr_alloc_set_opts(None,
                                                                   channel_output,
                                                                   self.tgt_format, sample_rate,
                                                                   channel_input, sample_format,
                                                                   sample_rate,
                                                                   0, None)
            if (not self.audio_convert_ctx or
                        swresample.swr_init(self.audio_convert_ctx) < 0):
                swresample.swr_free(self.audio_convert_ctx)
                raise FFmpegException('Cannot create sample rate converter.')

    self._packet = ffmpeg_init_packet()
    self._events = []  # They don't seem to be used!

    self.audioq = deque()
    # Make queue big enough to accomodate 1.2 sec?
    self._max_len_audioq = 50  # Need to figure out a correct amount
    if self.audio_format:
        # Buffer 1 sec worth of audio
        self._audio_buffer = \
            (c_uint8 * ffmpeg_get_audio_buffer_size(self.audio_format))()

    self.videoq = deque()
    self._max_len_videoq = 50  # Need to figure out a correct amount

    self.start_time = self._get_start_time()
    self._duration = timestamp_from_ffmpeg(file_info.duration)
    self._duration -= self.start_time

    # Flag to determine if the _fillq method was already scheduled
    self._fillq_scheduled = False
    self._fillq()
    # Don't understand why, but some files show that seeking without
    # reading the first few packets results in a seeking where we lose
    # many packets at the beginning. 
    # We only seek back to 0 for media which have a start_time > 0
    if self.start_time > 0:
        self.seek(0.0)
---
[A few hundred lines more...]
---

def get_next_video_timestamp(self):
    if not self.video_format:
        return

    if self.videoq:
        while True:
            # We skip video packets which are not video frames
            # This happens in mkv files for the first few frames.
            video_packet = self.videoq[0]
            if video_packet.image == 0:
                self._decode_video_packet(video_packet)
            if video_packet.image is not None:
                break
            self._get_video_packet()

        ts = video_packet.timestamp
    else:
        ts = None

    if _debug:
        print('Next video timestamp is', ts)
    return ts

def get_next_video_frame(self, skip_empty_frame=True):
    if not self.video_format:
        return

    while True:
        # We skip video packets which are not video frames
        # This happens in mkv files for the first few frames.
        video_packet = self._get_video_packet()
        if video_packet.image == 0:
            self._decode_video_packet(video_packet)
        if video_packet.image is not None or not skip_empty_frame:
            break

    if _debug:
        print('Returning', video_packet)

    return video_packet.image

def _get_start_time(self):
    def streams():
        format_context = self._file.context
        for idx in (self._video_stream_index, self._audio_stream_index):
            if idx is None:
                continue
            stream = format_context.contents.streams[idx].contents
            yield stream

    def start_times(streams):
        yield 0
        for stream in streams:
            start = stream.start_time
            if start == AV_NOPTS_VALUE:
                yield 0
            start_time = avutil.av_rescale_q(start,
                                             stream.time_base,
                                             AV_TIME_BASE_Q)
            start_time = timestamp_from_ffmpeg(start_time)
            yield start_time

    return max(start_times(streams()))

@property
def audio_format(self):
    return self._audio_format

@audio_format.setter
def audio_format(self, value):
    self._audio_format = value
    if value is None:
        self.audioq.clear()

您感兴趣的行是 self._file = ffmpeg_open_filename(asbytes_filename(filename))。这将我们带到这里，再次在 media/codecs/ffmpeg.py 中:

def ffmpeg_open_filename(filename):
"""Open the media file.

:rtype: FFmpegFile
:return: The structure containing all the information for the media.
"""
file = FFmpegFile()  # TODO: delete this structure and use directly AVFormatContext
result = avformat.avformat_open_input(byref(file.context),
                                      filename,
                                      None,
                                      None)
if result != 0:
    raise FFmpegException('Error opening file ' + filename.decode("utf8"))

result = avformat.avformat_find_stream_info(file.context, None)
if result < 0:
    raise FFmpegException('Could not find stream info')

return file

这就是事情变得困惑的地方:它调用一个 ctypes 函数 (avformat_open_input)，当给定一个文件时，该函数将获取其详细信息并填写我们的 FFmpegSource 类所需的所有信息。通过一些工作，您应该能够让 avformat_open_input 获取一个字节对象，而不是一个文件的路径，它将打开该文件以获取相同的信息。我很想这样做并包括一个工作示例，但我现在没有时间。然后，您需要使用新的 avformat_open_input 函数创建一个新的 ffmpeg_open_filename 函数，然后使用新的 ffmpeg_open_filename 函数创建一个新的 FFmpegSource 类。您现在需要的只是一个使用新 FFmpegSource 类的新 FFmpegDecoder 类。

然后您可以通过将它直接添加到您的 pyglet 包中来实现它。之后，您想要在 load() 函数(位于 media/__init__.py 中)添加对字节对象参数的支持，并将解码器覆盖到您的新解码器。在那里，您现在可以能够在不保存的情况下流式传输音频。

或者，您可以简单地使用已经支持它的包。 Python-vlc做。您可以使用示例 here从链接播放您想要的任何音频。如果您不只是为了挑战而这样做，我强烈建议您使用另一个包。否则:祝你好运。