audio - 如何使用时间戳注释标记和提取音频

Question

我想标记然后提取音频文件(audio.wav)的某些片段。段的开始和结束时间由DateTimeStamp(第一列)给出，而 Action 持续时间(以毫秒为单位)(第三列)在另一个文件(注释文件(annot.csv))中给出:

DateTimeStamp           Action  Duration of action in milliseconds
04/16/20 21:25:36:241   A       502
04/16/20 21:25:36:317   B       2253
04/16/20 21:25:36:734   X       118
04/16/20 21:25:36:837   C       10
04/16/20 21:25:37:537   D       797
04/16/20 21:25:37:606   X       70
04/16/20 21:25:37:874   A       1506
.                       .       .

audio.wav文件在文件annot.csv的第一个DateTimeStamp时间开始。如何使用annot.csv文件中的信息标记和提取audio.wav文件中的某个片段(例如，对应于Action X)？
我尝试使用librosa和pyAudioAnalysis软件包解决该问题，但找不到所需的信息。
任何帮助，不胜感激。

Answer 1

这里的关键是计算每个指定段的开始和结束(在音频样本索引中)。
可以通过先将毫秒转换为秒，然后再乘以音频的采样率来采样索引来完成此操作。
但是总的来说，我建议在处理诸如此类的时间序列时使用Pandas datetime和timedelta功能。下面是一些实现此目的的示例代码:

import io

import pandas
import numpy
import librosa


def read_data(f, date_format):
    df = pandas.read_csv(f, sep=',')

    # Use proper pandas datatypes
    df['Time'] = pandas.to_datetime(df['DateTimeStamp'], format=date_format)
    df['Duration'] = pandas.to_timedelta(df['Duration ms'], unit='ms')
    df = df.drop(columns=['DateTimeStamp', 'Duration ms'])

    # Compute start and end time of each segment
    # audio starts at time of first segment
    first = df['Time'].iloc[0]
    df['Start'] = df['Time'] - first
    df['End'] = df['Start'] + df['Duration']

    return df

def extract_segments(y, sr, segments):
    # compute segment regions in number of samples
    starts = numpy.floor(segments.Start.dt.total_seconds() * sr).astype(int)
    ends = numpy.ceil(segments.End.dt.total_seconds() * sr).astype(int)

    # slice the audio into segments
    for start, end in zip(starts, ends):
        audio_seg = y[start:end]
        print('extracting audio segment:', len(audio_seg), 'samples')

## Reproducible example
data = io.StringIO("""DateTimeStamp,Action,Duration ms
04/16/20 21:25:36:241,A,502
04/16/20 21:25:36:317,B,2253
04/16/20 21:25:36:734,X,118
04/16/20 21:25:36:837,C,10
04/16/20 21:25:37:537,D,797
04/16/20 21:25:37:606,X,70
04/16/20 21:25:37:874,A,1506
""")

segments = read_data(data, date_format="%m/%d/%y %H:%M:%S:%f")
print(segments)

path = librosa.util.example_audio_file()
y, sr = librosa.load(path, sr=16000, duration=10)
extract_segments(y, sr, segments)

应该输出类似

 Action                    Time        Duration           Start             End
0      A 2020-04-16 21:25:36.241 00:00:00.502000        00:00:00 00:00:00.502000
1      B 2020-04-16 21:25:36.317 00:00:02.253000 00:00:00.076000 00:00:02.329000
2      X 2020-04-16 21:25:36.734 00:00:00.118000 00:00:00.493000 00:00:00.611000
3      C 2020-04-16 21:25:36.837 00:00:00.010000 00:00:00.596000 00:00:00.606000
4      D 2020-04-16 21:25:37.537 00:00:00.797000 00:00:01.296000 00:00:02.093000
5      X 2020-04-16 21:25:37.606 00:00:00.070000 00:00:01.365000 00:00:01.435000
6      A 2020-04-16 21:25:37.874 00:00:01.506000 00:00:01.633000 00:00:03.139000
extracting audio segment: 8032 samples
extracting audio segment: 36048 samples
extracting audio segment: 1888 samples
extracting audio segment: 160 samples
extracting audio segment: 12752 samples
extracting audio segment: 1120 samples
extracting audio segment: 24097 samples