python-xarray - xarray.apply_ufunc() 与 GroupBy : unexpected number of dimensions

Question

我正在使用 xarray.apply_ufunc() 将函数应用于 xarray.DataArray .它适用于某些 NetCDF，但在尺寸、坐标等方面似乎具有可比性的其他 NetCDF 会失败。但是，代码适用的 NetCDF 与代码失败的 NetCDF 之间肯定存在差异，希望有人可以在看到代码和下面列出的文件的一些元数据后，评论问题是什么。

我正在运行以执行计算的代码是这样的:

# open the precipitation NetCDF as an xarray DataSet object
dataset = xr.open_dataset(kwrgs['netcdf_precip'])

# get the precipitation array, over which we'll compute the SPI
da_precip = dataset[kwrgs['var_name_precip']]

# stack the lat and lon dimensions into a new dimension named point, so at each lat/lon
# we'll have a time series for the geospatial point, and group by these points
da_precip_groupby = da_precip.stack(point=('lat', 'lon')).groupby('point')

# apply the SPI function to the data array
da_spi = xr.apply_ufunc(indices.spi,
                        da_precip_groupby)

# unstack the array back into original dimensions
da_spi = da_spi.unstack('point')

有效的 NetCDF 如下所示:

>>> import xarray as xr
>>> ds_good = xr.open_dataset("good.nc")
>>> ds_good
<xarray.Dataset>
Dimensions:  (lat: 38, lon: 87, time: 1466)
Coordinates:
  * lat      (lat) float32 24.5625 25.229166 25.895834 ... 48.5625 49.229168
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2017-02-01
Data variables:
    prcp     (lat, lon, time) float32 ...
Attributes:
    Conventions:               CF-1.6, ACDD-1.3
    ncei_template_version:     NCEI_NetCDF_Grid_Template_v2.0
    title:                     nClimGrid
    naming_authority:          gov.noaa.ncei
    standard_name_vocabulary:  Standard Name Table v35
    institution:               National Centers for Environmental Information...
    geospatial_lat_min:        24.5625
    geospatial_lat_max:        49.354168
    geospatial_lon_min:        -124.6875
    geospatial_lon_max:        -67.020836
    geospatial_lat_units:      degrees_north
    geospatial_lon_units:      degrees_east
    NCO:                       4.7.1
    nco_openmp_thread_number:  1
>>> ds_good.prcp
<xarray.DataArray 'prcp' (lat: 38, lon: 87, time: 1466)>
[4846596 values with dtype=float32]
Coordinates:
  * lat      (lat) float32 24.5625 25.229166 25.895834 ... 48.5625 49.229168
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2017-02-01
Attributes:
    valid_min:      0.0
    units:          millimeter
    valid_max:      2000.0
    standard_name:  precipitation_amount
    long_name:      Precipitation, monthly total

失败的 NetCDF 如下所示:

>>> ds_bad = xr.open_dataset("bad.nc")   >>> ds_bad
<xarray.Dataset>
Dimensions:  (lat: 38, lon: 87, time: 1483)
Coordinates:
  * lat      (lat) float32 49.3542 48.687534 48.020866 ... 25.3542 24.687532
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2018-07-01
Data variables:
    prcp     (lat, lon, time) float32 ...
Attributes:
    date_created:              2018-02-15 10:29:25.485927
    date_modified:             2018-02-15 10:29:25.486042
    Conventions:               CF-1.6, ACDD-1.3
    ncei_template_version:     NCEI_NetCDF_Grid_Template_v2.0
    title:                     nClimGrid
    naming_authority:          gov.noaa.ncei
    standard_name_vocabulary:  Standard Name Table v35
    institution:               National Centers for Environmental Information...
    geospatial_lat_min:        24.562532
    geospatial_lat_max:        49.3542
    geospatial_lon_min:        -124.6875
    geospatial_lon_max:        -67.020836
    geospatial_lat_units:      degrees_north
    geospatial_lon_units:      degrees_east
>>> ds_bad.prcp
<xarray.DataArray 'prcp' (lat: 38, lon: 87, time: 1483)>
[4902798 values with dtype=float32]
Coordinates:
  * lat      (lat) float32 49.3542 48.687534 48.020866 ... 25.3542 24.687532
  * lon      (lon) float32 -124.6875 -124.020836 ... -68.020836 -67.354164
  * time     (time) datetime64[ns] 1895-01-01 1895-02-01 ... 2018-07-01
Attributes:
    valid_min:      0.0
    long_name:      Precipitation, monthly total
    standard_name:  precipitation_amount
    units:          millimeter
    valid_max:      2000.0

当我针对上面的第一个文件运行代码时，它可以正常工作。使用第二个文件时，我收到如下错误:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/paperspace/git/climate_indices/scripts/process_grid_ufunc.py", line 278, in compute_write_spi
    kwargs=args_dict)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 974, in apply_ufunc
    return apply_groupby_ufunc(this_apply, *args)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 432, in apply_groupby_ufunc
    applied_example, applied = peek_at(applied)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/utils.py", line 133, in peek_at
    peek = next(gen)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 431, in <genexpr>
    applied = (func(*zipped_args) for zipped_args in zip(*iterators))
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 987, in apply_ufunc
    exclude_dims=exclude_dims)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 211, in apply_dataarray_ufunc
    result_var = func(*data_vars)
  File "/home/paperspace/anaconda3/envs/climate/lib/python3.6/site-packages/xarray/core/computation.py", line 579, in apply_variable_ufunc
    .format(data.ndim, len(dims), dims))
ValueError: applied function returned data with unexpected number of dimensions: 1 vs 2, for dimensions ('time', 'point')

任何人都可以评论可能是什么问题吗？

Answer 1

事实证明，作为输入有问题的 NetCDF 文件纬度坐标值按降序排列。 xarray.apply_ufunc()似乎要求坐标值按升序排列，至少是为了避免这个特定问题。这可以通过使用 NCO 的 ncpdq 反转违规维度的坐标值来轻松解决。使用 NetCDF 文件作为 xarray 的输入之前的命令。