python - 值错误: cannot reindex from a duplicate axis (python pandas)

Question

我在网上尝试了很多解决方案，但没有任何效果。 :(我正在尝试将数据缩放到 0-1 范围内，在我进入此数据文件之前它工作正常。代码或数据文件出了什么问题？

代码:

import pandas as pd

df = pd.read_csv('data_labelled.csv',index_col=False)

df_norm = ((df.ix[:, 1:-1]       - df.ix[:, 1:-1].min()) / 
           (df.ix[:, 1:-1].max() - df.ix[:, 1:-1].min()) )
print df_norm 
rslt =  pd.concat([df_norm, df.ix[:,-1]], axis=1)
rslt.to_csv('data_normalize.csv',index=False,header=False)

错误:

ankit@ankit21:~/rrd-xml/instances$ python normalize.py 
Traceback (most recent call last):
  File "normalize.py", line 6, in <module>
    df_norm = (df.ix[:, 1:-1] - df.ix[:, 1:-1].min()) / (df.ix[:, 1:-1].max() - df.ix[:, 1:-1].min())
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 889, in f
    return self._combine_series(other, na_op, fill_value, axis, level)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/frame.py", line 3183, in _combine_series
    return self._combine_series_infer(other, func, level=level, fill_value=fill_value)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/frame.py", line 3203, in _combine_series_infer
    return self._combine_match_columns(other, func, level=level, fill_value=fill_value)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/frame.py", line 3221, in _combine_match_columns
    func=func, other=right, axes=[left.columns, self.index])
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/internals.py", line 2480, in eval
    return self.apply('eval', **kwargs)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/internals.py", line 2457, in apply
    copy=align_copy)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/series.py", line 2170, in reindex_axis
    return self.reindex(index=labels, **kwargs)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/series.py", line 2151, in reindex
    return super(Series, self).reindex(index=index, **kwargs)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/generic.py", line 1773, in reindex
    method, fill_value, copy).__finalize__(self)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/generic.py", line 1790, in _reindex_axes
    fill_value=fill_value, copy=copy, allow_dups=False)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/generic.py", line 1876, in _reindex_with_indexers
    copy=copy)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/internals.py", line 3150, in reindex_indexer
    self.axes[axis]._can_reindex(indexer)
  File "/home/ankit/anaconda/lib/python2.7/site-packages/pandas/core/index.py", line 1860, in _can_reindex
    raise ValueError("cannot reindex from a duplicate axis")
ValueError: cannot reindex from a duplicate axis

我的 csv:

376.56,681.73,0,99.7,0,2,2394,0.1,0.1,0.1,2.717,9.377,2.82,1.94,0.04,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1933333333,0.0066666667,0.0066666667,2.717,9.377,2.5493333333,1.4266666667,0.012,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9666.6666667,249796.26667,1005802.4,0,2040728,71,4.12,5.05,1,499,699931.73333,1046524,0
376.56,681.73,0,99.44,0,2,2394,0.1133333333,0.1733333333,0.26,2.717,9.377,2.348,1.104,0.0013333333,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499.8,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1,0.1,0.1,2.717,9.377,2.82,1.94,0.04,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1933333333,0.0066666667,0.0066666667,2.717,9.377,2.5493333333,1.4266666667,0.012,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9666.6666667,249796.26667,1005802.4,0,2040728,71,4.12,5.05,1,499,699931.73333,1046524,0
376.56,681.73,0,99.44,0,2,2394,0.1133333333,0.1733333333,0.26,2.717,9.377,2.348,1.104,0.0013333333,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499.8,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,1
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,1
376.56,681.73,0,99.7,0,2,2394,0.1,0.1,0.1,2.717,9.377,2.82,1.94,0.04,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1933333333,0.0066666667,0.0066666667,2.717,9.377,2.5493333333,1.4266666667,0.012,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9666.6666667,249796.26667,1005802.4,0,2040728,71,4.12,5.05,1,499,699931.73333,1046524,0
376.56,681.73,0,99.44,0,2,2394,0.1133333333,0.1733333333,0.26,2.717,9.377,2.348,1.104,0.0013333333,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499.8,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.486666667,0,2,2394,0.1866666667,0.0266666667,0.3,2.717,9.377,2.112,0.8,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9763.2,249811.46667,1005403.7333,0,2040728,71,4.12,5.05,1,500,701100.53333,1046524,0
369.27133333,652.13333333,0,94.473333333,0,2,2394,2.02,1.7333333333,1.7733333333,2.717,9.377,1.9326666667,0.604,0,9810.6666667,250595.46667,1003885.6,0,2040728,71,4.0593333333,4.79,1,500,701786.93333,1046524,0
368.15,647.58,0,93.7,0,2,2394,2.3,2,2,2.717,9.377,1.91,0.58,0,9816,250716,1003660,0,2040728,71,4.05,4.75,1,500,701868,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1,0.1,0.1,2.717,9.377,2.82,1.94,0.04,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1933333333,0.0066666667,0.0066666667,2.717,9.377,2.5493333333,1.4266666667,0.012,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9666.6666667,249796.26667,1005802.4,0,2040728,71,4.12,5.05,1,499,699931.73333,1046524,0
376.56,681.73,0,99.44,0,2,2394,0.1133333333,0.1733333333,0.26,2.717,9.377,2.348,1.104,0.0013333333,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499.8,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.486666667,0,2,2394,0.1866666667,0.0266666667,0.3,2.717,9.377,2.112,0.8,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9763.2,249811.46667,1005403.7333,0,2040728,71,4.12,5.05,1,500,701100.53333,1046524,0
369.27133333,652.13333333,0,94.473333333,0,2,2394,2.02,1.7333333333,1.7733333333,2.717,9.377,1.9326666667,0.604,0,9810.6666667,250595.46667,1003885.6,0,2040728,71,4.0593333333,4.79,1,500,701786.93333,1046524,0
368.15,647.58,0,93.7,0,2,2394,2.3,2,2,2.717,9.377,1.91,0.58,0,9816,250716,1003660,0,2040728,71,4.05,4.75,1,500,701868,1046524,0
368.15,647.58,0,99.113333333,0,2,2394,0.4333333333,0.2266666667,0.32,2.717,9.377,1.91,0.58,0,9816,250716,1003660,0,2040728,71,4.05,4.75,1,500,701868,1046524,0
368.15,647.58,0,99.5,0,2,2394,0.3,0.1,0.2,2.717,9.377,1.91,0.58,0,9845.8666667,250742.13333,1001401.3333,0,2040728,71,4.05,4.75,1,500,702125.6,1046524,0
368.15,647.58,0,99.5,0,2,2394,0.3,0.1,0.2,2.717,9.377,1.91,0.58,0,9848,250744,1001240,0,2040728,71,4.05,4.75,1,500,702144,1046524,0
368.15,647.58,0,99.5,0,2,2394,0.3,0.1,0.2,2.717,9.377,1.91,0.58,0,9848,250744,1001240,0,2040728,71,4.05,4.75,1.9333333333,500,702144,1046524,0
368.15,647.58,0,99.5,0,2,2394,0.3,0.1,0.2,2.717,9.377,1.7326666667,0.44,0.0186666667,9848,250744,1001240,0,2040728,71,4.05,4.75,2,500,702144,1046524,0
368.15,647.58,0,99.5,0,2,2394,0.3,0.1,0.2,2.717,9.377,1.72,0.43,0.02,9848,250744,1001240,0,2040728,71,4.05,4.75,2,500,702144,1046524,0
368.15,647.58,0,99.033333333,0,2,2394,0.2066666667,0.1933333333,0.5733333333,2.717,9.377,1.72,0.43,0.02,9848,250744,1001240,0,2040728,71,4.05,4.75,2,500,702144,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1,0.1,0.1,2.717,9.377,2.82,1.94,0.04,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.1933333333,0.0066666667,0.0066666667,2.717,9.377,2.5493333333,1.4266666667,0.012,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,0,502,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9580,249720,1006208,0,2040728,71,4.12,5.05,1,499,699072,1046524,0
376.56,681.73,0,99.7,0,2,2394,0.2,0,0,2.717,9.377,2.53,1.39,0.01,9666.6666667,249796.26667,1005802.4,0,2040728,71,4.12,5.05,1,499,699931.73333,1046524,0
376.56,681.73,0,99.44,0,2,2394,0.1133333333,0.1733333333,0.26,2.717,9.377,2.348,1.104,0.0013333333,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,499.8,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.4,0,2,2394,0.1,0.2,0.3,2.717,9.377,2.32,1.06,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.486666667,0,2,2394,0.1866666667,0.0266666667,0.3,2.717,9.377,2.112,0.8,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0
376.56,681.73,0,99.5,0,2,2394,0.2,0,0.3,2.717,9.377,2.08,0.76,0,9680,249808,1005740,0,2040728,71,4.12,5.05,1,500,700064,1046524,0

我注意到的一件事是，当我用先前数据文件的同一列替换第 24 列时，它工作得很好!怎么办？

Answer 1

读取 csv 时尝试 header=None。

然后，您可以简单地执行以下操作:

df_norm = (df - df.min()).div(df.max() - df.min())

NaN 来自方差为零的列，即除以零。