python - 将重复元素设置为零

Question

如何将数组“data”中的重复元素转换为 0？它必须逐行完成。

data = np.array([[1,8,3,3,4],
                 [1,8,9,9,4]])

答案应该如下:

ans = array([[1,8,3,0,4],
             [1,8,9,0,4]])

Answer 1

方法 #1

一种方法 np.unique -

# Find out the unique elements and their starting positions
unq_data, idx = np.unique(data,return_index=True)

# Find out the positions for each unique element, their duplicate positions
dup_idx = np.setdiff1d(np.arange(data.size),idx)

# Set those duplicate positioned elemnents to 0s
data[dup_idx] = 0

sample 运行-

In [46]: data
Out[46]: array([1, 8, 3, 3, 4, 1, 3, 3, 9, 4])

In [47]: unq_data, idx = np.unique(data,return_index=True)
    ...: dup_idx = np.setdiff1d(np.arange(data.size),idx)
    ...: data[dup_idx] = 0
    ...: 

In [48]: data
Out[48]: array([1, 8, 3, 0, 4, 0, 0, 0, 9, 0])

方法 #2

您还可以使用排序和微分作为更快的方法-

# Get indices  for sorted data
sort_idx = np.argsort(data)

# Get duplicate indices and set those in data to 0s
dup_idx = sort_idx[1::][np.diff(np.sort(data))==0]
data[dup_idx] = 0

运行时测试 -

In [110]: data = np.random.randint(0,100,(10000))
     ...: data1 = data.copy()
     ...: data2 = data.copy()
     ...: 

In [111]: def func1(data):
     ...:     unq_data, idx = np.unique(data,return_index=True)
     ...:     dup_idx = np.setdiff1d(np.arange(data.size),idx)
     ...:     data[dup_idx] = 0
     ...: 
     ...: def func2(data):
     ...:     sort_idx = np.argsort(data)
     ...:     dup_idx = sort_idx[1::][np.diff(np.sort(data))==0]
     ...:     data[dup_idx] = 0
     ...:     

In [112]: %timeit func1(data1)
1000 loops, best of 3: 1.36 ms per loop

In [113]: %timeit func2(data2)
1000 loops, best of 3: 467 µs per loop

---

扩展到二维案例:

方法 #2 可以扩展到适用于二维数组的情况，避免像这样的任何循环 -

# Get indices  for sorted data
sort_idx = np.argsort(data,axis=1)

# Get sorted linear indices
row_offset = data.shape[1]*np.arange(data.shape[0])[:,None]
sort_lin_idx = sort_idx[:,1::] + row_offset

# Get duplicate linear indices and set those in data as 0s
dup_lin_idx = sort_lin_idx[np.diff(np.sort(data,axis=1),axis=1)==0]
data.ravel()[dup_lin_idx] = 0

sample 运行-

In [6]: data
Out[6]: 
array([[1, 8, 3, 3, 4, 0, 3, 3],
       [1, 8, 9, 9, 4, 8, 7, 9],
       [1, 8, 9, 9, 4, 8, 7, 3]])

In [7]: sort_idx = np.argsort(data,axis=1)
   ...: row_offset = data.shape[1]*np.arange(data.shape[0])[:,None]
   ...: sort_lin_idx = sort_idx[:,1::] + row_offset
   ...: dup_lin_idx = sort_lin_idx[np.diff(np.sort(data,axis=1),axis=1)==0]
   ...: data.ravel()[dup_lin_idx] = 0
   ...: 

In [8]: data
Out[8]: 
array([[1, 8, 3, 0, 4, 0, 0, 0],
       [1, 8, 9, 0, 4, 0, 7, 0],
       [1, 8, 9, 0, 4, 0, 7, 3]])