bash - 如何从 Bash 历史文件中删除重复的命令？

Question

我已经配置了我自己的.bash_myhistory

export HISTFILESIZE=
export HISTSIZE=
export HISTTIMEFORMAT="[%F %T] "

export HISTFILE=~/.bash_myhistory
PROMPT_COMMAND="history -a; history -r; $PROMPT_COMMAND"

当我运行 history 时，它显示了许多重复的输出:

 $ history | grep 'git rebase'
   75  [2018-05-23 16:39:39] git rebase -p dev_hypermouse 
  168  [2018-05-23 19:27:39] man git rebase 
  547  [2018-05-25 19:01:44] git rebase master 
  639  [2018-05-25 20:24:52] git rebase master 
  869  [2018-05-28 14:07:33] git rebase xxx
  921  [2018-05-28 16:12:20] git rebase dash_v2 
  922  [2018-05-28 16:12:33] man git rebase
  925  [2018-05-28 16:13:21] man git rebase
  927  [2018-05-28 16:15:42] git rebase xxx dash_v2 
  937  [2018-05-28 16:17:46] git rebase --onto dash_v2 xxx 296-ToS-component 
 2177  [2018-05-23 16:39:39] git rebase -p dev_hypermouse 
 2270  [2018-05-23 19:27:39] man git rebase 
 2649  [2018-05-25 19:01:44] git rebase master 
 2741  [2018-05-25 20:24:52] git rebase master 
 2971  [2018-05-28 14:07:33] git rebase xxx
 3023  [2018-05-28 16:12:20] git rebase dash_v2 
 3024  [2018-05-28 16:12:33] man git rebase
 3027  [2018-05-28 16:13:21] man git rebase
 3029  [2018-05-28 16:15:42] git rebase xxx dash_v2 
 3039  [2018-05-28 16:17:46] git rebase --onto dash_v2 xxx 296-ToS-component 
 4239  [2018-05-23 19:27:39] man git rebase 
 4618  [2018-05-25 19:01:44] git rebase master 
 4710  [2018-05-25 20:24:52] git rebase master 
 4940  [2018-05-28 14:07:33] git rebase xxx
 4992  [2018-05-28 16:12:20] git rebase dash_v2 
 4993  [2018-05-28 16:12:33] man git rebase
 4996  [2018-05-28 16:13:21] man git rebase
 4998  [2018-05-28 16:15:42] git rebase xxx dash_v2 
 5008  [2018-05-28 16:17:46] git rebase --onto dash_v2 xxx 296-ToS-component 
 ...

但是 $ cat ~/.bash_myhistory | grep 'git rebase' 不会:

man git rebase 
git rebase master 
git rebase master 
git rebase xxx
git rebase dash_v2 
man git rebase
man git rebase
git rebase xxx dash_v2 
git rebase --onto dash_v2 xxx 296-ToS-component 
man git rebase
man git rebase
history | grep git rebase
history | grep 'git rebase'

如何修复 history 的重复输出？

UPD

使用 export HISTCONTROL=ignoreboth:erasedups 历史看起来好多了，但重复项仍然存在:

$ history | grep 'git rebase'
   34  [2018-05-23 19:27:39] man git rebase 
  413  [2018-05-25 19:01:44] git rebase master 
  505  [2018-05-25 20:24:52] git rebase master 
  735  [2018-05-28 14:07:33] git rebase xxx
  787  [2018-05-28 16:12:20] git rebase dash_v2 
  788  [2018-05-28 16:12:33] man git rebase
  791  [2018-05-28 16:13:21] man git rebase
  793  [2018-05-28 16:15:42] git rebase xxx dash_v2 
  803  [2018-05-28 16:17:46] git rebase --onto dash_v2 xxx 296-ToS-component 
 2038  [2018-06-02 14:49:31] man git rebase
 2058  [2018-06-02 14:52:33] man git rebase
 2060  [2018-06-02 15:11:08] history | grep git rebase
 2061  [2018-06-02 15:11:13] history | grep 'git rebase'
 2063  [2018-06-02 15:12:45] cat .bash_myhistory | grep 'git rebase'
 2064  [2018-06-02 15:09:58] man git rebase
 2077  [2018-06-02 15:35:41] history | grep 'git rebase'
 2111  [2018-05-23 19:27:39] man git rebase 
 2490  [2018-05-25 19:01:44] git rebase master 
 2582  [2018-05-25 20:24:52] git rebase master 
 2812  [2018-05-28 14:07:33] git rebase xxx
 2864  [2018-05-28 16:12:20] git rebase dash_v2 
 2865  [2018-05-28 16:12:33] man git rebase
 2868  [2018-05-28 16:13:21] man git rebase
 2870  [2018-05-28 16:15:42] git rebase xxx dash_v2 
 2880  [2018-05-28 16:17:46] git rebase --onto dash_v2 xxx 296-ToS-component 
 4115  [2018-06-02 14:49:31] man git rebase
 4135  [2018-06-02 14:52:33] man git rebase
 4137  [2018-06-02 15:11:08] history | grep git rebase
 4138  [2018-06-02 15:11:13] history | grep 'git rebase'
 4140  [2018-06-02 15:12:45] cat .bash_myhistory | grep 'git rebase'
 4141  [2018-06-02 15:09:58] man git rebase
 4154  [2018-06-02 15:35:41] history | grep 'git rebase'

UPD
即使在添加 export HISTCONTROL=ignoreboth:erasedups 之后，历史看起来像:

25988  [2018-07-26 17:13:19] gd 1
25989  [2018-07-26 15:45:47] mc
25990  [2018-07-26 13:57:46] mc
25991  [2018-07-26 09:23:28] mc

我还注意到一些命令从历史记录中消失了 =(

Answer 1

export HISTCONTROL=ignoreboth:erasedups

来自 bash 手册页:

HISTCONTROL

A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not saved in the history list. A value of ignoredups causes lines matching the previous history entry to not be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.