javascript - 字符串标记生成器无法正确处理空标记或连续的多个分隔符

Question

所以我在 javascript 中有一个简单的 strtok 实现。由于某种原因，如果您尝试在字符串内连续传递多个分隔符，它会将所有其他分隔符解析为字符。例如"This,, and that"由 "," 界定和" "将返回 [ This 的数组, , , and , that ]。第二个逗号，因为它紧跟在另一个分隔符后面，因此被视为非分隔符字符。

代码如下:

//@param str: a string to tokenize
//@param tok: an array of chars to be delimiters
//@return: an array of strings.

var strtok = function(str, tok)
{
    if(str === "") return [];

    var ret = [],
    _buffer = str,
    sub,
    b = _buffer.charAt(0),
    start = 0,
    i = 0,
    len = _buffer.length; 

    while(i <= len) {
        b = _buffer.charAt(i);
        tok.forEach(function(elm) {
            if (b === elm) {              
                sub = _buffer.slice(start, i);
                if(!(sub in tok)) ret.push(sub);
                start = ++i;
                return;
            } 
        });  
        i++;
    }
    ret.push(_buffer.slice(start, len));
    return ret;
}

var str = "This,, that";
var tok = '. ,'.split('');
console.log(strtok(str,tok));
// ["This", ",", "that"]
str = "This,,, that";
console.log(strtok(str, tok));
// ["This", ",", " that"] <- notice the space before that

Answer 1

索引过度递增，罪魁祸首是 start =++i;，修复方法是 start = i + 1;。