javascript - 如何使用javascript将字符串解析为单词和标点符号

Question

我有一个字符串 test="你好，你们都好吗，我希望它很好!很好。期待见到你。

我正在尝试使用 javascript 将字符串解析为单词和标点符号。我能够分隔单词，但随后使用正则表达式标点符号消失

var result= test.match(/\b(\w|')+\b/g);

所以我的预期输出是

hello
how 
are 
you
all
doing
,
I
hope
that
it's
good
!
and 
fine
.
Looking
forward
to
see
you

Answer 1

简单方法

如果您使用第一种方法，并且 javascript 对“单词”的定义匹配。下面是一种更可定制的方法。

试试 test.split(/\s*\b\s*/)。它按单词边界 (\b) 拆分并吃掉空格。

"hello how are you all doing, I hope that it's good! and fine. Looking forward to see you."
    .split(/\s*\b\s*/);
// Returns:
["hello",
"how",
"are",
"you",
"all",
"doing",
",",
"I",
"hope",
"that",
"it",
"'",
"s",
"good",
"!",
"and",
"fine",
".",
"Looking",
"forward",
"to",
"see",
"you",
"."]

它是如何工作的。

var test = "This is. A test?"; // Test string.

// First consider splitting on word boundaries (\b).
test.split(/\b/); //=> ["This"," ","is",". ","A"," ","test","?"]
// This almost works but there is some unwanted whitespace.

// So we change the split regex to gobble the whitespace using \s*
test.split(/\s*\b\s*/) //=> ["This","is",".","A","test","?"]
// Now the whitespace is included in the separator
// and not included in the result.

涉及更多的解决方案。

如果您希望像“isn`t”和“one-thousand”这样的词被视为单个词，而 javascript 正则表达式将它们视为两个词，您将需要创建自己的词定义。

test.match(/[\w-']+|[^\w\s]+/g) //=> ["This","is",".","A","test","?"]

工作原理

这分别使用交替匹配实际单词和标点字符。正则表达式的前半部分 [\w-']+ 匹配您认为是单词的任何内容，后半部分 [^\w\s]+ 匹配任何内容你考虑标点符号。在此示例中，我只使用了不是单词或空格的任何内容。我还在末尾添加了一个 + 以便将多字符标点符号(例如正确编写的 ?! ‽)视为单个字符，如果您不想删除 +。