nlp - WordNet 词典编纂器文件究竟是什么？了解 WordNet 的工作原理

Question

想了解WordNet的文件格式，主要文档是WNDB和 WNINPUT .正如我在 WNDB 中所了解的，有名为 index.something 的文件和 data.something ，这里something可以 noun, adv, vrb, adj .

所以，如果我想了解有关 dog 这个词的信息作为 noun ，我会查看 index.noun , 搜索词 dog ，这给了我一行:

dog n 7 5 @ ~ #m #p %p 7 1 02086723 10133978 10042764 09905672 07692347 03907626 02712903  

根据 WNDB 文档，这条线代表这些数据:

lemma  pos  synset_cnt  p_cnt  [ptr_symbol...]  sense_cnt  tagsense_cnt   synset_offset  [synset_offset...] 

哪里 lemma是这个词， pos是表明它是名词的标识符， synset_cnt告诉我们这个词包含多少个同义词， p_cnt告诉我们有多少指向这些同义词集的指针， [ptr_symbol]是一个指针数组， sense_cnt和 tagsense_cnt我不明白，想解释一下，和 synset_offset是一个或多个要研究的同义词集 data.noun文件

好的，所以我知道这些指针指向某些东西，以下是它们的描述，如 WNINPUT 所写:

@    Hypernym 
 ~    Hyponym 
#m    Member holonym 
#p    Part holonym 
%p    Part meronym 

我不知道如何为这个名词找到一个 Hypernym，但让我们继续:

其他重要数据是 synset_offset s，它们是:

02086723 10133978 10042764 09905672 07692347 03907626 02712903  

我们来看第一个， 02086723 , 在 data.noun :

02086723 05 n 03 dog 0 domestic_dog 0 Canis_familiaris 0 023 @ 02085998 n 0000 @ 01320032 n 0000 #m 02086515 n 0000 #m 08011383 n 0000 ~ 01325095 n 0000 ~ 02087384 n 0000 ~ 02087513 n 0000 ~ 02087924 n 0000 ~ 02088026 n 0000 ~ 02089774 n 0000 ~ 02106058 n 0000 ~ 02112993 n 0000 ~ 02113458 n 0000 ~ 02113610 n 0000 ~ 02113781 n 0000 ~ 02113929 n 0000 ~ 02114152 n 0000 ~ 02114278 n 0000 ~ 02115149 n 0000 ~ 02115478 n 0000 ~ 02115987 n 0000 ~ 02116630 n 0000 %p 02161498 n 0000 | a member of the genus Canis (probably descended from the common wolf) that has been domesticated by man since prehistoric times; occurs in many breeds; "the dog barked all night" 

如您所见，我们找到了以 02086723 开头的行.该行的内容在 WNDB 中描述为:

synset_offset  lex_filenum  ss_type  w_cnt  word  lex_id  [word  lex_id...]  p_cnt  [ptr...]  [frames...]  |   gloss 

我们已经知道的synset_offset，

lex_filenum说在哪个词典编纂者文件中是我们的词(这是我最不明白的部分) ,
ss_type是 n它告诉我们它是一个名词，
w_cnt :两位十六进制整数，表示同义词集中的单词数，在本例中为 03 ，这意味着我们在这个同义词集中有 3 个单词: dog 0 domestic_dog 0 Canis_familiaris 0 ，每一个后面跟着一个数字，称为:
lex_id : 一位十六进制整数，当附加到 lemma 时，唯一标识词典编纂者文件中的意义

p_cnt: counts the number of pointers, which in our case is `023`, so we have 23 pointers, wow

后 p_cnt ，然后是指针，每个指针的格式为:

pointer_symbol  synset_offset  pos  source/target 

哪里 pointer_symbol是关于像我展示的那些符号 (@, ~, ...),
synset_offset : 是 pos对应的数据文件中目标synset的字节偏移量
source/target : 字段区分词法和语义指针。它是一个四字节字段，包含两个两位十六进制整数。前两位数字表示当前(源)同义词集中的词号，后两位数字表示目标同义词集中的词号。值 0000 表示 pointer_symbol 表示当前(源)同义词集和由 synset_offset 指示的目标同义词集之间的语义关系。

好的，让我们检查第一个指针:

@ 02085998 n 0000

这是一个带有符号 @ 的指针，表示它是 Hypernym , 并指向具有偏移量 02085998 的同义词集类型 n (名词)和 source/target是 0000
当我在 data.noun 中搜索时，我得到

02085998 05 n 02 canine 0 canid 0 011 @ 02077948 n 0000 #m 02085690 n 0000 + 02688440 a 0101 ~ 02086324 n 0000 ~ 02086723 n 0000 ~ 02116752 n 0000 ~ 02117748 n 0000 ~ 02117987 n 0000 ~ 02119787 n 0000 ~ 02120985 n 0000 %p 02442560 n 0000 | any of various fissiped mammals with nonretractile claws and typically long muzzles  

这是一个 Hypernym的 dog .所以这就是你如何找到同义词之间的关系。我猜狗行中的指针符号只是为了告知我可以为狗这个词找到哪些类型的关系？不是多余的吗？因为这些指针符号已经在每个 synset_offsets 中了。正如我们所见。当我们看每一个 synset_offset在 data.noun ，我们可以看到那些指针符号，那么为什么它们在 index.noun 中是必要的文件？

另外，请注意我根本没有使用 lexicographers 文件。我知道在 data.noun ，特别是在该领域 lex_filenum ，我可以知道 dog的数据结构在哪里位于，但 的结构是什么？ ?如您所见，我可以通过查看 index 找到上位词和许多其他关系。和 data文件，我没有使用任何所谓的词典编纂者文件

Answer 1

是的，Wordnet 文档很难阅读...
您正在查找此页面:https://wordnet.princeton.edu/wordnet/man/lexnames.5WN.html

During WordNet development synsets are organized into forty-five lexicographer files based on syntactic category and logical groupings

这些分组是超下位词层次本体的某种平行集群(平面分组)。
简而言之:
从文档:
文件格式 [在 WordNet-3.0/dict/ 中的词典编纂者文件]

Each line in lexnames contains 3 tab separated fields, and is terminated with a newline character. The first field is the two digit decimal integer file number. (The first file in the list is numbered 00 .) The second field is the name of the lexicographer file that is represented by that number, and the third field is an integer that indicates the syntactic category of the synsets contained in the file. This is simply a shortcut for programs and scripts, since the syntactic category is also part of the lexicographer file's name.

在外行的解释中(我):

It's just a standard of how you should assign the values for the 2nd column in the files, e.g. data.nouns, data.verbs, etc.

Traditionally, Wordnet creators/maintainers should name their files accordingly but sometimes, it's easier to just put all nouns together and use the index of denote the synset's category.

类别的准则如下:

File Number Name    Contents
00  adj.all all adjective clusters
01  adj.pert    relational adjectives (pertainyms)
02  adv.all all adverbs
03  noun.Tops   unique beginner for nouns
04  noun.act    nouns denoting acts or actions
05  noun.animal nouns denoting animals
06  noun.artifact   nouns denoting man-made objects
07  noun.attribute  nouns denoting attributes of people and objects
08  noun.body   nouns denoting body parts
09  noun.cognition  nouns denoting cognitive processes and contents
10  noun.communication  nouns denoting communicative processes and contents
11  noun.event  nouns denoting natural events
12  noun.feeling    nouns denoting feelings and emotions
13  noun.food   nouns denoting foods and drinks
14  noun.group  nouns denoting groupings of people or objects
15  noun.location   nouns denoting spatial position
16  noun.motive nouns denoting goals
17  noun.object nouns denoting natural objects (not man-made)
18  noun.person nouns denoting people
19  noun.phenomenon nouns denoting natural phenomena
20  noun.plant  nouns denoting plants
21  noun.possession nouns denoting possession and transfer of possession
22  noun.process    nouns denoting natural processes
23  noun.quantity   nouns denoting quantities and units of measure
24  noun.relation   nouns denoting relations between people or things or ideas
25  noun.shape  nouns denoting two and three dimensional shapes
26  noun.state  nouns denoting stable states of affairs
27  noun.substance  nouns denoting substances
28  noun.time   nouns denoting time and temporal relations
29  verb.body   verbs of grooming, dressing and bodily care
30  verb.change verbs of size, temperature change, intensifying, etc.
31  verb.cognition  verbs of thinking, judging, analyzing, doubting
32  verb.communication  verbs of telling, asking, ordering, singing
33  verb.competition    verbs of fighting, athletic activities
34  verb.consumption    verbs of eating and drinking
35  verb.contact    verbs of touching, hitting, tying, digging
36  verb.creation   verbs of sewing, baking, painting, performing
37  verb.emotion    verbs of feeling
38  verb.motion verbs of walking, flying, swimming
39  verb.perception verbs of seeing, hearing, feeling
40  verb.possession verbs of buying, selling, owning
41  verb.social verbs of political and social activities and events
42  verb.stative    verbs of being, having, spatial relations
43  verb.weather    verbs of raining, snowing, thawing, thundering
44  adj.ppl participial adjectives

所以例如在 WordNet-3.0/dict/data.noun ，我们看到以下几行:

00034213 03 n 01 phenomenon 0 008 @ 00029677 n 0000 ~ 11408559 n 0000 ~ 11408733 n 0000 ~ 11408914 n 0000 ~ 11410625 n 0000 ~ 11418138 n 0000 ~ 11418460 n 0000 ~ 11529295 n 0000 | any state or process known through the senses rather than by intuition or reasoning  
00034479 04 n 01 thing 0 001 @ 00037396 n 0000 | an action; "how could you do such a thing?"  

查看第二列，为 phenomenon值为 03指向 noun.Tops .
对于 thing ，它的值为 04指的是 noun.act .

恕我直言，根据使用情况，这些分配可能没有用。它们主要用于创建 wordnet 以及我们如何轻松地将本体层次结构扁平化为简单的扁平集群。