java - 我想使用 WordNet 查找单词相似度

Question

我正在做“网络视频分类”的最后一年项目，其中一部分是查找特定单词的相似(同义词)单词，我想从中删除相似的术语。

我懂Java语言，所以我选择了“Java的单词相似度”ws4j

为此我只使用了 WS4J1.0.1 jar 文件，即使我没有下载任何额外的文件，例如 WordNet 词法数据库或 sqlite 数据库来存储它。因为在该网站中他们提到所有内容都包含在这个 jar 文件中预编译的。

当我执行Demo程序时SimilarityCalculationDemo.java ，我收到以下错误:

    java.sql.BatchUpdateException: batch entry 0: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.Stmt.executeBatch(Stmt.java:226)
    at org.sqlite.Stmt.executeBatch(Stmt.java:226)
    at edu.cmu.lti.jawjaw.db.SQL.createIndexIfNotExists(SQL.java:118)
    at edu.cmu.lti.jawjaw.db.SQL.createSQLConnection(SQL.java:98)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:55)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
                             atedu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
        java.sql.SQLException: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.DB.newSQLException(DB.java:383)
    at org.sqlite.DB.newSQLException(DB.java:387)
    at org.sqlite.DB.throwex(DB.java:374)
    at org.sqlite.NativeDB.prepare(Native Method)
    at org.sqlite.DB.prepare(DB.java:123)
    at org.sqlite.Stmt.execute(Stmt.java:113)
    at edu.cmu.lti.jawjaw.db.SQL.setPragmaCacheSize(SQL.java:137)
    at edu.cmu.lti.jawjaw.db.SQL.createSQLConnection(SQL.java:99)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:55)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
java.sql.SQLException: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.DB.newSQLException(DB.java:383)
    at org.sqlite.DB.newSQLException(DB.java:387)
    at org.sqlite.DB.throwex(DB.java:374)
    at org.sqlite.NativeDB.prepare(Native Method)
    at org.sqlite.DB.prepare(DB.java:123)
    at org.sqlite.PrepStmt.<init>(PrepStmt.java:42)
    at org.sqlite.Conn.prepareStatement(Conn.java:404)
    at org.sqlite.Conn.prepareStatement(Conn.java:399)
    at org.sqlite.Conn.prepareStatement(Conn.java:383)
    at edu.cmu.lti.jawjaw.db.SQL.prepareStatements(SQL.java:151)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:56)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
Exception in thread "main" java.lang.NullPointerException
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:125)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
Java Result: 1

我使用 Netbeans IDE 7.4 和 JDK 6。

任何人都可以帮助我，如何克服这个问题，因为互联网上关于 ws4j 的可用文档较少。

Answer 1

嗯，我无法重现您的错误。对我来说，它使用 Eclipse 开箱即用，非常完美，所以我会尽力帮助您准确地重现我所做的事情

1. 从https://ws4j.googlecode.com/files/ws4j-1.0.1.jar下载ws4j-1.0.1.jar并确保下载后的大小为 41,362,723 字节(至少，这是 eclipse 在我的 linux 盒子中告诉我的)

2. 使用java 7

3. 创建一个简单的 Eclipse 项目并将 jar 放到那里。然后将jar添加到构建路径中(右键->构建路径->添加)

4. 创建适当的包和类来容纳演示类

5. 只需运行演示，您就会得到类似的结果

   edu.cmu.lti.ws4j.impl.HirstStOnge   0.0
   edu.cmu.lti.ws4j.impl.LeacockChodorow   1.3862943611198906
   edu.cmu.lti.ws4j.impl.Lesk  0.0
   edu.cmu.lti.ws4j.impl.WuPalmer  0.4
   edu.cmu.lti.ws4j.impl.Resnik    2.5031573470157453
   edu.cmu.lti.ws4j.impl.JiangConrath  0.11150424023847051
   edu.cmu.lti.ws4j.impl.Lin   0.3582442863008455
   edu.cmu.lti.ws4j.impl.Path  0.14285714285714285
   Done in 1951 msec.