Sonarqube升级到6.7 : Unrecoverable indexation failures

Question

目前我们正处于从 v4.5.7 到 v6.7.1 的升级过程中。

SonarQube已成功升级到v5.6.7(包括数据库升级)，目前正在更新到v6.7.1。

新的 SonarQube 版本在空数据库 ( bundled H2 DB ) 上启动，运行良好。之后，所有插件及其版本都会得到相应的验证和更新。

现在 Sonar 服务启动失败，并出现错误后台初始化失败。停止 SonarQube java.lang.IllegalStateException:不可恢复的索引失败。

我删除了 data/es5 文件夹并重新尝试启动 sonar 但没有成功。非常感谢任何解决该问题的帮助。

网络日志:

  INFO  web[][o.s.p.ProcessEntryPoint] Starting web
  INFO  web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
  INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
  INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
  INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
  INFO  web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
  INFO  web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9002]
  INFO  web[][o.s.s.p.LogServerVersion] SonarQube Server / 6.7.1.35068 / 426519346f51f7b980a76f9050f983110550509d
  INFO  web[][o.sonar.db.Database] Create JDBC data source for jdbc:oracle:thin:@host:****:instance
  INFO  web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /.../sonar/sonarqube-6.7.1
  INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Branch / 1.0.0.507 / e9d161eaa2993f79890dd5d08230f3f566e6be93
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Checkmarx / 8.41.0-20161229-0945
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Developer / 1.0.0.240 / 25c709aeb128caa14725b943422cca19def50493
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Flex / 2.3 / 79167f762219bea1fd181ff5ef6ca4b34abd1675
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin GitHub / 1.4.2.1027 / dcfb02a5fa40c219d527563f5b8c793966523ca7
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Governance / 2.0.2.3011 / 4756b2f9d98d28cf2762d91eba1b48839a1edbe1
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Groovy / 1.5 / dcbcd1eeb8f61c8351e5dd1e51e11e52d186df35
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin LDAP / 2.2.0.608 / 79dc3fa4393a29667673c70182f3016288b548b7
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin License / 3.3.0.1341 / 6b63240af64c92afe9421ece35c99c70f1b97991
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Pitest / 0.9 / 0c011514ce025e5524fdcaa0c917eacf9d40f312
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SVG Badges / 3.0.1
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarABAP / 3.5.0.1080 / c37baa20eb9d24a27961ffe25d95ae4b716a3193
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarC# / 6.5.0.3766 / 6011a1e9fb002edb044ae3092d92bcfb7d9d8a2d
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarCFamily / 5.0.0.9359 / 6001ee3bce59e4bb9a17e5edbe40b4b99bc2901a
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarCOBOL / 4.1.1.2663 / 846e542a08e9e472f0216da6c30e66a96bcbc821
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarJS / 3.2.0.5506 / 86da09565f891121df248835665f5d8ed98772b6
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarJava / 4.15.0.12310 / 572454b93016ec73a53fe0e07b2ffdc356d21ba9
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPHP / 2.11.0.2485 / 741861a29e5f9a26c6c99c06268facb6c4f4a882
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPLI / 1.7.0.1117 / 2e07afcb9ecb6a2e96c980b48f7195b82ffdd014
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPLSQL / 3.2.0.1753 / 455f471cfde991b2cd1a86a939ff410bf3006059
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarPython / 1.8.0.1496 / 3fe3bc4d0273a5721ea2fb368dc45b1bb82fede3
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarQube :: Plugins :: SCM :: Git / 1.3.0.869 / 4da53e3f9e55f4f2e5796625cb0c5768ed152079
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarQube :: Plugins :: SCM :: SVN / 1.6.0.860 / 2111fdbd1dddda4ad6d4ed6486fd0b18c1010d3b
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarRPG / 2.2.0.1005 / 146ae32dade755799da4fa9655e865d8757fddfa
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarSwift / 3.1.0.2067 / 22db8593d10c835675860598372d41ba8d5f0b8e
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarTS / 1.1.0.1079 / 042c9e65239a47d92d305f9767f730b3cc1e5ed3
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarTSQL / 1.1.0.2177 / e8811dfecb71d328c4deca89297bc100b270313c
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarVB / 4.2.0.248 / 75e2175c7464b814699abd6719b964c027717683
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarVB6 / 2.3.0.992 / 8c9e2016cb2e325b2fdeba4d303181b33bbbda4b
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin SonarXML / 1.4.3.1027 / 39588245cecf538bb27be4e496ff303b0143d20b
  INFO  web[][o.s.s.p.ServerPluginRepository] Deploy plugin Web / 2.5.0.476 / 636872f5d37fa7a440fe07d08d504e1a881225e5
  INFO  web[][c.c.sonar.CxPlugin] Checkmarx plugin loaded
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@6cdba960 [pattern=UrlPattern{inclusions=[/api/system/migrate_db/*, ...], exclusions=[/api/properties*, ...]}]
  INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port ****
  INFO  web[][o.s.s.p.UpdateCenterClient] Update center: https://update.sonarsource.org/update-center.properties (no proxy)
  INFO  web[][c.q.p.s.b.f.PreferredFontProvider] SVGImageGenerator will be using font 'Verdana' in order to compute SVG badges width.
  INFO  web[][c.q.p.s.b.w.SVGImageGenerator] SVGImageGenerator is now ready.
  INFO  web[][c.q.p.s.b.w.g.QualityGateBadgeGenerator] QualityGateBadgeGenerator is now ready.
  INFO  web[][c.q.p.s.b.w.m.MeasureBadgeGenerator] MeasureBadgeGenerator is now ready.
  INFO  web[][c.q.p.s.b.w.c.CeActivityBadgeGenerator] CeActivityBadgeGenerator is now ready.
  WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/editions/show_license
  WARN  web[][o.s.a.s.w.WebService$Action] The response example is not set on action api/views/definition
  INFO  web[][o.s.s.s.LogServerId] Server ID: *********
  WARN  web[][o.s.s.u.DeprecatedViews] Widget 'Checkmarx' (checkmarx-project-widget) is ignored. See org.sonar.api.web.page.PageDefinition to define pages.
  WARN  web[][o.s.s.u.DeprecatedViews] Page 'Checkmarx' (/cx_config/index) is ignored. See org.sonar.api.web.page.PageDefinition to define pages.
  INFO  web[][org.sonar.INFO] Security realm: LDAP
  INFO  web[][org.sonar.INFO] Security realm started
  INFO  web[][o.s.s.t.TelemetryDaemon] Sharing of SonarQube statistics is enabled.
  INFO  web[][o.s.s.n.NotificationDaemon] Notification service started (delay 60 sec.)
  INFO  web[][o.s.s.s.GeneratePluginIndex] Generate scanner plugin index
  INFO  web[][o.s.s.s.RegisterPlugins] Register plugins
  INFO  web[][o.s.s.s.RegisterMetrics] Register metrics
  INFO  web[][o.s.s.r.RegisterRules] Register rules
  INFO  web[][o.s.s.q.BuiltInQProfileRepositoryImpl] Load quality profiles
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Register quality profiles
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile cpp/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile c/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile rpg/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile js/Sonar way Recommended
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile js/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile py/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile pli/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile tsql/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile vb/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile plsql/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile cs/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile java/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile cobol/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile web/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile flex/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile xml/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile php/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile php/PSR-2
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile php/Drupal
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile vbnet/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile abap/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile grvy/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile objc/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile swift/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile ts/Sonar way
  INFO  web[][o.s.s.q.RegisterQualityProfiles] Update profile ts/Sonar way recommended
  INFO  web[][o.s.s.s.RegisterPermissionTemplates] Register permission templates
  INFO  web[][o.s.s.s.RenameDeprecatedPropertyKeys] Rename deprecated property keys
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceFilter@1c655890 [pattern=UrlPattern{inclusions=[/api/measures/component/*, ...], exclusions=[/api/properties*, ...]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.DeprecatedPropertiesWsFilter@4f45e5bd [pattern=UrlPattern{inclusions=[/api/properties/*], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.ws.WebServiceReroutingFilter@29719313 [pattern=UrlPattern{inclusions=[/api/components/bulk_update_key, ...], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.InitFilter@694be25 [pattern=UrlPattern{inclusions=[/sessions/init/*], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.OAuth2CallbackFilter@61014c56 [pattern=UrlPattern{inclusions=[/oauth2/callback/*], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LoginAction@44a4512f [pattern=UrlPattern{inclusions=[/api/authentication/login], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.LogoutAction@67e2e76 [pattern=UrlPattern{inclusions=[/api/authentication/logout], exclusions=[]}]
  INFO  web[][o.s.s.p.w.MasterServletFilter] Initializing servlet filter org.sonar.server.authentication.ws.ValidateAction@12d61b62 [pattern=UrlPattern{inclusions=[/api/authentication/validate], exclusions=[]}]
  INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] ...
  INFO  web[][o.s.s.es.BulkIndexer] 201463 requests processed (3357 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 357701 requests processed (2603 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 506390 requests processed (2478 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 636193 requests processed (2163 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 766613 requests processed (2173 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (2220 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 899827 requests processed (0 items/sec)
  INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [components/component] done | time=815372ms
  INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [views/view] ...
  INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [views/view] done | time=3122ms
  INFO  web[][o.s.s.e.IndexerStartupTask] Indexing of type [issues/issue] ...
  INFO  web[][o.s.s.es.BulkIndexer] 758897 requests processed (12648 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 1395232 requests processed (10605 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 1989212 requests processed (9899 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 2567812 requests processed (9643 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 3159399 requests processed (9859 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 3320485 requests processed (2684 items/sec)
  INFO  web[][o.s.s.es.BulkIndexer] 3320485 requests processed (0 items/sec)
  ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
  java.lang.IllegalStateException: Unrecoverable indexation failures
at org.sonar.server.es.IndexingListener$1.onFinish(IndexingListener.java:39)
at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:117)
at org.sonar.server.issue.index.IssueIndexer.doIndex(IssueIndexer.java:247)
at org.sonar.server.issue.index.IssueIndexer.indexOnStartup(IssueIndexer.java:95)
at org.sonar.server.es.IndexerStartupTask.indexUninitializedTypes(IndexerStartupTask.java:68)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.sonar.server.es.IndexerStartupTask.execute(IndexerStartupTask.java:55)
at java.util.Optional.ifPresent(Optional.java:159)
at org.sonar.server.platform.platformlevel.PlatformLevelStartup$1.doPrivileged(PlatformLevelStartup.java:84)
at org.sonar.server.user.DoPrivileged.execute(DoPrivileged.java:45)
at org.sonar.server.platform.platformlevel.PlatformLevelStartup.start(PlatformLevelStartup.java:80)
at org.sonar.server.platform.Platform.executeStartupTasks(Platform.java:196)
at org.sonar.server.platform.Platform.access$400(Platform.java:46)
at org.sonar.server.platform.Platform$1.lambda$doRun$1(Platform.java:121)
at org.sonar.server.platform.Platform$AutoStarterRunnable.runIfNotAborted(Platform.java:371)
at org.sonar.server.platform.Platform$1.doRun(Platform.java:121)
at org.sonar.server.platform.Platform$AutoStarterRunnable.run(Platform.java:355)
at java.lang.Thread.run(Thread.java:745)
  INFO  web[][o.s.p.StopWatcher] Stopping process

Answer 1

终于，我们的问题得到了解决。系统配置似乎是此问题的根源。我们在一台大约有的服务器上运行它。 25GB 可用空间并最终出现索引错误。使用具有更多可用空间的服务器有助于将 Sonarqube 成功升级到 v6.7.1。