apache-nifi - apache nifi Stateless - 无法设置 Controller 服务的参数(DBCPConnectionPool 1.10.0)

Question

我正在关注 NiFi 1.10 stateless guildeline创建一个在 mysql 数据库中执行 sql 的简单进程组。我已将必要的数据库 Controller 服务参数放入参数上下文。

它在 nifi canvas 中运行良好。然后我将它添加到注册表并准备一个 json parm 文件:stateless-simpledb.json

{
  "registryUrl": "http://localhost:18080",
  "bucketId": "cac8f127-e328-45c1-a4cb-0e03dc837ceb",
  "flowId": "cc2753f2-78f3-4449-a2fd-343dfeaafe15",
  "flowVersion": "3",
  "parameters": {
    "lastIngestId" : "20000",
   "mysql-jdbc-driver-name" : "com.mysql.jdbc.Driver",
   "db-user" : "root",
   "db-password" : "password",
   "db-con-url" : "jdbc:mysql://localhost:3306/mms",
   "jdbc-jar-path" : "/program/jdbc/mysql-connector-java.jar"

  }
}

并运行一次性命令:

/program/nifi/bin/nifi.sh stateless RunFromRegistry Once --file /app/poc/nifi-stateless/conf/stateless-simpledb.json

它引发错误:

       === FlowFileRepository Type ===
        org.apache.nifi.controller.repository.RocksDBFlowFileRepository
                org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
        org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
                org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
        org.apache.nifi.controller.repository.VolatileFlowFileRepository
                org.apache.nifi:nifi-framework-nar:1.10.0 || /program/nifi-1.10.0/work/stateless-nars/nifi-framework-nar-1.10.0.nar-unpacked
        === End FlowFileRepository types ===
23:32:32.626 [main] INFO org.apache.nifi.stateless.bootstrap.ExtensionDiscovery - Successfully discovered extensions in 4411 milliseconds
23:32:32.633 [main] DEBUG org.apache.nifi.stateless.core.ComponentFactory - Setting context class loader to org.apache.nifi.nar.InstanceClassLoader@50fa5938 (parent = org.apache.nifi.nar.NarClassLoader[/program/nifi-1.10.0/work/stateless-nars/nifi-dbcp-service-nar-1.10.0.nar-unpacked]) to create org.apache.nifi.dbcp.DBCPConnectionPool
23:32:32.647 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input #{jdbc-jar-path} found 1 Parameter references: [org.apache.nifi.parameter.StandardParameterReference@2d3eecda]
23:32:32.650 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input /program/jdbc/mysql-connector-java.jar found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 500 millis found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 8 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 0 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 8 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input 30 mins found 0 Parameter references: []
23:32:32.651 [main] DEBUG org.apache.nifi.parameter.ExpressionLanguageAwareParameterParser - For input -1 found 0 Parameter references: []
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.nifi.bootstrap.RunStatelessNiFi.main(RunStatelessNiFi.java:69)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.nifi.StatelessNiFi.main(StatelessNiFi.java:103)
        ... 5 more
Caused by: java.lang.RuntimeException: Failed to enable Controller Service {id=691ecc97-ff46-3a5e-8aad-37dc568bc247, name=MYSQL-MMS-stateless-test, type=class org.apache.nifi.dbcp.DBCPConnectionPool} because validation failed: ['Database Connection URL' is invalid because Database Connection URL is required, 'Database Driver Class Name' is invalid because Database Driver Class Name is required]
        at org.apache.nifi.stateless.core.StatelessControllerServiceLookup.enableControllerServices(StatelessControllerServiceLookup.java:133)
        at org.apache.nifi.stateless.core.StatelessFlow.<init>(StatelessFlow.java:153)
        at org.apache.nifi.stateless.core.StatelessFlow.createAndEnqueueFromJSON(StatelessFlow.java:469)
        at org.apache.nifi.stateless.runtimes.Program.runLocal(Program.java:133)
        at org.apache.nifi.stateless.runtimes.Program.launch(Program.java:67)
        ... 10 more

似乎 apache nifi 无状态功能无法设置 Controller 服务，即使它在“进程组”范围内也是如此。

谁有什么建议吗？

Answer 1

如评论中所述，这似乎是 Controller 服务验证的一个已知问题。

这可以通过使用 Nifi 1.12 及更高版本来避免，因为它已在以下 jira 中修复:https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-7380

虽然我不完全确定这一点，但这也可能只是表明您的 Controller 服务配置不正确。这值得仔细检查。