apache - Nifi org.apache.thrift.transport.TTransportException

Question

我有Nifi 1.4.0和Hive 2.3.0。 Metastore Service运行正常，但由于某些原因Nifi无法执行PutHiveStreaming Processor。

以下是完整的堆栈。有什么想法吗？

        at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordsError$1(PutHiveStreaming.java:527)
        at org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$392/1467727491.apply(Unknown Source)
        at org.apache.nifi.processor.util.pattern.ExceptionHandler$OnError.lambda$andThen$0(ExceptionHandler.java:54)
        at org.apache.nifi.processor.util.pattern.ExceptionHandler$OnError$$Lambda$394/2094052256.apply(Unknown Source)
        at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordError$2(PutHiveStreaming.java:545)
        at org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$389/23901131.apply(Unknown Source)
        at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:148)
        at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$12(PutHiveStreaming.java:677)
        at org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$383/664174107.process(Unknown Source)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2174)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2144)
        at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:631)
        at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$4(PutHiveStreaming.java:555)
        at org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$379/701280946.execute(Unknown Source)
        at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
        at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184)
        at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:555)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://
Caused by: org.apache.nifi.util.hive.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {m
Caused by: org.apache.thrift.transport.TTransportException: null

Answer 1

Apache NiFi 1.4.0中的Hive处理器是针对Apache NiFi 1.2.1构建的，因此不能保证可与Apache Hive 2.3.0一起使用。另外，Apache NiFi和特定于供应商的Hive版本之间可能不兼容。例如，如果您使用的是Hortonworks Data Platform(HDP)，则其Hive版本基于1.2.x，但更接近于2.0。 Apache NiFi的Hive处理器与HDP 2.5+不兼容，您可能希望使用仅NiFi的Hortonworks Data Flow(HDF)软件包。 NiFi的此版本是针对HDP Hive 1.2.x版本构建的。

话虽如此，但由于NiFi基准仍为1.2.x，因此上述解决方案均不能保证可与Hive 2.x(无论是Apache还是特定于供应商)一起使用。