Elasticsearch 到 BigQuery 管道部署在云数据融合实例上失败

Question

我正在部署一个数据融合管道，它从 elasticsearch 的索引中获取数据并将该数据加载到 bigQuery 表。管道仅由 Elasticsearch 插件连接器到 BigQuery 连接器组成。当我运行管道时，它会生成以下错误。
错误的来源是

io.cdap.cdap.internal.app.runtime.distributed.remote.RemoteExecutionTwillRunnerService#548-runtime-startup-1

实际错误是

java.io.IOException: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection timed out (Connection timed out)
    at io.cdap.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:88) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.distributed.remote.RemoteExecutionTwillPreparer.lambda$start$0(RemoteExecutionTwillPreparer.java:436) ~[na:na]
    at io.cdap.cdap.internal.app.runtime.distributed.remote.RemoteExecutionTwillRunnerService$ControllerFactory.lambda$create$0(RemoteExecutionTwillRunnerService.java:524) ~[na:na]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_232]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_232]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_232]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_232]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection timed out (Connection timed out)
    at com.jcraft.jsch.Util.createSocket(Util.java:349) ~[com.jcraft.jsch-0.1.54.jar:na]
    at com.jcraft.jsch.Session.connect(Session.java:215) ~[com.jcraft.jsch-0.1.54.jar:na]
    at com.jcraft.jsch.Session.connect(Session.java:183) ~[com.jcraft.jsch-0.1.54.jar:na]
    at io.cdap.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:85) ~[na:na]
    ... 7 common frames omitted
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_232]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_232]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_232]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_232]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_232]
    at java.net.Socket.connect(Socket.java:607) ~[na:1.8.0_232]
    at java.net.Socket.connect(Socket.java:556) ~[na:1.8.0_232]
    at java.net.Socket.<init>(Socket.java:452) ~[na:1.8.0_232]
    at java.net.Socket.<init>(Socket.java:229) ~[na:1.8.0_232]
    at com.jcraft.jsch.Util.createSocket(Util.java:343) ~[com.jcraft.jsch-0.1.54.jar:na]
    ... 10 common frames omitted

我该如何解决这个问题？

Answer 1

请参阅 https://cloud.google.com/data-fusion/docs/how-to/create-instance 中记录的网络要求.

具体来说:

New projects start with a default network. The default network is pre-populated with a firewall rule, default-allow-ssh, that allows incoming connections on TCP port 22 from any source to any instance in the network. If such a rule doesn't exist in the network used by your Cloud Data Fusion instance, you need to create such a rule.