sql - 从Polybase连接到hadoop时，连接被拒绝

Question

尝试在Ubuntu 16.04中创建从SQL Server 2017到Hadoop的外部表时，抛出以下错误

Msg 105019, Level 16, State 1, Line 1 EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: Call From DESKTOP-VE8KNAG/xxx.xxx.x.xxx to xxx.xxx.x.x:54310 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused: Error [Call From DESKTOP-VE8KNAG/1xxx.xxx.x.xxx to xxx.xxx.x.x:54310 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused] occurred while accessing external file.'

外部源位置是从core-site.xml中获取的

Tmp文件夹已创建并向用户添加了权限，并将其添加到core-site.xml中

所有节点都在Hadoop中运行
10625数据节点
10869 SecondaryNameNode
17113资源管理器
17434节点管理器
10490名称节点
21566 Jps

sql查询

create EXTERNAL DATA SOURCE [HDP2] 
WITH (TYPE = HADOOP,
 LOCATION = N'hdfs://xxx.xxx.x.x:54310',
 CREDENTIAL = [HDPUser])
GO

create EXTERNAL TABLE [dbo].CLASS_DIM_EXP (
    [CLASS_ID] [varchar](8) NOT NULL,
    [CLASS_DESC] [varchar](100) NULL,
    [INSERT_DATE] [datetime2](7) NOT NULL,
    [LAST_UPDATE_DATETIME] [datetime2](7) NOT NULL)
WITH   (LOCATION='/user/pdw_user',
DATA_SOURCE = HDP2,
FILE_FORMAT = TSV,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0);

核心-site.xml

<property>
  <name>hadoop.tmp.dir</name>
  <value>/app/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
</property>

<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>

是否需要更改某些内容？

Answer 1

这里的问题是cores-site.xml文件包含hdfs:// localhost:54310，需要用对应的IP地址hdfs://xxx.xxx.x.x:54310替换它。