个人中心
gpt4 book ai didi

windows - webHDFS curl-在Windows上协商

转载 作者：行者123 更新时间：2023-12-02 20:16:57 24 4
gpt4 key购买 nike

以下命令在Linux上有效，但在Windows上失败。在运行命令之前，我使用kinit获取有效的Kerberos票证。

curl -v -i --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt "http://<HDP-Cluster>:50070/webhdfs/v1/?op=LISTSTATUS"
预期结果(Linux输出): 
> {"FileStatuses":{"FileStatus":[
> {"accessTime":0,"blockSize":0,"childrenNum":17,"fileId":18894,"group":"hadoop","length":0,"modificationTime":1589364305719,"owner":"yarn","pathSuffix":"app-logs","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":16432,"group":"hdfs","length":0,"modificationTime":1566323580684,"owner":"hdfs","pathSuffix":"apps","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16419,"group":"hdfs","length":0,"modificationTime":1566323394432,"owner":"hdfs","pathSuffix":"atsv2","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},
> {"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":77127,"group":"hdfs","length":0,"modificationTime":1566983700653,"owner":"hdfs","pathSuffix":"benchmarks","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
> ]}}
Windows的输出: 
* Connected to <SERVER> (<IP>) port 50070 (#0)
* Server auth using Negotiate with user ''
> GET /webhdfs/v1/?op=LISTSTATUS HTTP/1.1
> Host: <SERVER>:50070
> Authorization: Negotiate TlRMTVNTUAABAAAAt4II4gAAAAAAAAAAAAAAAAAAAAAKAGNFAAAADw==
> User-Agent: curl/7.70.0
> Accept: */*
> Cookie: hadoop.auth=
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 java.lang.IllegalArgumentException
< Date: Wed, 15 Jul 2020 09:32:34 GMT
< Date: Wed, 15 Jul 2020 09:32:34 GMT
< Pragma: no-cache
< X-FRAME-OPTIONS: SAMEORIGIN
* Replaced cookie hadoop.auth="" for domain <SERVER.Full.Domain>, path /, expire 0
< Set-Cookie: hadoop.auth=; Path=/; HttpOnly
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Type: text/html;charset=iso-8859-1
< Content-Length: 293
<
{ [293 bytes data]
100   293  100   293    0     0   3367      0 --:--:-- --:--:-- --:--:--  3367HTTP/1.1 403 java.lang.IllegalArgumentException
Date: Wed, 15 Jul 2020 09:32:34 GMT
Date: Wed, 15 Jul 2020 09:32:34 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Set-Cookie: hadoop.auth=; Path=/; HttpOnly
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 293

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 java.lang.IllegalArgumentException</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /webhdfs/v1/. Reason:
<pre>    java.lang.IllegalArgumentException</pre></p>
</body>
</html>

* Connection #0 to host <server> left intact
curl 版本窗口: 
curl -V
curl 7.70.0 (x86_64-w64-mingw32) libcurl/7.70.0 OpenSSL/1.1.1g (Schannel) zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.40.0
Release-Date: 2020-04-29
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP
curl版本Linux: 
curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.36 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets
我在 When using --negotiate with curl, is a keytab file required?上找到了有关如何在Linux上安装的详细说明，但找不到Windows的GSS-API，Windows的SSPI是否与Linux的GSS-API相同？

最佳答案

在以下两个方面，我找到了答案:
When using --negotiate with curl, is a keytab file required?

Caveat: the Windows versions of curl are compiled to support the Microsoft SSPI libraries, not the GSSAPI libraries that are required by Hadoop REST services


:(
Accessing kerberos secured WebHDFS without SPnego
显示一个Java代码，您在其中获得了一个委派 token 并将其传递给WebHDFS RestAPI。这样，您无需使用--negotiate。

关于windows - webHDFS curl-在Windows上协商，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/62912236/

24 4 0
文章推荐： php - Docker 镜像容器 - 我在哪里可以存储我的文件？
文章推荐： python - numpy安装缺少numpy.distutils，theano尝试使用它
文章推荐： nginx - Kubernetes Minikube Secrets 似乎未安装在 Pod 中
文章推荐： hadoop - hive 将字符串转换为时间戳会丢弃毫秒信息
行者123
个人简介

我是一名优秀的程序员,十分优秀！

滴滴打车优惠券免费领取
滴滴打车优惠券
全站热门文章
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com