json - 将JSON Tweet导入到Hive中的问题

Question

我使用docker在cloudera quickstart上工作，我正在尝试将表创建到hive界面。
这是我的代码。

add jar hdfs:///user/cloudera/hive-serdes-1.0-SNAPSHOT.jar
drop table if exists tweets;
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweeted_status STRUCT<
 text:STRING,
 user1:STRUCT<screen_name:STRING,name:STRING>,
 retweet_count:INT>,
entities STRUCT<
 urls:ARRAY<STRUCT<expanded_url:STRING>>,
 user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
 hashtags:ARRAY<STRUCT<text:STRING>>>,
text STRING,
user1 STRUCT<
 screen_name:STRING,
 name:STRING,
 friends_count:INT,
 followers_count:INT,
 statuses_count:INT,
 verified:BOOLEAN,
 utc_offset:INT,
 time_zone:STRING>,
 in_reply_to_screen_name STRING
 )
 ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
 LOCATION '/user/cloudera/';
 load data inpath '/user/cloudera/search.json' into table tweets;

当我运行“从推文中选择*”时，出现此错误

Fetching results ran into the following error(s):
Bad status for request TFetchResultsReq(fetchType=0, operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, operationType=0, operationId=THandleIdentifier(secret='\xf2e\xcc\xb6v\x8eC"\xae^x\x89*\xd6j\xa7', guid='h\xce\xacgmZIP\x8d\xcc\xc0\xe8C\t\x1a\x0c')), orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=0, errorMessage='java.io.IOException: java.io.IOException: Not a file: hdfs://quickstart.cloudera:8020/user/cloudera/2015_11_18', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: java.io.IOException: Not a file: hdfs://quickstart.cloudera:8020/user/cloudera/2015_11_18:25:24', 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:366', 'org.apache.hive.service.cli.operation.OperationManager:getOperationNextRowSet:OperationManager.java:275', 'org.apache.hive.service.cli.session.HiveSessionImpl:fetchResults:HiveSessionImpl.java:752', 'sun.reflect.GeneratedMethodAccessor19:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:606', 

Answer 1

不要将您的用户文件夹用作Hive表位置。用户文件夹用于一般文件存储，例如它尝试读取的2015_11_18目录，而不是整个Hive结构。

例如，执行LOCATION '/user/cloudera/tweets';。

如果您不在意在删除表时是否删除了所有内容，也可以只创建一个常规托管表。