hadoop - 无法根据查询参数从Hbase中获取数据

Question

如何从HBase获取数据？我有一个包含 empId、name、startDate、endDate 和其他列的表。现在我想从基于 empId、startDate 和 endDate 的 HBase 表中获取数据。在正常的 SQL 中，我可以使用:

select * from tableName where empId=val and date>=startDate and date<=endDate

我如何在 HBase 中执行此操作，因为它将数据存储为键值对？关键是empId。

Answer 1

在 HBase shell 中获取过滤行是很棘手的。由于 shell 是 JRuby-based，因此您也可以在此处使用 Ruby 命令:

import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import org.apache.hadoop.hbase.filter.BinaryComparator
import org.apache.hadoop.hbase.filter.FilterList
import java.text.SimpleDateFormat
import java.lang.Long

def dateToBytes(val)
  Long.toString(
    SimpleDateFormat.new("yyyy/MM/dd").parse(val).getTime()).to_java_bytes
end

# table properties
colfam='c'.to_java_bytes;
col_name='name';
col_start='startDate';
col_end='endDate';

# query params
q_name='name2';
q_start='2012/08/14';
q_end='2012/08/24';

# filters
f_name=SingleColumnValueFilter.new(
         colfam, col_name.to_java_bytes, 
         CompareFilter::CompareOp::EQUAL, 
         BinaryComparator.new(q_name.to_java_bytes));

f_start=SingleColumnValueFilter.new(
          colfam, col_start.to_java_bytes, 
          CompareFilter::CompareOp::GREATER_OR_EQUAL,
          BinaryComparator.new(dateToBytes(q_start)));

f_end=SingleColumnValueFilter.new(
          colfam, col_end.to_java_bytes, 
          CompareFilter::CompareOp::LESS_OR_EQUAL,
          BinaryComparator.new(dateToBytes(q_end)));

filterlist= FilterList.new([f_name, f_start, f_end]);

# get the result
scan 'mytable', {"FILTER"=>filterlist}

类似地在 Java 中构造一个 FilterList :

// Query params
String nameParam = "name2";
String startDateParam = "2012/08/14";
String endDateParam = "2012/08/24";

Filter nameFilter = 
  new SingleColumnValueFilter(colFam, nameQual, CompareOp.EQUAL,
        Bytes.toBytes(nameParam));

//getBytesFromDate(): parses startDateParam and create a byte array out of it
Filter startDateFilter = 
  new SingleColumnValueFilter(colFam, startDateQual,
        CompareOp.GREATER_OR_EQUAL, getBytesFromDate(startDateParam));

Filter endDateFilter = 
  new SingleColumnValueFilter(colFam, endDateQual,
        CompareOp.LESS_OR_EQUAL, getBytesFromDate(endDateParam));

FilterList filters = new FilterList();
filters.addFilter(nameFilter);
filters.addFilter(startDateFilter);
filters.addFilter(endDateFilter);

HTable htable = new HTable(conf, tableName);

Scan scan = new Scan();
scan.setFilter(filters);
ResultScanner rs = htable.getScanner(scan);
//process your result...