apache - 扩展自定义 Apache Knox 服务以查询多个 HBase 表

Question

我目前正致力于扩展 Apache Knox 与 HDP 2.3.2 上的 HBase 交互的功能。我在 Apache Knox 上创建了一个名为 Decode 的新网关，用于查询 HBase。解码网关是使用 HBase 网关作为模板构建的。拓扑已被编辑为以下查询:

curl -ku admin:admin-password  -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/hbase/MyHBaseTable/HBaseRowKey123* 

将返回第 123 行的数据(在 Base 64 中)

有没有办法改变Decode网关的rewrite.xml和service.xml，这样查询就不需要通过Hbase了，例如:

curl -ku admin:admin-password  -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/MyHBaseTable/HBaseRowKey123* 

我知道不在内置 HBase 网关中使用这听起来可能很奇怪，但总体目标是扩展 Decode 网关，以便它能够从 HBase 查询多个表和/或多行数据，而不是一行一次。

目前我的 Decode rewrite.xml 是:

<rules>
  <rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
    <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
  </rule>
  <rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode{**}">
    <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
  </rule>
  <filter name="WEBHBASE/webhbase/status/outbound">
    <content type="*/json">
      <apply path="$[LiveNodes][*][name]" rule="WEBHBASE/webhbase/address/outbound"/>
    </content>
    <content type="*/xml">
      <apply path="/ClusterStatus/LiveNodes/Node/@name" rule="WEBHBASE/webhbase/address/outbound"/>
    </content>
  </filter>
</rules>

和 service.xml:

<service role="DECODE" name="decode" version="0.0.1">
     <routes>
         <route path="/decode/**"/>
     <route path="/decode/?**">
             <rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
         </route>
         <route path="/decode/**?**">
             <rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
         </route>
         <route path="/decode/status/cluster?**">
             <rewrite apply="WEBHBASE/webhbase/status/outbound" to="response.body"/>
         </route>
         <route path="/decode/*/regions?**">
             <rewrite apply="WEBHBASE/webhbase/regions/outbound" to="response.body"/>
         </route>
      </routes> <dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/> </service>

Answer 1

就像@lmccay 一样，我看不出您展示的内容有什么必然的错误，所以我决定亲自尝试一下。

在默认拓扑文件中，我将 WEBHBASE 服务复制到 DECODE 服务，如图所示。

/usr/hdp/current/knox-server/conf/topologies/default.xml

<service>
    <role>WEBHBASE</role>
    <url>http://my-hbase-hostname:60080</url>
</service>
<service>
    <role>DECODE</role>
    <url>http://my-hbase-hostname:60080</url>
</service>

然后我从

复制了 hbase service.xml 和 rewrite.xml

/usr/hdp/current/knox-server/data/services/hbase/0.98.0

到一个新目录

/usr/hdp/current/knox-server/data/services/decode/0.0.1

并将它们修改为如下所示的版本。我基本上只是使用匹配的大小写将所有出现的 hbase 替换为解码。

/usr/hdp/current/knox-server/data/services/decode/0.0.1/service.xml

<service role="DECODE" name="decode" version="0.0.1">
    <routes>
        <route path="/decode/?**">
            <rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
        </route>
        <route path="/decode/**?**">
            <rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
        </route>
        <route path="/decode/status/cluster?**">
            <rewrite apply="DECODE/decode/status/outbound" to="response.body"/>
        </route>
        <route path="/decode/*/regions?**">
            <rewrite apply="DECODE/decode/regions/outbound" to="response.body"/>
        </route>
    </routes>
    <dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/>
    <testURLs>
        <testURL>/decode/version</testURL>
        <testURL>/decode/version/cluster</testURL>
        <testURL>/decode/status/cluster</testURL>
        <testURL>/decode</testURL>
    </testURLs>
</service>

/usr/hdp/current/knox-server/data/services/decode/0.0.1/rewrite.xml

<rules>

    <rule dir="IN" name="DECODE/decode/root/inbound" pattern="*://*:*/**/decode/?{**}">
        <rewrite template="{$serviceUrl[DECODE]}/?{**}"/>
    </rule>

    <rule dir="IN" name="DECODE/decode/path/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
        <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
    </rule>

    <rule name="DECODE/decode/location/outbound">
        <match pattern="*://*:*/{path=**}?{**}"/>
        <rewrite template="{$frontend[url]}/decode/{path=**}?{**}"/>
    </rule>

    <rule name="DECODE/decode/address/outbound">
        <match pattern="{host}:{port}"/>
        <rewrite template="{$frontend[url]}/hbase-region?host={host}?port={port}"/>
        <encrypt-query/>
    </rule>

    <filter name="DECODE/decode/headers/outbound">
        <content type="application/x-http-headers">
            <apply path="Location" rule="DECODE/decode/location/outbound"/>
        </content>
    </filter>

    <filter name="DECODE/decode/status/outbound">
        <content type="*/json">
            <apply path="$[LiveNodes][*][name]" rule="DECODE/decode/address/outbound"/>
        </content>
        <content type="*/xml">
            <apply path="/ClusterStatus/LiveNodes/Node/@name" rule="DECODE/decode/address/outbound"/>
        </content>
    </filter>

    <filter name="DECODE/decode/regions/outbound">
        <content type="*/json">
            <apply path="$[Region][*][location]" rule="DECODE/decode/address/outbound"/>
        </content>
        <content type="*/xml">
            <apply path="/TableInfo/Region/@location" rule="DECODE/decode/address/outbound"/>
        </content>
    </filter>

</rules>

然后我可以使用下面的 curl 命令查询表。这在我相信回答您的问题的 URL 中不包含 hbase。

curl -k -u guest:******** -H 'Accept: application/json' https://localhost:8443/gateway/default/decode/t1/*
{"Row":[{"key":"ZjE6djE=","Cell":[{"column":"ZjI6djI=","timestamp":1453139991995,"$":"ZjM6djM="}]}]}