java - RGBIR 的探戈相机预览

Question

我正在使用 Tango 的 videoOverlaySample 演示。

我想查看红外数据(单独或带有颜色)而不是颜色。因此，我在它出现的两个地方用 TANGO_CAMERA_RGBIR 替换了 TANGO_CAMERA_COLOR。

但是屏幕是黑色的。

代码如下:

/*
 * Copyright 2014 Google Inc. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.projecttango.experiments.videooverlaysample;

import java.util.ArrayList;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.util.Log;
import android.widget.Toast;

import com.google.atap.tangoservice.Tango;
import com.google.atap.tangoservice.Tango.OnTangoUpdateListener;
import com.google.atap.tangoservice.TangoCameraIntrinsics;
import com.google.atap.tangoservice.TangoCameraPreview;
import com.google.atap.tangoservice.TangoConfig;
import com.google.atap.tangoservice.TangoCoordinateFramePair;
import com.google.atap.tangoservice.TangoEvent;
import com.google.atap.tangoservice.TangoPoseData;
import com.google.atap.tangoservice.TangoXyzIjData;

/**
 * An example showing the usage of TangoCameraPreview class
 * Usage of TangoCameraPreviewClass:
 * To use this class, we first need initialize the TangoCameraPreview class with the activity's 
 * context and connect to the camera we want by using connectToTangoCamera class.Once the connection 
 * is established we need to manually update the TangoCameraPreview's texture by using the
 * onFrameAvailable callbacks.
 * Note:
 * To use TangoCameraPreview class we need to ask the user permissions for MotionTracking 
 * at the minimum level. This is because in Java all the call backs such as 
 * onPoseAvailable,onXyzIjAvailable, onTangoEvents, onFrameAvailable are set together at once. 
 */
public class MainActivity extends Activity {
    private TangoCameraPreview tangoCameraPreview;
    private Tango mTango;
    private boolean mIsConnected;
    private boolean mIsPermissionGranted;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        tangoCameraPreview = new TangoCameraPreview(this);
        mTango = new Tango(this);
        startActivityForResult(
                Tango.getRequestPermissionIntent(Tango.PERMISSIONTYPE_MOTION_TRACKING),
                Tango.TANGO_INTENT_ACTIVITYCODE);
        setContentView(tangoCameraPreview);
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        // Check which request we're responding to
        if (requestCode == Tango.TANGO_INTENT_ACTIVITYCODE) {
            // Make sure the request was successful
            if (resultCode == RESULT_CANCELED) {
                Toast.makeText(this, "Motion Tracking Permissions Required!",
                        Toast.LENGTH_SHORT).show();
                finish();
            } else {
                startCameraPreview();
                mIsPermissionGranted = true;
            }
        }
    }

    // Camera Preview
    private void startCameraPreview() {
        // Connect to color camera
        tangoCameraPreview.connectToTangoCamera(mTango,
                TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);
        // Use default configuration for Tango Service.
        TangoConfig config = mTango.getConfig(TangoConfig.CONFIG_TYPE_DEFAULT);
        config.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true);
        mTango.connect(config);
        mIsConnected = true;

        // No need to add any coordinate frame pairs since we are not using 
        // pose data. So just initialize.
        ArrayList<TangoCoordinateFramePair> framePairs = new ArrayList<TangoCoordinateFramePair>();
        mTango.connectListener(framePairs, new OnTangoUpdateListener() {
            @Override
            public void onPoseAvailable(TangoPoseData pose) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onFrameAvailable(int cameraId) {

                // Check if the frame available is for the camera we want and
                // update its frame on the camera preview.
                if (cameraId == TangoCameraIntrinsics.TANGO_CAMERA_RGBIR) {
                    tangoCameraPreview.onFrameAvailable();
                }
            }

            @Override
            public void onXyzIjAvailable(TangoXyzIjData xyzIj) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onTangoEvent(TangoEvent event) {
                // We are not using OnPoseAvailable for this app
            }
        });
    }

    @Override
    protected void onPause() {
        super.onPause();
        if(mIsConnected) {
            mTango.disconnect();
            tangoCameraPreview.disconnectFromTangoCamera();
            mIsConnected = false;
        }
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (!mIsConnected && mIsPermissionGranted) {
            startCameraPreview();
        }
    }
}

Answer 1

你不能使用

tangoCameraPreview.connectToTangoCamera(mTango,TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);

Java API 不提供带 RGBIR 的 connectToTangoCamera，仅提供彩色和鱼眼相机(参见 here)

要显示深度图像而不是彩色图像，您必须手动计算深度图像。因此，您大致必须:

1. 将具有给定时间戳的姿势的点云转换为相机帧。
2. 投影点云点(x,y,z)接收像素(x,y)
3. 您需要一个具有相机帧分辨率 (1240x720) 的阵列。
4. 用 0(黑色)填充数组。
5. 将 z 值存储在给定的像素位置(对像素进行上采样，因为深度相机的分辨率仅为 320x180)
6. 在 OpenGL 中，您可以轻松地将数组用于纹理。

有关更多详细信息，我建议您查看 C 示例 rgb-depth-sync