javascript - tensorflowjs 加载重新训练的 coco-ssd 模型 - 在浏览器中不起作用

Question

我使用 Python 模型/研究/对象检测 API 用我自己的数据集重新训练 coco-ssd。我已经保存了模型并且模型在 ipython notebook 中工作。我用 tfjs_converter 来转换它tensorflowjs_converter --input_format=tf_saved_model --output_format=tensorflowjs --output_node_names='detection_boxes,detection_classes,detection_scores,num_detections' --saved_model_tags=serve ./saved_model ./web_model

测试 1；我的代码

    image.src = imageURL;
    var img;
    const runButton = document.getElementById('run');
    runButton.onclick = async () => {
        console.log('model start');
        const model = await modelPromise;

        console.log('model loaded');
        const zeros = tf.zeros([1, 224, 224, 3]);

        const batched = tf.tidy(() => {
        if (!(image instanceof tf.Tensor)) {
            img = tf.fromPixels(image);
         }
      // Reshape to a single-element batch so we can pass it to executeAsync.
          return img.expandDims(0);
       });
       console.log('model loaded - now predict .. start');
       const result = await model.executeAsync(batched) ;
       console.log('model loaded - now predict - ready'); //  Error seen 
       batched.dispose();
       tf.dispose(result);

    model loaded - now predict .. start ( i tried chaning the model to Coco-ssd model same error)
   tensor_array.ts:116 Uncaught (in promise) Error: TensorArray : Could not write to TensorArray index 0,
          because the value dtype is int32, but TensorArray dtype is float32.
      at e.write (tensor_array.ts:116)
      at tensor_array.ts:162
      at Array.forEach (<anonymous>)
      at e.writeMany (tensor_array.ts:162)
      at e.scatter (tensor_array.ts:252)
      at control_executor.ts:127
      at callbacks.ts:17
      at Object.next (callbacks.ts:17)
      at callbacks.ts:17```

     Test 2; ---- using tfjs-model/coco-ssd/demo ----------------------------------
      did yarn , yarn watch  

    I replaced the coo-ssd model which works correctly, with my re-trained model  (only switched the models)
      //BASE_PATH = "https://storage.googleapis.com/tfjs-models/savedmodel/";
        BASE_PATH = "http://localhost:1234/web_model/";
    //this.modelPath = "" + BASE_PATH + this.getPrefix(e) + 
    "/tensorflowjs_model.pb", this.weightPath = "" + BASE_PATH + 
    this.getPrefix(e) + "/weights_manifest.json";
    ``this.modelPath = "" + BASE_PATH +  "tensorflowjs_model.pb", 
      this.weightPath = "" +BASE_PATH + "weights_manifest.json";``

    I get an error
    io_utils.ts:116 Uncaught (in promise) RangeError: byte length of float32Array should be a multiple of 4
        at new Float32Array (<anonymous>)
        at o (io_utils.ts:116)
        at Object.decodeWeights (io_utils.ts:79)
        at e.<anonymous> (frozen_model.ts:109)
        at exports_regularizers.ts:47
        at Object.next (exports_regularizers.ts:47)
        at s (exports_regularizers.ts:47)```


    model loaded - now predict .. start ( i tried chaning the model to Coco-ssd model same error)
    ```tensor_array.ts:116 Uncaught (in promise) Error: TensorArray : Could not write to TensorArray index 0,
          because the value dtype is int32, but TensorArray dtype is float32.
        at e.write (tensor_array.ts:116)
        at tensor_array.ts:162
        at Array.forEach (<anonymous>)
        at e.writeMany (tensor_array.ts:162)
        at e.scatter (tensor_array.ts:252)
        at control_executor.ts:127```
        at callbacks.ts:17
        at Object.next (callbacks.ts:17)
        at callbacks.ts:17

Answer 1

该错误与您用于预测的张量图像有关。

tf.fromPixel 创建张量图像，其值范围为 0 到 255，dtype 为 int。由于您的模型正在等待 dtype float32 的张量，您可以将类型转换为 float 或更改张量值以适应 0 和 1 之间

• 转换为 float32

img = tf.fromPixels(image).cast('float32')

• 移动值以适应 0 和 1 之间

img = tf.fromPixels(image).div(256)