어디까지 흘러가야 답을 찾을수 있을까 ㅠㅠ
아래 사이트에서 pb와 tflite를 모두 제공해주어서 한번 시도
[링크 : https://tfhub.dev/google/aiy/vision/classifier/birds_V1/1]
위 링크의 pb를 확인해보면 아래와 같이 나오는데, tag-set이 'serve' 라는게 없어서 변환하려니 안된다
$ saved_model_cli show --dir ./ --all
2021-02-02 16:42:12.474893: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:42:12.474941: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
MetaGraphDef with tag-set: '' contains the following SignatureDefs:
signature_def['default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['images'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 224, 224, 3)
name: hub_input/images:0
The given SavedModel SignatureDef contains the following output(s):
outputs['default'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 965)
name: prediction:0
Method name is:
signature_def['image_classifier']:
The given SavedModel SignatureDef contains the following input(s):
inputs['images'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 224, 224, 3)
name: hub_input/images:0
The given SavedModel SignatureDef contains the following output(s):
outputs['logits'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 965)
name: prediction:0
Method name is:
이건 다른 모델.. 도대체 어떻게 봐야 하는걸까 ㅠㅠ
$ saved_model_cli show --dir ./ --all
2021-02-02 16:42:44.443703: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:42:44.443752: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['input_tensor'] tensor_info:
dtype: DT_UINT8
shape: (1, -1, -1, 3)
name: serving_default_input_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_anchor_indices'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100)
name: StatefulPartitionedCall:0
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100, 4)
name: StatefulPartitionedCall:1
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100)
name: StatefulPartitionedCall:2
outputs['detection_multiclass_scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100, 91)
name: StatefulPartitionedCall:3
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100)
name: StatefulPartitionedCall:4
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (1)
name: StatefulPartitionedCall:5
outputs['raw_detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 1917, 4)
name: StatefulPartitionedCall:6
outputs['raw_detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 1917, 91)
name: StatefulPartitionedCall:7
Method name is: tensorflow/serving/predict
Defined Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
input_tensor: TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor')
혹시나 해서 다시 한번 select_tf_ops 옵션을 빼고 해보았지만 역시나 안된다.
눈에 들어오는 에러는 아래것 정도인데 -emit-select-tf-ops 옵션을 누구에게 주어야 하는건지 모르겠다.
그리고 custom op nor flex op. flex op는 또 무엇인가...
tensorflow.lite.python.convert.ConverterError: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
tf.Size {device = ""}
$ cat c.py
import tensorflow as tf
saved_model_dir="./"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
#converter.target_spec.supported_ops = [
# tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
# tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
#]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
$ python3 c.py
2021-02-02 16:45:08.234068: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:45:08.234112: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-02-02 16:45:10.382490: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:10.382710: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-02-02 16:45:10.382742: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-02-02 16:45:10.382775: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (mini2760p): /proc/driver/nvidia/version does not exist
2021-02-02 16:45:10.383265: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:26.331917: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored output_format.
2021-02-02 16:45:26.331970: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:319] Ignored drop_control_dependency.
2021-02-02 16:45:26.331981: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:325] Ignored change_concat_input_ranges.
2021-02-02 16:45:26.333113: I tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: ./
2021-02-02 16:45:26.442652: I tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2021-02-02 16:45:26.442721: I tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: ./
2021-02-02 16:45:26.442798: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-02-02 16:45:26.752919: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2021-02-02 16:45:26.824027: I tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2021-02-02 16:45:26.900734: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2494085000 Hz
2021-02-02 16:45:27.788741: I tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: ./
2021-02-02 16:45:28.227404: I tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 1894293 microseconds.
2021-02-02 16:45:34.080047: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:194] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2021-02-02 16:45:35.369335: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): error: 'tf.Size' op is neither a custom op nor a flex op
error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
tf.Size {device = ""}
Traceback (most recent call last):
File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 210, in toco_convert_protos
model_str = wrap_toco.wrapped_toco_convert(model_flags_str,
File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/wrap_toco.py", line 32, in wrapped_toco_convert
return _pywrap_toco_api.TocoConvert(
Exception: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
tf.Size {device = ""}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c.py", line 9, in <module>
tflite_model = converter.convert()
File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 739, in convert
result = _convert_saved_model(**converter_kwargs)
File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 632, in convert_saved_model
data = toco_convert_protos(
File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 216, in toco_convert_protos
raise ConverterError(str(e))
tensorflow.lite.python.convert.ConverterError: <unknown>:0: error: loc(callsite(callsite("Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField_1/Size@__inference___call___21591" at "StatefulPartitionedCall@__inference_signature_wrapper_23250") at "StatefulPartitionedCall")): 'tf.Size' op is neither a custom op nor a flex op
<unknown>:0: note: loc("StatefulPartitionedCall"): called from
<unknown>:0: error: failed while converting: 'main': Ops that can be supported by the flex runtime (enabled via setting the -emit-select-tf-ops flag):
tf.Size {device = ""}
saved_model_cli 명령어를 이용해서 변환하려는데 tensorrt가 안들어가면 인자가 부족하다고 하고
넣으면 libvinfer 에러가 나고... 후...
$ saved_model_cli convert --dir=. --output_dir=output --tag_set serving_default tensorrt
2021-02-02 16:52:12.317957: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-02-02 16:52:12.318003: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-02-02 16:52:13.640651: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2021-02-02 16:52:13.640699: F tensorflow/compiler/tf2tensorrt/stub/nvinfer_stub.cc:49] getInferLibVersion symbol not found.
중지됨 (core dumped)
[링크 : https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/saved_model_cli.py]
-emi-select-tf-ops=true가 flatbuffer_translate 에 전달되는 옵션인가?
| // RUN: flatbuffer_translate -mlir-to-tflite-flatbuffer %s -emit-select-tf-ops=true -emit-builtin-tflite-ops=false -o - | flatbuffer_to_string - | FileCheck %s |
[링크 : http://110.249.209.116/tiansongzhao/QT-Platform/-/blob/ffe4404132bbba3c690232c9f846ac160aa38e65/Software/resource/samples/tensorflow/tensorflow/compiler/mlir/lite/tests/mlir2flatbuffer/flex_exclusively.mlir]
MLIR (Multi-Level Intermediate Representation)
[링크 : https://mlir.llvm.org/]
결국은 돌아돌아 원점인가..
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.
tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.
]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
[링크 : https://www.tensorflow.org/lite/guide/ops_select#convert_a_model]
TFLite interpreter가 select ops 를 가지고 있는지 확인해라..
tensorflow.a 가 그럼 그 옵션을 받아야 한다는건가?
| Try using TF select ops. However, you may needs to ensure your TFLite interpreter has these select ops for inference. |
[링크 : https://stackoverflow.com/questions/65661737/]
