구차니의 잡동사니 모음

텐서플로우 object detection 관련 시작 지점으로 보면 되려나?

[링크 : http://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2.md]

작동 여부는 확인 못함

다만.. debugutilsbad 아래꺼라 정상작동은 보증 못하려나?

[링크 : http://gstreamer.freedesktop.org/documentation/debugutilsbad/fpsdisplaysink.html]

텐서플로우 모델에서 메타데이터를 보다보니 에러가 발생하는 녀석도 있어서

찾아는 보는데 도대체 먼지 알 수가 없네..

[링크 : http://https://www.tensorflow.org/tfx/serving/serving_basic]

[링크 : https://stackoverflow.com/questions/58927662/]

+ 2021.02.16

오는길에 다시 보니 tensorflow model garden / research / object detection 에서 구현된

내용들이지 엄밀하게는 tensorflow 자체의 구현은 아니다.

tensorflow를 가지고 구현한 내용이라고 해야하려나?

-

model ssd

type ssd_mobilenet_v2_keras 를

ssd_mobilenet_v2 로 바꾸었더니 아래와 같은 에러가 발생했다.

INFO:tensorflow:Maybe overwriting train_steps: 1
I0212 20:50:37.009305 140651210348352 config_util.py:552] Maybe overwriting train_steps: 1
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0212 20:50:37.009468 140651210348352 config_util.py:552] Maybe overwriting use_bfloat16: False
Traceback (most recent call last):
  File "model_main_tf2.py", line 113, in <module>
    tf.compat.v1.app.run()
  File "/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/minimonk/.local/lib/python3.8/site-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/home/minimonk/.local/lib/python3.8/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "model_main_tf2.py", line 104, in main
    model_lib_v2.train_loop(
  File "/home/minimonk/src/SSD-MobileNet-TF/object_detection/model_lib_v2.py", line 507, in train_loop
    detection_model = MODEL_BUILD_UTIL_MAP['detection_model_fn_base'](
  File "/home/minimonk/src/SSD-MobileNet-TF/object_detection/builders/model_builder.py", line 1106, in build
    return build_func(getattr(model_config, meta_architecture), is_training,
  File "/home/minimonk/src/SSD-MobileNet-TF/object_detection/builders/model_builder.py", line 377, in _build_ssd_model
    _check_feature_extractor_exists(ssd_config.feature_extractor.type)
  File "/home/minimonk/src/SSD-MobileNet-TF/object_detection/builders/model_builder.py", line 249, in _check_feature_extractor_exists
    raise ValueError('{} is not supported. See `model_builder.py` for features '
ValueError: ssd_mobilenet_v2 is not supported. See `model_builder.py` for features extractors compatible with different versions of Tensorflow

model_builder.py를 열어보라는데 여러개 파일이 나타난다.?

$ sudo find / -name model_builder.py
/home/minimonk/src/SSD-MobileNet-TF/object_detection/builders/model_builder.py
/home/minimonk/src/SSD-MobileNet-TF/models/research/object_detection/builders/model_builder.py
/home/minimonk/src/SSD-MobileNet-TF/models/research/lstm_object_detection/model_builder.py
/home/minimonk/src/SSD-MobileNet-TF/build/lib/object_detection/builders/model_builder.py

lstm 어쩌구를 제외하면 용량이 동일하니 같은 파일로 간주하고 하나를 열어보니 다음과 같이 나오는데..

if tf_version.is_tf2() 에 의해서 사용가능한 녀석은.. 

ssd_mobilenet_v2_fpn_keras 와

ssd_mobilenet_v2_keras 뿐이다 -_-

기대했던 ssd_mobilenet_v2는  tf1 ㅠㅠ

$ vi /home/minimonk/src/SSD-MobileNet-TF/object_detection/builders/model_builder.py
if tf_version.is_tf2():
  from object_detection.models import center_net_hourglass_feature_extractor
  from object_detection.models import center_net_mobilenet_v2_feature_extractor
  from object_detection.models import center_net_mobilenet_v2_fpn_feature_extractor
  from object_detection.models import center_net_resnet_feature_extractor
  from object_detection.models import center_net_resnet_v1_fpn_feature_extractor
  from object_detection.models import faster_rcnn_inception_resnet_v2_keras_feature_extractor as frcnn_inc_res_keras
  from object_detection.models import faster_rcnn_resnet_keras_feature_extractor as frcnn_resnet_keras
  from object_detection.models import ssd_resnet_v1_fpn_keras_feature_extractor as ssd_resnet_v1_fpn_keras
  from object_detection.models import faster_rcnn_resnet_v1_fpn_keras_feature_extractor as frcnn_resnet_fpn_keras
  from object_detection.models.ssd_mobilenet_v1_fpn_keras_feature_extractor import SSDMobileNetV1FpnKerasFeatureExtractor
  from object_detection.models.ssd_mobilenet_v1_keras_feature_extractor import SSDMobileNetV1KerasFeatureExtractor
  from object_detection.models.ssd_mobilenet_v2_fpn_keras_feature_extractor import SSDMobileNetV2FpnKerasFeatureExtractor
  from object_detection.models.ssd_mobilenet_v2_keras_feature_extractor import SSDMobileNetV2KerasFeatureExtractor
  from object_detection.predictors import rfcn_keras_box_predictor
  if sys.version_info[0] >= 3:
    from object_detection.models import ssd_efficientnet_bifpn_feature_extractor as ssd_efficientnet_bifpn

if tf_version.is_tf1():
  from object_detection.models import faster_rcnn_inception_resnet_v2_feature_extractor as frcnn_inc_res
  from object_detection.models import faster_rcnn_inception_v2_feature_extractor as frcnn_inc_v2
  from object_detection.models import faster_rcnn_nas_feature_extractor as frcnn_nas
  from object_detection.models import faster_rcnn_pnas_feature_extractor as frcnn_pnas
  from object_detection.models import faster_rcnn_resnet_v1_feature_extractor as frcnn_resnet_v1
  from object_detection.models import ssd_resnet_v1_fpn_feature_extractor as ssd_resnet_v1_fpn
  from object_detection.models import ssd_resnet_v1_ppn_feature_extractor as ssd_resnet_v1_ppn
  from object_detection.models.embedded_ssd_mobilenet_v1_feature_extractor import EmbeddedSSDMobileNetV1FeatureExtractor
  from object_detection.models.ssd_inception_v2_feature_extractor import SSDInceptionV2FeatureExtractor
  from object_detection.models.ssd_mobilenet_v2_fpn_feature_extractor import SSDMobileNetV2FpnFeatureExtractor
  from object_detection.models.ssd_mobilenet_v2_mnasfpn_feature_extractor import SSDMobileNetV2MnasFPNFeatureExtractor
  from object_detection.models.ssd_inception_v3_feature_extractor import SSDInceptionV3FeatureExtractor
  from object_detection.models.ssd_mobilenet_edgetpu_feature_extractor import SSDMobileNetEdgeTPUFeatureExtractor
  from object_detection.models.ssd_mobilenet_v1_feature_extractor import SSDMobileNetV1FeatureExtractor
  from object_detection.models.ssd_mobilenet_v1_fpn_feature_extractor import SSDMobileNetV1FpnFeatureExtractor
  from object_detection.models.ssd_mobilenet_v1_ppn_feature_extractor import SSDMobileNetV1PpnFeatureExtractor
  from object_detection.models.ssd_mobilenet_v2_feature_extractor import SSDMobileNetV2FeatureExtractor
  from object_detection.models.ssd_mobilenet_v3_feature_extractor import SSDMobileNetV3LargeFeatureExtractor
  from object_detection.models.ssd_mobilenet_v3_feature_extractor import SSDMobileNetV3SmallFeatureExtractor
  from object_detection.models.ssd_mobiledet_feature_extractor import SSDMobileDetCPUFeatureExtractor
  from object_detection.models.ssd_mobiledet_feature_extractor import SSDMobileDetDSPFeatureExtractor
  from object_detection.models.ssd_mobiledet_feature_extractor import SSDMobileDetEdgeTPUFeatureExtractor
  from object_detection.models.ssd_mobiledet_feature_extractor import SSDMobileDetGPUFeatureExtractor
  from object_detection.models.ssd_pnasnet_feature_extractor import SSDPNASNetFeatureExtractor
  from object_detection.predictors import rfcn_box_predictor

[링크 : https://stackoverflow.com/questions/65938445/]

+

와.. ssd_mobilenet_v2_fpn_keras를 돌리는데 메모리 부족으로 죽어버리네 ㄷㄷ

눈에 보이는건.. additional_layer_depth 인가.. 이걸 줄이고 해봐야 겠네..

    feature_extractor {
      type: 'ssd_mobilenet_v2_fpn_keras'
      use_depthwise: true
      fpn {
        min_level: 3
        max_level: 7
        additional_layer_depth: 128
      }
      min_depth: 16
      depth_multiplier: 1.0
      conv_hyperparams {
        activation: RELU_6,
        regularizer {
          l2_regularizer {
            weight: 0.00004
          }
        }
        initializer {
          random_normal_initializer {
            stddev: 0.01
            mean: 0.0
          }
        }
        batch_norm {
          scale: true,
          decay: 0.997,
          epsilon: 0.001,
        }
      }
      override_base_feature_extractor_hyperparams: true
    }

/home/minimonk/.local/lib/python3.8/site-packages/tensorflow/python/keras/backend.py:434: UserWarning: `tf.keras.backend.set_learning_phase` is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the `training` argument of the `__call__` method of your layer or model.
  warnings.warn('`tf.keras.backend.set_learning_phase` is deprecated and '
2021-02-12 21:23:47.163320: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1258291200 exceeds 10% of free system memory.
죽었음

+

depth를 줄이고 해보니 되는척 하다가 또 에러가 발생 ㅋㅋ

    ValueError: Number of feature maps is expected to equal the length of `num_anchors_per_location`.

되는 척 하더니 안되네? ㅠㅠ

AssertionError: Some Python objects were not bound to checkpointed values, likely due to changes in the Python program: [MirroredVariable:{
  0: <tf.Variable 'block_8_depthwise/depthwise_kernel:0' shape=(3, 3, 384, 1) dtype=float32, numpy=

...

WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
W0212 21:37:27.939997 140162079770432 util.py:168] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.

모델 생성해서 보니 피라미드라고 불릴 만큼 크고 아름답다(?)

원본은 변환하다 문제가 생긴건지 잘 올려져서 그냥 크롬에서 줄여서 올리는데 티가 안나네

이걸 모바일 디바이스에서 돌릴순 있는게 맞나... ㄷㄷ

It stands for Feature Pyramid Network. Its a subnetwork which outputs feature maps of different resolutions. An explanation of FPN using detectron2 as an example is here: https://medium.com/@hirotoschwert/digging-into-detectron-2-part-2-dd6e8b0526e

[링크 : https://stackoverflow.com/questions/63653903]

pipeline.config 파일의 설정에 대한 페이지는 없나?

있는거 뜯어보니 상위 엘리먼트(?)는 아래와 같이 5개로 나눠진다.

이름만 보면 직관적이긴 한데 model / training / evaluation 세가지 그리고 training, evaluation에 대한 읽기 설정인 듯.

model {}

train_config {}

train_input_reader {}

eval_config {}

eval_input_reader {}

google model garden 에서 받아서 파일에서 분석을 해보니 아래와 같은 종류가 나온다.

아래는 research / object_detection 아래만 검색한 내용

60/2 vs 30/1 어느게 맞는 것인가..

[링크 : https://gstreamer.freedesktop.org/documentation/videorate/index.html?gi-language=c]

[링크 : https://unix.stackexchange.com/questions/342815/how-to-send-ffmpeg-output-to-framebuffer]

[링크 : https://ffmpeg.org/ffmpeg-devices.html]

[링크 : https://github.com/tensorflow/models/blob/master/research/object_detection/configs/tf2/ssd_mobilenet_v2_320x320_coco17_tpu-8.config]

[링크 : https://blog.naver.com/bdh0727/221537759295]

+

num_train_steps=1로 하니 cpu 로만 학습해도 1회 뿐이라 금세 끝난다.

$ python3 model_main_tf2.py --pipeline_config_path=ssd_mobilenet_v2_320x320_coco17_tpu-8/pipeline.config --model_dir=trained-checkpoint --alsologtostderr --num_train_steps=1 --sample_1_of_n_eval_examples=1 --num_eval_steps=1
$ find ./ -type f -mmin -10
/trained-checkpoint/train/events.out.tfevents.1612780803.mini2760p.5335.2928.v2

$ python3 exporter_main_v2.py --input_type image_tensor --pipeline_config_path ./ssd_mobilenet_v2_320x320_coco17_tpu-8/pipeline.config --trained_checkpoint_dir ./trained-checkpoint --output_directory exported-model/mobile-model
$ find ./ -type f -mmin -10
./exported-model/mobile-model/saved_model/variables/variables.index
./exported-model/mobile-model/saved_model/variables/variables.data-00000-of-00001
./exported-model/mobile-model/saved_model/saved_model.pb
./exported-model/mobile-model/checkpoint/checkpoint
./exported-model/mobile-model/checkpoint/ckpt-0.data-00000-of-00001
./exported-model/mobile-model/checkpoint/ckpt-0.index
./exported-model/mobile-model/pipeline.config

[링크 : https://github.com/abhimanyu1990/SSD-Mobilenet-Custom-Object-Detector-Model-using-Tensorflow-2] <<

[링크 : https://stackoverflow.com/questions/64510791/tf2-object-detection-api-model-main-tf2-py-validation-loss]

[링크 : https://ichi.pro/ko/tensorflow-gaegche-gamji-gaideu-tensorflow-2-252181752953859]

+

[링크 : https://neptune.ai/blog/how-to-train-your-own-object-detector-using-tensorflow-object-detection-api]

+

[링크 : https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/auto_examples/plot_object_detection_saved_model.html]

[링크 : https://github.com/tensorflow/models/tree/master/research/object_detection/configs/tf2]

[링크 : https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md]

요런식으로 해두는걸 tee 라고 하는듯

gst-launch-1.0 filesrc location=song.ogg ! decodebin ! tee name=t ! queue ! audioconvert ! audioresample ! autoaudiosink t. ! queue ! audioconvert ! goom ! videoconvert ! autovideosink

[링크 : https://gstreamer.freedesktop.org/documentation/coreelements/tee.html]