i.mx8의 selfie_segmentrer.py 에서 발췌
| # Set backend and delegates if self.backend == "CPU": if self.platform == "i.MX8MP": backend = "true:CPU custom=NumThreads:4" else: backend = "true:CPU custom=NumThreads:2" else: if self.platform == "i.MX8MP": os.environ["USE_GPU_INFERENCE"] = "0" backend = ( "true:npu custom=Delegate:External,ExtDelegateLib:libvx_delegate.so" ) else: backend = "true:npu custom=Delegate:External,ExtDelegateLib:libethosu_delegate.so" |
| + " ! videoconvert ! video/x-raw,format=RGB ! tensor_converter ! " + "tensor_transform mode=arithmetic option=typecast:float32,div:255.0 ! " + "tensor_filter framework=tensorflow-lite model=" + self.tflite_model + " accelerator=" + backend + " name=tensor_filter latency=1 ! tensor_sink name=tensor_sink " |
gsteramer의 엘리먼트를 기본으로 NN에 맞춰서 몇개 추가한 것 같은데
[링크 : https://nnstreamer.github.io/gst/nnstreamer/tensor_converter/README.html]
[링크 : https://nnstreamer.github.io/gst/nnstreamer/tensor_transform/README.html]
[링크 : https://nnstreamer.github.io/gst/nnstreamer/tensor_decoder/README.html]
[링크 : https://nnstreamer.github.io/gst/nnstreamer/tensor_filter/README.html]
[링크 : https://nnstreamer.github.io/gst/nnstreamer/tensor_sink/README.html]
[링크 : https://nnstreamer.github.io/gst/nnstreamer/elements/gsttensor_sink.html]
사용예에서 보면 converter / transform / filter / sink 순으로 쓰는 듯
| [CAM] - [videoconvert] - [videoscale] - [tee] -+- [queue] - [videoconvert] - [cairooverlay] - [ximagesink] +- [queue] - [videoscale] - [tensor_converter] - [tensor_transform] - [tensor_filter] - [tensor_sink] |
[링크 : https://nnstreamer.github.io/how-to-run-examples.html]
+
gst-inspector tensor_converter
더보기
| root@imx8mpevk:~# gst-inspect-1.0 tensor_converter Factory Details: Rank none (0) Long-name TensorConverter Klass Converter/Tensor Description Converts an audio, video, text, or arbitrary stream to a tensor stream of C-Array for neural network framework filters Author MyungJoo Ham <myungjoo.ham@samsung.com></myungjoo.ham@samsung.com> Plugin Details: Name nnstreamer Description nnstreamer plugin library Filename /usr/lib/gstreamer-1.0/libnnstreamer.so Version 2.4.0 License LGPL Source module nnstreamer Binary package nnstreamer Origin URL https://github.com/nnstreamer/nnstreamer GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstTensorConverter Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)RGB, (string)BGR, (string)RGBx, (string)BGRx, (string)xRGB, (string)xBGR, (string)RGBA, (string)BGRA, (string)ARGB, (string)ABGR, (string)GRAY8, (string)GRAY16_BE, (string)GRAY16_LE } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] interlace-mode: progressive audio/x-raw format: { (string)S8, (string)U8, (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)F32LE, (string)F32BE, (string)F64LE, (string)F64BE } rate: [ 1, 2147483647 ] channels: [ 1, 2147483647 ] layout: interleaved text/x-raw format: utf8 application/octet-stream other/tensors format: flexible framerate: [ 0/1, 2147483647/1 ] application/octet-stream other/protobuf-tensor framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible } framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: frames-per-tensor : The number of frames in output tensor flags: readable, writable Unsigned Integer. Range: 1 - 4294967295 Default: 1 input-dim : Input tensor dimension from inner array flags: readable, writable String. Default: "" input-type : Type of each element of the input tensor flags: readable, writable String. Default: "" mode : Converter mode. e.g., mode=custom-code:. For detail, refer to </registered callback name>https://github.com/nnstreamer/nnstreamer/blob/main/gst/nnstreamer/elements/gsttensor_converter.md#custom-converter flags: readable, writable String. Default: "" name : The name of the object flags: readable, writable String. Default: "tensorconverter0" parent : The parent of the object flags: readable, writable Object of type "GstObject" set-timestamp : The flag to set timestamp when received a buffer with invalid timestamp flags: readable, writable Boolean. Default: true silent : Produce verbose output flags: readable, writable Boolean. Default: true sub-plugins : Registrable sub-plugins list flags: readable String. Default: "python3,protobuf" |
gst-inspector tensor_transform
더보기
| root@imx8mpevk:~# gst-inspect-1.0 tensor_transform Factory Details: Rank none (0) Long-name TensorTransform Klass Filter/Tensor Description Transforms other/tensor dimensions for different models or frameworks Author MyungJoo Ham <myungjoo.ham@samsung.com> Plugin Details: Name nnstreamer Description nnstreamer plugin library Filename /usr/lib/gstreamer-1.0/libnnstreamer.so Version 2.4.0 License LGPL Source module nnstreamer Binary package nnstreamer Origin URL https://github.com/nnstreamer/nnstreamer GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseTransform +----GstTensorTransform Pad Templates: SINK template: 'sink' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible } framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible } framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: acceleration : Orc acceleration flags: readable, writable Boolean. Default: true apply : Select tensors to apply, separated with ',' in case of multiple tensors. Default to apply all tensors. flags: readable, writable String. Default: "" mode : Mode used for transforming tensor flags: readable, writable Enum "gtt_mode_type" Default: -1, "unknown" (0): dimchg - Mode for changing tensor dimensions, option=FROM_DIM:TO_DIM (with a regex, ^([0-9]|1[0-5]):([0-9]|1[0-5])$, where NNS_TENSOR_RANK_LIMIT is 16) (1): typecast - Mode for casting type of tensor, option=(^[u]?int(8|16|32|64)$|^float(16|32|64)$) (2): arithmetic - Mode for arithmetic operations with tensor, option=[typecast:TYPE,][per-channel:(false|true@DIM),]add|mul|div:NUMBER[@CH_IDX], ... (3): transpose - Mode for transposing shape of tensor, option=D1':D2':D3':D4 (fixed to 3) (4): stand - Mode for statistical standardization of tensor, option=(default|dc-average)[:TYPE][,per-channel:(false|true)] (5): clamp - Mode for clamping all elements of tensor into the range, option=CLAMP_MIN:CLAMP_MAX (-1): unknown - Unknown or not-implemented-yet mode name : The name of the object flags: readable, writable String. Default: "tensortransform0" option : Option for the tensor transform mode ? flags: readable, writable String. Default: null parent : The parent of the object flags: readable, writable Object of type "GstObject" qos : Handle Quality-of-Service events flags: readable, writable Boolean. Default: false silent : Produce verbose output ? flags: readable, writable Boolean. Default: true transpose-rank-limit: The rank limit of transpose, which varies per version of nnstreamer and may be lower than the global rank limit if it is over 4. flags: readable Unsigned Integer. Range: 0 - 16 Default: 4 |
gst-inspector tensor_decoder
더보기
| root@imx8mpevk:~# gst-inspect-1.0 tensor_decoder Factory Details: Rank none (0) Long-name TensorDecoder Klass Converter/Tensor Description Converts tensor stream of C-Array for neural network framework filters to audio or video stream Author Jijoong Moon <jijoong.moon@samsung.com> Plugin Details: Name nnstreamer Description nnstreamer plugin library Filename /usr/lib/gstreamer-1.0/libnnstreamer.so Version 2.4.0 License LGPL Source module nnstreamer Binary package nnstreamer Origin URL https://github.com/nnstreamer/nnstreamer GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseTransform +----GstTensorDecoder Pad Templates: SINK template: 'sink' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: static num_tensors: [ 1, 16 ] framerate: [ 0/1, 2147483647/1 ] other/tensors format: flexible framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: ANY Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: config-file : sets config file path which contains plugins properties flags: ** (gst-inspect-1.0:1706): WARNING **: 05:54:08.872: /usr/src/debug/nnstreamer/2.4.0/gst/nnstreamer/elements/gsttensor_decoder.c:592: invalid property id 13 for "config-file" of type 'GParamString' in 'GstTensorDecoder' readable, writable String. Default: null mode : Decoder mode flags: readable, writable String. Default: "" name : The name of the object flags: readable, writable String. Default: "tensordecoder0" option1 : option for specific decoder modes, 1st one. flags: readable, writable String. Default: null option2 : option for specific decoder modes, 2nd one. flags: readable, writable String. Default: null option3 : option for specific decoder modes, 3rd one. flags: readable, writable String. Default: null option4 : option for specific decoder modes, 4th one. flags: readable, writable String. Default: null option5 : option for specific decoder modes, 5th one. flags: readable, writable String. Default: null option6 : option for specific decoder modes, 6th one. flags: readable, writable String. Default: null option7 : option for specific decoder modes, 7th one. flags: readable, writable String. Default: null option8 : option for specific decoder modes, 8th one. flags: readable, writable String. Default: null option9 : option for specific decoder modes, 9th one. flags: readable, writable String. Default: null parent : The parent of the object flags: readable, writable Object of type "GstObject" qos : Handle Quality-of-Service events flags: readable, writable Boolean. Default: false silent : Produce verbose output flags: readable, writable Boolean. Default: true sub-plugins : Registrable sub-plugins list flags: readable String. Default: "protobuf,direct_video,bounding_boxes,image_segment,python3,octet_stream,pose_estimation,tensor_region,image_labeling" |
gst-inspector tensor_filter
더보기
| root@imx8mpevk:~# gst-inspect-1.0 tensor_filter Factory Details: Rank none (0) Long-name TensorFilter Klass Filter/Tensor Description Handles NN Frameworks (e.g., tensorflow) as Media Filters with other/tensor type stream Author MyungJoo Ham <myungjoo.ham@samsung.com> Plugin Details: Name nnstreamer Description nnstreamer plugin library Filename /usr/lib/gstreamer-1.0/libnnstreamer.so Version 2.4.0 License LGPL Source module nnstreamer Binary package nnstreamer Origin URL https://github.com/nnstreamer/nnstreamer GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseTransform +----GstTensorFilter Pad Templates: SINK template: 'sink' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible } framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible } framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' SRC: 'src' Pad Template: 'src' Element Properties: accelerator : Set accelerator for the subplugin with format (true/false):(comma separated ACCELERATOR(s)). true/false determines if accelerator is to be used. list of accelerators determines the backend (ignored with false). Example, if GPU, NPU can be used but not CPU - true:npu,gpu,!cpu. The full list of accelerators can be found in nnstreamer_plugin_api_filter.h. Note that only a few subplugins support this property. flags: readable, writable String. Default: "" custom : Custom properties for subplugins ? flags: readable, writable String. Default: "" framework : Neural network framework flags: readable, writable String. Default: "auto" input : Input tensor dimension from inner array, up to 4 dimensions ? flags: readable, writable String. Default: "" input-combination : Select the input tensor(s) to invoke the models flags: readable, writable String. Default: "" inputlayout : Set channel first (NCHW) or channel last layout (NHWC) or None for input data. Layout of the data can be any or NHWC or NCHW or none for now. flags: readable, writable String. Default: "" inputname : The Name of Input Tensor flags: readable, writable String. Default: "" inputranks : The Rank of the Input Tensor, which is separated with ',' in case of multiple Tensors flags: readable String. Default: "" inputtype : Type of each element of the input tensor ? flags: readable, writable String. Default: "" invoke-dynamic : Flexible tensors whose memory size changes can be used asinput and output of the tensor filter. With this option, the output caps is always in the format of flexible tensors. flags: readable, writable Boolean. Default: false is-updatable : Indicate whether a given model to this tensor filter is updatable in runtime. (e.g., with on-device training) flags: readable, writable Boolean. Default: false latency : Turn on performance profiling for the average latency over the recent 10 inferences in microseconds. Currently, this accepts either 0 (OFF) or 1 (ON). flags: readable, writable Integer. Range: 0 - 1 Default: -1 latency-report : Report to the pipeline the estimated tensor-filter element latency. flags: readable, writable Boolean. Default: false model : File path to the model file. Separated with ',' in case of multiple model files(like caffe2) flags: readable, writable String. Default: "" name : The name of the object flags: readable, writable String. Default: "tensorfilter0" output : Output tensor dimension from inner array, up to 4 dimensions ? flags: readable, writable String. Default: "" output-combination : Select the output tensor(s) from the input tensor(s) and/or model output flags: readable, writable String. Default: "" outputlayout : Set channel first (NCHW) or channel last layout (NHWC) or None for output data. Layout of the data can be any or NHWC or NCHW or none for now. flags: readable, writable String. Default: "" outputname : The Name of Output Tensor flags: readable, writable String. Default: "" outputranks : The Rank of the Out Tensor, which is separated with ',' in case of multiple Tensors flags: readable String. Default: "" outputtype : Type of each element of the output tensor ? flags: readable, writable String. Default: "" parent : The parent of the object flags: readable, writable Object of type "GstObject" qos : Handle Quality-of-Service events flags: readable, writable Boolean. Default: false shared-tensor-filter-key: Multiple element instances of tensor-filter in a pipeline may share a single resource instance if they share the same framework (subplugin) and neural network model. Designate "shared-tensor-filter-key" to declare and share such instances. If it is NULL, it means the model representations is not shared. flags: readable, writable String. Default: "" silent : Produce verbose output flags: readable, writable Boolean. Default: true sub-plugins : Registrable sub-plugins list flags: readable String. Default: "custom,custom-easy,cpp,python3,tvm,tensorflow2-lite" throughput : Turn on performance profiling for the average throughput in the number of outputs per seconds (i.e., FPS), multiplied by 1000 to represent a floating point using an integer. Currently, this accepts either 0 (OFF) or 1 (ON). flags: readable, writable Integer. Range: 0 - 1 Default: -1 |
gst-inspector tensor_sink
더보기
| root@imx8mpevk:~# gst-inspect-1.0 tensor_sink Factory Details: Rank none (0) Long-name TensorSink Klass Sink/Tensor Description Sink element to handle tensor stream Author Samsung Electronics Co., Ltd. Plugin Details: Name nnstreamer Description nnstreamer plugin library Filename /usr/lib/gstreamer-1.0/libnnstreamer.so Version 2.4.0 License LGPL Source module nnstreamer Binary package nnstreamer Origin URL https://github.com/nnstreamer/nnstreamer GObject +----GInitiallyUnowned +----GstObject +----GstElement +----GstBaseSink +----GstTensorSink Pad Templates: SINK template: 'sink' Availability: Always Capabilities: other/tensor framerate: [ 0/1, 2147483647/1 ] other/tensors format: { (string)static, (string)flexible, (string)sparse } framerate: [ 0/1, 2147483647/1 ] Element has no clocking capabilities. Element has no URI handling capabilities. Pads: SINK: 'sink' Pad Template: 'sink' Element Properties: async : Go asynchronously to PAUSED flags: readable, writable Boolean. Default: true blocksize : Size in bytes to pull per buffer (0 = default) flags: readable, writable Unsigned Integer. Range: 0 - 4294967295 Default: 4096 emit-signal : Emit signal for new data, stream start, eos flags: readable, writable Boolean. Default: true enable-last-sample : Enable the last-sample property flags: readable, writable Boolean. Default: true last-sample : The last sample received in the sink flags: readable Boxed pointer of type "GstSample" max-bitrate : The maximum bits per second to render (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 max-lateness : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited) flags: readable, writable Integer64. Range: -1 - 9223372036854775807 Default: -1 name : The name of the object flags: readable, writable String. Default: "tensorsink0" parent : The parent of the object flags: readable, writable Object of type "GstObject" processing-deadline : Maximum processing time for a buffer in nanoseconds flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 20000000 qos : Generate Quality-of-Service events upstream flags: readable, writable Boolean. Default: true render-delay : Additional render delay of the sink in nanoseconds flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 signal-rate : New data signals per second (0 for unlimited, max 500) flags: readable, writable Unsigned Integer. Range: 0 - 500 Default: 0 silent : Produce verbose output flags: readable, writable Boolean. Default: true stats : Sink Statistics flags: readable Boxed pointer of type "GstStructure" average-rate: 0 dropped: 0 rendered: 0 sync : Sync on the clock flags: readable, writable Boolean. Default: false throttle-time : The time to keep between rendered buffers (0 = disabled) flags: readable, writable Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 ts-offset : Timestamp offset in nanoseconds flags: readable, writable Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 Element Signals: "eos" : void user_function (GstElement * object, gpointer user_data); "stream-start" : void user_function (GstElement * object, gpointer user_data); "new-data" : void user_function (GstElement * object, GstBuffer * arg0, gpointer user_data); |
'embeded > i.mx 8m plus' 카테고리의 다른 글
| openVX, verisilicon(vivante) (0) | 2025.08.14 |
|---|---|
| gstreamer pipeline / appsink (0) | 2025.08.14 |
| nxp eiq 우분투 실행 실패 (0) | 2025.07.31 |
| nxp g2d_blit (0) | 2025.04.01 |
| sounddevice on arm i.mx8 evk (0) | 2024.05.14 |
