- 
+ 
run_detector demo result
## C++ Application Demos
-### Object Classification
+### Classification
-Source code location: `./demos/classification/classification_sync.cpp`
+Source file: `./demos/classification/classification_sync.cpp`
+
run_detector demo result
imagenet_classification demo result
@@ -260,7 +199,7 @@ Profiler data has been written to profiler.json
### YOLOv5 Object Detection
-Source code location: `./demos/object_detection/yolo_1channel.cpp`
+Source file: `./demos/object_detection/yolo_1channel.cpp`
@@ -320,25 +246,25 @@ save file : result.jpg
yolov5s demo result
-### Multi-Channel Object Detection
+### Multi-stream Object Detection
-Source code location: `./demos/object_detection/yolo_demo_multi.cpp`
+Source file: `./demos/object_detection/yolo_demo_multi.cpp`
- 
+ 
yolov5s multi result
### Pose Estimation
-Source code location: `./demos/pose_estimation/yolo_pose.cpp`
+Source file: `./demos/pose_estimation/yolo_pose.cpp`
+
yolov5s multi result
@@ -418,37 +332,30 @@ save file : result.jpg
### Image Segmentation
-Source code location: `./demos/segmentation/segmentation.cpp`
+Source file: `./demos/segmentation/segmentation.cpp`
-#### Image Segmentation
+#### Segmentation
```bash
-./bin/pidnet -m assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
+./bin/segmentation -m assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
```
```bash
-(.venv) rock@rock-5b-plus:~/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-runtime/dx_app$ ./bin/pidnet -m assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
-modelPath: assets/models/DeepLabV3PlusMobileNetV2_2.dxnn
+rock@rock-5b-plus:~/ssd/deepx/v2.1.0/dx_app$ ./bin/segmentation -m assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
+modelpath: assets/models/DeepLabV3PlusMobileNetV2_2.dxnn
+parameter: 0
+imgFile: sample/8.jpg
videoFile:
cameraInput: 0
-asyncInference: 0
-DXRT v2.9.5
-[ ] -> npu_0 -> [ ]
- InferenceEngine /mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/workspace/res/models/models-1_60_1/DeepLabV3PlusMobileNetV2_2.dxnn
- Task[0] npu_0, NPU, NPU memory usage 393316160 bytes (input 1228800 bytes, output 52428800 bytes)
- inputs
- input.1, UINT8, [1, 640, 640, 3 ], 0
- outputs
- 1063, FLOAT, [1, 640, 640, 32 ], 0
-
-
-sh: 1: ldconfig: not found
-640x640 <- 676x338
-save file : result.jpg
+Result saved to result.jpg
+[DXAPP] [INFO] total time : 316640 us
+[DXAPP] [INFO] per frame time : 316640 us
+[DXAPP] [INFO] fps : 3.16456
+
```
```bash
-./bin/od_pid -m0 assets/models/YOLOV5S_3.dxnn -m1 assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
+./bin/od_segmentation -m0 assets/models/YOLOV5S_3.dxnn -p0 1 -m1 assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
```
```bash
-(.venv) rock@rock-5b-plus:~/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-runtime/dx_app$ ./bin/od_pid -m0 assets/models/YOLOV5S_3.dxnn -m1 assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
+rock@rock-5b-plus:~/ssd/deepx/v2.1.0/dx_app$ ./bin/od_segmentation -m0 assets/models/YOLOV5S_3.dxnn -p0 1 -m1 assets/models/DeepLabV3PlusMobileNetV2_2.dxnn -i sample/8.jpg
od_modelpath: assets/models/YOLOV5S_3.dxnn
seg_modelpath: assets/models/DeepLabV3PlusMobileNetV2_2.dxnn
imgFile: sample/8.jpg
videoFile:
cameraInput: 0
-DXRT v2.9.5
-[ ] -> npu_0 -> [ ]
- InferenceEngine /mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/workspace/res/models/models-1_60_1/YOLOV5S_3.dxnn
- Task[0] npu_0, NPU, NPU memory usage 50736064 bytes (input 786432 bytes, output 131072 bytes)
- inputs
- images, UINT8, [1, 512, 1536 ], 0
- outputs
- BBOX, BBOX, [1, unknown ], 0
-
-
-[ ] -> npu_0 -> [ ]
- InferenceEngine /mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/workspace/res/models/models-1_60_1/DeepLabV3PlusMobileNetV2_2.dxnn
- Task[1] npu_0, NPU, NPU memory usage 393316160 bytes (input 1228800 bytes, output 52428800 bytes)
- inputs
- input.1, UINT8, [1, 640, 640, 3 ], 0
- outputs
- 1063, FLOAT, [1, 640, 640, 32 ], 0
-
-
-sh: 1: ldconfig: not found
-YOLO created : 16128 boxes, 80 classes, 3 layers.
+cfg.numBoxes: 16128
YoloParam:
- conf_threshold: 0.25, score_threshold: 0.3, iou_threshold: 0.4, num_classes: 80, num_layers: 3
- - LayerParam: [ 64 x 64 x 3boxes], anchorWidth [10, 16, 33, ], anchorHeight [13, 30, 23, ], tensor index [0, ]
- - LayerParam: [ 32 x 32 x 3boxes], anchorWidth [30, 62, 59, ], anchorHeight [61, 45, 119, ], tensor index [1, ]
- - LayerParam: [ 16 x 16 x 3boxes], anchorWidth [116, 156, 373, ], anchorHeight [90, 198, 326, ], tensor index [2, ]
+ - LayerParam: [ name : 378, 64 x 64 x 3boxes], anchorWidth [10, 16, 33, ], anchorHeight [13, 30, 23, ], tensor index [0, ]
+ - LayerParam: [ name : 439, 32 x 32 x 3boxes], anchorWidth [30, 62, 59, ], anchorHeight [61, 45, 119, ], tensor index [1, ]
+ - LayerParam: [ name : 500, 16 x 16 x 3boxes], anchorWidth [116, 156, 373, ], anchorHeight [90, 198, 326, ], tensor index [2, ]
- classes: [person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, trafficlight, firehydrant, stopsign, parkingmeter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sportsball, kite, baseballbat, baseballglove, skateboard, surfboard, tennisracket, bottle, wineglass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hotdog, pizza, donut, cake, chair, couch, pottedplant, bed, diningtable, toilet, tv, laptop, mouse, remote, keyboard, cellphone, microwave, oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddybear, hairdrier, toothbrush, ]
-640x640 <- 676x338
+YOLO created : 16128 boxes, 80 classes,
Detected 13 boxes.
- BBOX:car(2) 0.909042, (1.04781, 227.536, 92.2432, 303.1)
- BBOX:car(2) 0.888733, (425.907, 222.871, 512.271, 320.548)
- BBOX:car(2) 0.871704, (86.6853, 233.949, 142.035, 281.4)
- BBOX:car(2) 0.857452, (316.556, 228.92, 358.95, 260.397)
- BBOX:car(2) 0.854294, (364.961, 211.574, 482.898, 294.025)
- BBOX:car(2) 0.828323, (298.004, 231.673, 322.384, 252.117)
- BBOX:car(2) 0.809814, (122.821, 228.76, 168.794, 268.967)
- BBOX:car(2) 0.768341, (159.306, 230.32, 187.039, 261.545)
- BBOX:car(2) 0.689728, (262.998, 232.582, 276.37, 243.308)
- BBOX:car(2) 0.557953, (229.21, 232.742, 246.731, 241.766)
- BBOX:car(2) 0.53157, (277.398, 232.602, 296.908, 241.041)
- BBOX:car(2) 0.522827, (170.016, 214.851, 214.887, 258.137)
- BBOX:truck(7) 0.355202, (173.136, 216.022, 215.157, 256.622)
-save file : result.jpg
+ BBOX:car(2) 0.89303, (0.480949, 227.935, 93.051, 302.253)
+ BBOX:car(2) 0.882933, (426.194, 222.593, 511.927, 321.371)
+ BBOX:car(2) 0.862958, (86.1822, 234.01, 142.1, 281.351)
+ BBOX:car(2) 0.848508, (317.336, 228.731, 358.4, 260.354)
+ BBOX:car(2) 0.84496, (366.256, 210.955, 482.539, 293.438)
+ BBOX:car(2) 0.818492, (297.631, 231.888, 322.464, 252.024)
+ BBOX:car(2) 0.777943, (122.689, 229.362, 168.695, 268.896)
+ BBOX:car(2) 0.711904, (158.728, 229.85, 187.161, 261.946)
+ BBOX:car(2) 0.666848, (262.793, 232.572, 276.692, 243.45)
+ BBOX:car(2) 0.52669, (169.424, 215.719, 215.506, 257.233)
+ BBOX:car(2) 0.514303, (277.343, 232.588, 296.351, 241.116)
+ BBOX:car(2) 0.506402, (228.703, 232.507, 246.448, 241.995)
+ BBOX:truck(7) 0.343093, (173.549, 215.718, 214.675, 256.886)
+Result saved to result.jpg
+[DXAPP] [INFO] total time : 300656 us
+[DXAPP] [INFO] per frame time : 300656 us
+[DXAPP] [INFO] fps : 3.33333
```
+
+```bash
+pip install mkdocs mkdocs-material mkdocs-video pymdown-extensions mkdocs-with-pdf markdown-grid-tables
+```
+
+
+
+### Build Documentation
+
+
+
+```bash
+cd docs
+mkdocs build
+```
+
+
+
+After the build completes, `DEEPX_DX-APP_UM_v2.1.0.pdf` will be generated in the current directory.
+
+### Start the Documentation Server
+
+You can access the documentation in a web browser.
+
+
+
+```bash
+mkdocs serve
+```
+
+
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-com.md b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-com.md
index 7e4b37e9e..dbbcd3777 100644
--- a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-com.md
+++ b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-com.md
@@ -4,18 +4,18 @@ sidebar_position: 2
# DX-COM Model Compiler
-## Introduction
+## Overview
-**DX-COM (Model Compiler)** can generate NPU instruction sets based on the provided ONNX model and configuration file, compiling ONNX models into DXNN model file format that can be executed on NPU. DXNN files contain the generated instruction sets and weight information.
+**DX-COM (model compiler)** generates NPU instruction sets based on the provided ONNX model and configuration file, compiling the ONNX model into a DXNN model file that can run on the NPU. A DXNN file contains the generated instruction set and weight data.
## System Requirements
-| CPU | x86 (x64) |
-| ---------------- | -------------------------------- |
-| Memory (RAM) | ≥ 16GB |
-| Storage Space | ≥ 8GB |
-| Operating System | Ubuntu 18.04, 20.04, 22.04 (x64) |
-| LDD Version | ≥ 2.28 |
+| CPU | x86 (x64) |
+| ------------ | ------------------------------ |
+| Memory (RAM) | ≥ 16GB |
+| Storage | ≥ 8GB |
+| OS | Ubuntu 18.04/20.04/22.04 (x64) |
+| LDD Version | ≥ 2.28 |
## Installation
@@ -24,7 +24,7 @@ sidebar_position: 2
- libgl1-mesa-glx
- libglib2.0-0
-Use the following command to install the required libraries.
+Install the required libraries using the following command.
@@ -34,14 +34,14 @@ sudo apt-get install -y --no-install-recommends libgl1-mesa-glx libglib2.0-0 mak
-### Download DX-COM Installation Package
+### Download the DX-COM Package
-Please download the DX-COM installation package from the DX-M1 SDK [Resources Download](../download.md) page and extract it
+Download the DX-COM package from the DX-M1 SDK [Downloads](../download.md) page and extract it.
```bash
-tar -xvf dx_com_M1_v1.60.1.tar.gz
+tar -xvf dx_com_M1_v2.1.0.tar.gz
```
@@ -50,11 +50,11 @@ tar -xvf dx_com_M1_v1.60.1.tar.gz
### Model Configuration
-Parameters used for model compilation need to be configured in a JSON file.
+The parameters used for compilation must be configured in a JSON file.
#### Model Input Configuration
-Set the ONNX model input layer name and model input shape.
+Set the input layer name and input shape for the ONNX model.
```vim
inputs : ONNX input name shape info
@@ -71,23 +71,23 @@ inputs : ONNX input name shape info
512,
512
]
-}
+ }
```
-#### Model Calibration Configuration
+#### Calibration Method Configuration
-Set parameters for the calibration method.
+Configure the parameters for the calibration method.
-**calibration_num**: Number of images used for calibration
+**calibration_num**: number of images used for calibration.
-If you want to minimize model accuracy degradation, it is recommended to try different `calibration_num` values. You can compile the model with different settings (such as `calibration_num=1`, `5`, `10`, `100` or `1000`), measure the accuracy under each setting, and choose the value with the highest accuracy.
+To minimize accuracy loss, try different `calibration_num` values. You can compile the model with different settings (e.g., `calibration_num=1`, `5`, `10`, `100`, or `1000`), measure the accuracy for each setting, and choose the one with the best accuracy.
-**calibration_method**: Calibration method
+**calibration_method**: calibration method.
-Available options: `["ema", "minmax"]`
+Options: `["ema", "minmax"]`
:::tip
-In most cases, `"ema"` provides better accuracy, but in some cases `"minmax"` works better.
+In most cases, `"ema"` provides better accuracy, but in some scenarios `"minmax"` may work better.
:::
**Example:**
@@ -99,15 +99,15 @@ In most cases, `"ema"` provides better accuracy, but in some cases `"minmax"` wo
...
```
-#### Data Loading Configuration
+#### Data Loader Configuration
-Set parameters for the calibration dataset. It is recommended to select the same batch of data used for training or inference for calibration.
+Configure the parameters for the calibration dataset. It is recommended to use the same dataset you use for training or inference.
-- dataset_path: Root directory path of the calibration dataset (files will be searched recursively).
+- dataset_path: root directory of the calibration dataset (files will be searched recursively).
-- file_extensions: File extensions of target images.
+- file_extensions: target image file extensions.
-- preprocessings: Image preprocessing methods.
+- preprocessings: image preprocessing steps.
**Example:**
@@ -129,29 +129,29 @@ Set parameters for the calibration dataset. It is recommended to select the same
...
```
-Input image preprocessing configuration
-
-The preprocessing pipeline required before images are input to the ONNX model. These configurations are used to preprocess the calibration dataset during model compilation.
-Items marked with (\*) will be executed on the NPU side, thereby reducing CPU processing burden. Therefore, these preprocessing operations migrated to NPU should be omitted in the runtime software. If no preprocessing is needed, this section can be skipped.
-
-| Preprocessing Item | Parameters/Sub-options | Description | Example |
-| ------------------ | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |
-| `convertColor` | `RGB2BGR`, `BGR2RGB`, `BGR2GRAY`, `BGR2YCrCb` | Convert color channel order | `"RGB2BGR"` |
-| `resize` | `mode` | `default` (opencv) or `torchvision` | `"default"` / `"torchvision"` |
-| | `width`, `height` | Target image size (only `default` mode) | `width: 224`, `height: 224` |
-| | `size` | Target short side size (only `torchvision` mode) | `size: 256` |
-| | `interpolation` | Interpolation method (optional)
**`default`** mode options: "NEAREST", "LINEAR"(default), "AREA", "CUBIC", "LANCZOS4"
**`torchvision`** mode options:
"NEAREST", "LANCZOS", "BILINEAR"(default), "BICUBIC", "BOX", "HAMMING" | `LINEAR`, `BILINEAR` | -| `centercrop` | `width`, `height` | Crop specified size from image center | `width: 224`, `height: 224` | -| `transpose` | `axis` | Dimension transformation order | `[2, 0, 1]` | -| `normalize (*)` | `mean`, `std` | Normalization processing, supports single value or array | `[0.5,0.5,0.5]` | -| `mul (*)` | `x` | Multiply all pixels by a value | `x: 255` | -| `add (*)` | `x` | Add a value to all pixels | `x: 0.1` | -| `subtract (*)` | `x` | Subtract a value from all pixels | `x: 0.1` | -| `div (*)` | `x` | Divide all pixels by a value | `x: 255.0` | -| `expandDim` | `axis` | Expand a dimension at specified position | `axis: 0` | +Input Image Preprocessing Configuration + +The preprocessing pipeline required before feeding images into the ONNX model. These configurations are used to preprocess the calibration dataset during compilation. +Items marked with (\*) will be executed on the NPU side to reduce CPU workload. Therefore, preprocessing steps moved to the NPU should be omitted in runtime software. If no preprocessing is required, you can skip this section. + +| Preprocessing | Parameters / Options | Description | Example | +| --------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | +| `convertColor` | `RGB2BGR`, `BGR2RGB`, `BGR2GRAY`, `BGR2YCrCb` | Convert color channel order | `"RGB2BGR"` | +| `resize` | `mode` | `default` (opencv) or `torchvision` | `"default"` / `"torchvision"` | +| | `width`, `height` | Target image size (default mode only) | `width: 224`, `height: 224` | +| | `size` | Target short-side size (torchvision mode only) | `size: 256` | +| | `interpolation` | Interpolation (optional)
Options for **`default`**: "NEAREST", "LINEAR"(default), "AREA", "CUBIC", "LANCZOS4"
Options for **`torchvision`**:
"NEAREST", "LANCZOS", "BILINEAR"(default), "BICUBIC", "BOX", "HAMMING" | `LINEAR`, `BILINEAR` | +| `centercrop` | `width`, `height` | Crop the given size from the image center | `width: 224`, `height: 224` | +| `transpose` | `axis` | Dimension order (permute) | `[2, 0, 1]` | +| `normalize (*)` | `mean`, `std` | Normalization; supports scalar or array | `[0.5,0.5,0.5]` | +| `mul (*)` | `x` | Multiply all pixels by a value | `x: 255` | +| `add (*)` | `x` | Add a value to all pixels | `x: 0.1` | +| `subtract (*)` | `x` | Subtract a value from all pixels | `x: 0.1` | +| `div (*)` | `x` | Divide all pixels by a value | `x: 255.0` | +| `expandDim` | `axis` | Expand one dimension at the specified axis | `axis: 0` | :::info -Items marked with (\*) will be executed on the NPU side +Items marked with (\*) will be executed on the NPU side. ::: #### Model Configuration Example @@ -212,31 +212,31 @@ Items marked with (\*) will be executed on the NPU side ] ``` -#### Accuracy Enhancement Scheme Configuration +#### Enhanced Accuracy Scheme Configuration -To further improve model accuracy, you can choose to use the accuracy enhancement scheme (Enhanced Scheme), with scheme names **DXQ-P0 ~ DXQ-P5**. +To further improve model accuracy, you can enable the Enhanced Scheme options named **DXQ-P0 \~ DXQ-P5**. :::tip -- Each enhancement scheme is independent, meaning multiple schemes can be enabled simultaneously. -- Using these schemes **does not always bring accuracy improvement**, it is recommended to evaluate the effect based on actual situations; -- Enabling these schemes in **CPU-only environments will significantly increase compilation time**. +- Each scheme is independent, so you can enable multiple schemes at the same time. +- These schemes **do not always improve accuracy**; evaluate the results based on your use case. +- Enabling these schemes in a **CPU-only environment can significantly increase compilation time**. ::: -| Scheme ID | Name | Parameter Description | -| ---------- | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **DXQ-P0** | Operation Fusion (Fusing) | `alpha`: Fusion ratio, value range `0 < alpha < 1.0`, **recommended value: 0.5** | -| **DXQ-P1** | Find Optimal Parameters | `true/false`, enabling will significantly increase CPU compilation time | -| **DXQ-P2** | Relax Calibration Range to Find Optimal Parameters | `alpha`: Minimum value ratio of calibration range, `0 < alpha < beta`, recommended value: **0.1**, `beta`: Maximum value ratio of calibration range, `alpha < beta < ∞`, recommended value: **1.0** | -| **DXQ-P3** | Parameter Fine-tuning (Method 1) | `num_samples`: Number of data samples used for calibration, `0 < num_samples < ∞`, **recommended value: 1024** | -| **DXQ-P4** | Parameter Fine-tuning (Method 2) | Same as DXQ-P3 | -| **DXQ-P5** | Parameter Fine-tuning (Method 3) | Same as DXQ-P3 | +| Scheme ID | Name | Parameter Description | +| ---------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| **DXQ-P0** | Fusing | `alpha`: fusing ratio, range `0 < alpha < 1.0`, **recommended: 0.5** | +| **DXQ-P1** | Search Best Parameters | `true/false`; enabling it significantly increases CPU compilation time | +| **DXQ-P2** | Relax Range Search | `alpha`: min ratio of calibration range, `0 < alpha < beta`, recommended: **0.1**; `beta`: max ratio, `alpha < beta < ∞`, recommended: **1.0** | +| **DXQ-P3** | Fine-tuning (Method 1) | `num_samples`: number of samples for calibration, `0 < num_samples < ∞`, **recommended: 1024** | +| **DXQ-P4** | Fine-tuning (Method 2) | Same as DXQ-P3 | +| **DXQ-P5** | Fine-tuning (Method 3) | Same as DXQ-P3 | :::tip -Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different fine-tuning methods**. +Although DXQ-P3, DXQ-P4, and DXQ-P5 share the same parameters, they use **different fine-tuning methods**. ::: -**Accuracy Enhancement Configuration Example** +**Enhanced scheme configuration example** ```json ''' @@ -263,7 +263,7 @@ Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different f } ``` -**Accuracy Enhancement Single Scheme Control Configuration Example** +**Enhanced scheme enable/disable example** ```json ''' @@ -278,18 +278,136 @@ Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different f ''' ``` -| Name | Compilation Speed | Accuracy Improvement Tendency | -| ------ | ----------------- | ----------------------------- | -| DXQ-P0 | Very Fast | Low | -| DXQ-P1 | Fast | Low-Medium | -| DXQ-P2 | Slower | Medium-High | -| DXQ-P3 | Very Slow | High | -| DXQ-P4 | Very Slow | High | -| DXQ-P5 | Very Slow | High | +| Name | Compile Speed | Accuracy Improvement Tendency | +| ------ | ------------- | ----------------------------- | +| DXQ-P0 | Very fast | Low | +| DXQ-P1 | Fast | Low-Medium | +| DXQ-P2 | Slow | Medium-High | +| DXQ-P3 | Very slow | High | +| DXQ-P4 | Very slow | High | +| DXQ-P5 | Very slow | High | + +### PPU Post-processing Acceleration + +PPU (Post-Processing Unit) provides hardware-accelerated post-processing for object detection models. If your target model uses a YOLO-family architecture, you can add PPU configuration parameters in the model compilation config file. + +#### Supported PPU Types + +There are two supported object detection model architectures for PPU: + +- **Type 0 (Anchor-Based YOLO)** + + Designed for anchor-based models, such as YOLOv3, YOLOv4, YOLOv5, and YOLOv7. + +- **Type 1 (Anchor-Free YOLO)** + + Designed for anchor-free models, such as YOLOX. + +#### Identify ONNX Nodes That Can Use PPU + +
+
+
+
+
+ Trace backward from the model outputs to find detection heads whose `Conv` output shape is `[1, num_anchors*(5+num_classes), H, W]`. These conv outputs are typically followed by `reshape`, `permute`, and other post-processing operations.
-### Execute Model Compilation
+ Using [yolo5s.onnx](https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5s.onnx) as an example: YOLOv5s has `num_anchors=3` and `num_classes=80`, so the `Conv` nodes to look for have output shape `[1, 255, H, W]`.
+ By tracing backward from the model outputs, you can identify three `Conv` nodes: `Conv_196`, `Conv_308`, and `Conv_420`. Configure these node names in the PPU section of the compilation config file.
-When the model configuration file is completed, use the following command to compile the model
+
+
+
+
+ Trace backward from the model outputs. For each detection scale, identify three types of `Conv` nodes: `bbox`, `obj_conf`, and `cls_conf`.
+
+ - `bbox`: conv layer that outputs bounding box regression values `[1, 4, H, W]`
+
+ - `obj_conf`: conv layer that outputs object confidence scores `[1, 1, H, W]`
+
+ - `cls_conf`: conv layer that outputs class prediction scores (e.g. `[1, 80, H, W]`)
+
+ Using [YOLOXS-1.onnx](https://sdk.deepx.ai/modelzoo/onnx/YOLOXS-1.onnx) as an example: YOLOXS has three scales. You need to identify nine `Conv` nodes in total (`bbox`, `obj_conf`, `cls_conf` for each scale).
+ By tracing backward from the model outputs, you can identify these 9 `Conv` nodes and then configure their names in the PPU section of the compilation config file.
+
+
+
+
+
+#### Parameter Description
+
+| Parameter | Type | Description |
+| ---------------- | ------ | -------------------------------------------------------------------------------------------------------------------- |
+| `type` | int | Model type identifier. Setting `type=1` indicates an **Anchor-Free YOLO model**. |
+| `conf_thres` | float | Confidence threshold for filtering detections.
**Note: fixed at compile time and cannot be changed at runtime.** | +| `num_classes` | int | Number of classes supported by the model. | +| `layer` | list | Detection head configuration list. Each element corresponds to one head and contains the following fields: | +| `layer.bbox` | string | Layer name that outputs **Bounding Box** coordinates. | +| `layer.obj_conf` | string | Layer name that outputs **Object Confidence**. | +| `layer.cls_conf` | string | Layer name that outputs **Class Confidence**. | + +### Run Model Compilation + +After finishing the model configuration file, compile the model using the following command.
@@ -303,7 +421,11 @@ dx_com -m -c -o
-#### Compilation Example
+#### Compilation Examples
+
+:::tip
+The `sample` directory under dx-com contains multiple example compilation config files.
+:::
@@ -324,7 +446,54 @@ $./dx_com/dx_com \
:::tip
-**DX-COM** only supports model input with 1 batchsize. If model input is (**batchsize**,C,H,W) or (**batchsize**,H,W,C), please ensure batchsize is 1
+**DX-COM** only supports models with batch size 1. If the model input shape is (**batchsize**, C, H, W) or (**batchsize**, H, W, C), make sure `batchsize` is set to 1.
+:::
+
+After successful compilation, the DXNN model will be saved under the `output/mobilenetv1` directory.
+
+## Build DX-COM Documentation
+
+:::tip
+For more details on how to use DX-COM, please build the full documentation and refer to it.
+:::
+
+### Clone the DX-ALL-SUITE Repository
+
+:::tip
+Clone the specified DX-ALL-SUITE version following [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite).
:::
-After successful compilation, the dxnn format model is saved in the output/mobilenetv1 folder.
+### Install MkDocs
+
+
+
+```bash
+pip install mkdocs mkdocs-material mkdocs-video pymdown-extensions mkdocs-with-pdf markdown-grid-tables
+```
+
+
+
+### Build Documentation
+
+
+
+```bash
+cd dx-all-suite/dx-compiler
+mkdocs build
+```
+
+
+
+After the build completes, `DEEPX_DX-COM_UM_v2.1.0_Nov_2025.pdf` will be generated in the current directory.
+
+### Start the Documentation Server
+
+You can access the documentation in a web browser.
+
+
+
+```bash
+mkdocs serve
+```
+
+
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-model-zoo.md b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-model-zoo.md
index c104b4e32..20e2238cf 100644
--- a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-model-zoo.md
+++ b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-model-zoo.md
@@ -4,12 +4,12 @@ sidebar_position: 7
# DX Model Zoo
-**DX Model Zoo** contains a series of pre-compiled dxnn models for object classification, object detection, image segmentation, face recognition, image denoising, and more.
-Each model provides the original model source, ONNX model, compiled DXNN model, compilation configuration file downloads, along with benchmark, accuracy, and other data.
+**DX Model Zoo** includes a series of pre-compiled DXNN models for image classification, object detection, image segmentation, face recognition, image denoising, and more.
+Each model provides the original model source, the ONNX model, the compiled DXNN model, the compilation config file download, and benchmark/accuracy data.
## DX Model Link
-Link: [**DX Model Zoo**](https://developer.deepx.ai/wp-content/modelzoo/model_zoo_fin.html)
+Link: [**DX Model Zoo**](https://developer.deepx.ai/wp-content/modelzoo/modelzoo_v2.0.0.html)
@@ -18,28 +18,34 @@ Link: [**DX Model Zoo**](https://developer.deepx.ai/wp-content/modelzoo/model_zo
## Installation
-### Clone DX-ALL-SUITE Repository
+:::info
+DX-Model-Zoo is only supported on x86 Ubuntu 20.04 LTS or 22.04 LTS.
+:::
+
+### Clone the DX-ALL-SUITE Repository
:::tip
-Please follow [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite) to clone the specified version of the DX-ALL-SUITE repository
+Clone the specified DX-ALL-SUITE version following [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite).
:::
### Install DXMZ (Deepx Model Zoo)
-Navigate to the `dx-all-suite/dx-modelzoo` directory
+#### Manual Installation
-
+Enter the `dx-all-suite/dx-modelzoo` directory.
+
+
```bash
cd dx-all-suite/dx-modelzoo
-./setup.sh
+./install.sh
```
-After installation, a `./venv-dx-modelzoo-local` virtual environment will be created in the current path,
+After installation, a virtual environment will be created at `./venv-dx-modelzoo-local`.
-
+
```bash
source ./venv-dx-modelzoo-local/bin/activate
@@ -49,7 +55,7 @@ source ./venv-dx-modelzoo-local/bin/activate
## CLI Usage
-### List Model List
+### List Models
Command:
@@ -57,9 +63,9 @@ Command:
dxmz models
```
-Description: Used to list the model list on Deepx Model Zoo
+Description: list all models available in Deepx Model Zoo.
-
+
```bash
dxmz models
@@ -67,17 +73,7 @@ dxmz models
-```bash
-(venv-dx-modelzoo-local) rock@rock-5b-plus:~/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-modelzoo$ dxmz models
-/mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-modelzoo/venv-dx-modelzoo-local/lib/python3.11/site-packages/pydantic_settings/main.py:426: UserWarning: Config key `pyproject_toml_table_header` is set in model_config but will be ignored because no PyprojectTomlConfigSettingsSource source is configured. To use this config key, add a PyprojectTomlConfigSettingsSource source to the settings sources via the settings_customise_sources hook.
- self._settings_warn_unused_config_keys(sources, self.model_config)
-/mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-modelzoo/venv-dx-modelzoo-local/lib/python3.11/site-packages/pydantic_settings/main.py:426: UserWarning: Config key `toml_file` is set in model_config but will be ignored because no TomlConfigSettingsSource source is configured. To use this config key, add a TomlConfigSettingsSource source to the settings sources via the settings_customise_sources hook.
- self._settings_warn_unused_config_keys(sources, self.model_config)
-Available Model List:
-['AlexNet', 'BiSeNetV1', 'BiSeNetV2', 'DeepLabV3PlusDRN', 'DeepLabV3PlusMobileNetV2', 'DeepLabV3PlusMobilenet', 'DeepLabV3PlusResNet101', 'DeepLabV3PlusResNet50', 'DeepLabV3PlusResnet', 'DenseNet121', 'DenseNet161', 'DnCNN_15', 'DnCNN_25', 'DnCNN_50', 'EfficientNetB2', 'EfficientNetV2S', 'HarDNet39DS', 'MobileNetV1', 'MobileNetV2', 'MobileNetV3Large', 'MobileNetV3Small', 'OSNet0_25', 'OSNet0_5', 'RegNetX400MF', 'RegNetX800MF', 'RegNetY200MF', 'RegNetY400MF', 'RegNetY800MF', 'RepVGGA1', 'ResNeXt26_32x4d', 'ResNeXt50_32x4d', 'ResNet101', 'ResNet152', 'ResNet18', 'ResNet34', 'ResNet50', 'SSDMV1', 'SSDMV2Lite', 'SqueezeNet1_0', 'SqueezeNet1_1', 'VGG11', 'VGG11BN', 'VGG13', 'VGG13BN', 'VGG19BN', 'WideResNet101_2', 'WideResNet50_2', 'YOLOv5m_Face', 'YOLOv5s_Face', 'YOLOv7_Face', 'YOLOv7_TTA_Face', 'YOLOv7_W6_Face', 'YOLOv7_W6_TTA_Face', 'YOLOv7s_Face', 'YoloV3', 'YoloV5L', 'YoloV5M', 'YoloV5N', 'YoloV5S', 'YoloV7', 'YoloV7E6', 'YoloV7Tiny', 'YoloV8L', 'YoloV8M', 'YoloV8N', 'YoloV8S', 'YoloV8X', 'YoloV9C', 'YoloV9S', 'YoloV9T', 'YoloXS']
-```
-
-### Model Accuracy Evaluation
+### Evaluate Model Accuracy
Command:
@@ -85,7 +81,7 @@ Command:
dxmz eval [--onnx | --dxnn ] --data_dir
```
-Description: Used to measure the accuracy of the specified model
+Description: measure the accuracy of a specified model.
```bash
# To use onnx runtime:
@@ -100,12 +96,43 @@ dxmz eval --dxnn --data_dir
+
+```bash
+pip install mkdocs mkdocs-material mkdocs-video pymdown-extensions mkdocs-with-pdf markdown-grid-tables
+```
+
+
-- [**ImageNet**](http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_img_val.tar)
-- [**COCO2017**](http://images.cocodataset.org/zips/val2017.zip)
-- [**PascalVOC - VOC2007**](http://host.robots.ox.ac.uk/pascal/VOC/voc2007/index.html)
-- [**PascalVOC - VOC2012**](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
-- [**Cityscapes**](https://www.cityscapes-dataset.com/)
-- [**WiderFace**](http://shuoyang1213.me/WIDERFACE/)
-- [**BSD68**](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/)
+### Build Documentation
+
+
+
+```bash
+cd docs
+mkdocs build
+```
+
+
+
+After the build completes, `dx-modelzoo_v0.1.0.pdf` will be generated under `output_web`.
+
+### Start the Documentation Server
+
+You can access the documentation in a web browser.
+
+
+
+```bash
+mkdocs serve
+```
+
+
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-npu-driver.md b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-npu-driver.md
index 43260b474..760695492 100644
--- a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-npu-driver.md
+++ b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-npu-driver.md
@@ -4,13 +4,13 @@ sidebar_position: 4
# DXRT-NPU-Driver
-**DXRT-NPU-Driver** is the driver software for Radxa AICore DX-M1. Users need to install this driver software on the host where the AICore DX-M1 is installed.
+**DXRT-NPU-Driver** is the driver software for the Radxa AI Core DX-M1. You need to install this driver on the host where the DX-M1 is installed.
## Installation
-### Check Hardware Connection Status
+### Check Hardware Connection
-Please refer to [Hardware Installation](../hardware_install) to install AICore DX-M1
+Install the Radxa AI Core DX-M1 by following [Hardware Installation](../hardware_install).
@@ -24,7 +24,7 @@ lspci
0001:11:00.0 Processing accelerators: DEEPX Co., Ltd. DX_M1 (rev 01)
```
-On some systems, the name may not be printed and the following information will be output:
+On some systems, the device name may not be shown and you may see the following output instead:
```bash
0b:00.0 1200: 1ff4:0000
@@ -40,15 +40,15 @@ sudo apt install git gcc g++ cmake ninja-build python3-dev python3-venv kmod lib
-### Clone DX-ALL-SUITE Repository
+### Clone the DX-ALL-SUITE Repository
:::tip
-Please follow [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite) to clone the specified version of the DX-ALL-SUITE repository
+Clone the specified DX-ALL-SUITE version following [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite).
:::
-### Install Driver
+### Install the Driver
-Navigate to the `dx-all-suite/dx-runtime/dx_rt_npu_linux_driver/modules` directory
+Enter the `dx-all-suite/dx-runtime/dx_rt_npu_linux_driver/modules` directory.
@@ -59,49 +59,93 @@ cd dx-all-suite/dx-runtime/dx_rt_npu_linux_driver/modules
:::tip
-DXRT-NPU-Driver installation has two methods, you can use
+There are three ways to install DXRT-NPU-Driver. Choose any one of the following:
-- Script installation
+- Install via .deb package
-- Makefile compilation
- :::
+- Build via script
-- Script installation
+- Build via Makefile
-
+:::
+
+#### Install via .deb Package
+
+
+
+```bash
+sudo apt install ./release/latest/dxrt-driver-dkms_1.8.0-2_all.deb
+```
+
+
+
+#### Build via Script
+
+Build dx-npu-driver using the `build.sh` script.
+
+
+
+```bash
+sudo ./build.sh -d m1 -m deepx -c install
+```
+
+
+
+```bash
+ Usage:
+ build.sh
+
+ options:
+ -d, --device [device] select target device: m1 v3
+ -m, --module [module] select PCIe module: deepx
+ -k, --kernel [kernel dir] 'KERNEL_DIR=[kernel dir]', The directory where the kernel source is located
+ default: /lib/modules/6.1.84-7-rk2410/build)
+ -a, --arch [arch] set 'ARCH=[arch]' Target CPU architecture for cross-compilation, default: aarch64
+ -t, --compiler [cross tool] 'CROSS_COMPILE=[cross tool]' cross compiler binary, e.g aarch64-linux-gnu-
+ -i, --install [install dir] 'INSTALL_MOD_PATH=[install dir]', module install directory
+ install to: [install dir]/lib/modules/[KERNELRELEASE]/extra/
+ -c, --command [command] clean | install | uninstall
+ - uninstall: Remove the module files installed on the host PC.
+ -j, --jops [jobs] set build jobs
+ -f, --debug [debug] set debug feature [debugfs | log | all]
+ -v, --verbose build verbose (V=1)
+ -h, --help show this help
+```
+
+Reboot the host.
+
+
- ```bash
- ./build.sh -d m1 -m deepx
- sudo ./build.sh -d m1 -m deepx -c install
- ```
+```bash
+sudo reboot
+```
-
+
-- Makefile compilation
+#### Build via Makefile
- Compile dx-npu-driver
+Build dx-npu-driver using make.
-
+
- ```bash
- make DEVICE=m1 PCIE=deepx
- sudo make DEVICE=m1 PCIE=deepx install
- ```
+```bash
+sudo make DEVICE=m1 PCIE=deepx install
+```
-
+
- Configure automatic driver loading on boot
+Configure the driver to load on boot.
-
+
- ```bash
- sudo depmod -A
- sudo cp dx_dma.conf /etc/modprobe.d/
- ```
+```bash
+sudo depmod -A
+sudo cp dx_dma.conf /etc/modprobe.d/
+```
-
+
-Restart host
+Reboot the host.
@@ -127,13 +171,24 @@ dxrt_driver 45056 2
dx_dma 483328 9 dxrt_driver
```
-### Uninstall Driver
+### Uninstall the Driver
+
+#### Uninstall (Installed via .deb)
```bash
-cd dx-all-suite/dx-runtime/dx_rt_npu_linux_driver/modules
-sudo ./build.sh -d m1 -m deepx -c uninstall
+sudo apt remove dxrt-driver-dkms
+```
+
+
+
+#### Uninstall (Installed via script)
+
+
+
+```bash
+./build.sh -d m1 -m deepx -c uninstall
```
diff --git a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-rt.md b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-rt.md
index 84d7c7eb6..d0807af73 100644
--- a/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-rt.md
+++ b/i18n/en/docusaurus-plugin-content-docs/current/aicore/dx-m1/dx-sdk/dx-rt.md
@@ -4,20 +4,20 @@ sidebar_position: 5
# DX-RT Runtime
-**DX-RT (Runtime Software)** is the runtime for users to interact with DEEPX NPU hardware, enabling pre-processing and post-processing during inference, managing NPU input and output.
-DX-RT supports inference using dxnn format models compiled by DX-COM.
+**DX-RT (runtime software)** is the runtime used to interact with DEEPX NPU hardware. It provides pre/post-processing during inference and manages NPU inputs and outputs.
+DX-RT supports running inference with DXNN models compiled by DX-COM.
## Installation
-### Clone DX-ALL-SUITE Repository
+### Clone the DX-ALL-SUITE Repository
:::tip
-Please follow [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite) to clone the specified version of the DX-ALL-SUITE repository
+Clone the specified DX-ALL-SUITE version following [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite).
:::
-### Compile DX-RT
+### Build DX-RT
-Navigate to the `dx-all-suite/dx-runtime/dx_rt` directory
+Enter the `dx-all-suite/dx-runtime/dx_rt` directory.
@@ -27,9 +27,9 @@ cd dx-all-suite/dx-runtime/dx_rt
-#### Create Python Virtual Environment
+#### Create a Python Virtual Environment
-This virtual environment is used to install DXRT Python API. Users can create Python virtual environment in any directory
+This virtual environment is used to install the DXRT Python API. You can create a Python virtual environment in any directory.
@@ -41,20 +41,45 @@ pip3 install --upgrade pip
-#### Install onnxruntime
+#### Install Required Dependencies
```bash
-./install.sh --arch aarch64 --onnxruntime
+./install.sh --all
```
-#### Compile and Install DX-RT
+```bash
+Usage: ./install.sh [OPTIONS]
+Install necessary components and libraries for the project.
+
+System Requirements:
+ • Architecture: x86_64 or aarch64
+ • RAM: 8GB or more
+
+Options:
+ --help Display this help message and exit.
+ --arch Specify the target CPU architecture. Valid options: [x86_64, aarch64].
+ --dep Install core dependencies such as cmake, gcc, ninja, etc and python3.
+ --onnxruntime (Optional) Install the ONNX Runtime library.
+ --all Install all dependencies and the ONNX Runtime library.
+
+ --python_version Specify the Python version to install (e.g., 3.10.4).
+ * Minimum supported version: .
+ * If not specified:
+ - For Ubuntu 20.04+, the OS default Python 3 will be used.
+ - For Ubuntu 18.04, Python will be source-built.
+ --venv_path Specify the path for the virtual environment.
+ * If this option is omitted, no virtual environment will be created.
+ --help Display this help message and exit.
+```
+
+#### Build and Install DX-RT
:::tip
-DX-RT installation script will automatically install dx-engine Python API
+The DX-RT installation script will automatically install the dx-engine Python API.
:::
@@ -65,9 +90,9 @@ DX-RT installation script will automatically install dx-engine Python API
-### Register dxrt-service Service
+### Register the dxrt-service
-DX-RT supports background multi-threaded operations. To enable multi-threading features, you need to register dxrt-service
+DX-RT supports background multi-threading. To enable this feature, you need to register `dxrt-service`.
@@ -79,10 +104,10 @@ sudo systemctl enable dxrt.service
-### Install DX-RT Python Library
+### Install the DX-RT Python Package
:::tip
-DX-RT installation script will automatically install dx-engine Python API
+The DX-RT installation script will automatically install the dx-engine Python API.
:::
@@ -94,197 +119,425 @@ pip3 install .
+### Uninstall DX-RT
+
+
+
+```bash
+cd dx-all-suite/dx-runtime/dx_rt
+./build.sh --uninstall
+```
+
+
+
## Usage
### dxrt-cli
-`dxrt-cli` can read Radxa AICore DX-M1 device status
-Source code location: `dx_rt/cli/dxrt-cli.cpp`
+`dxrt-cli` reads the Radxa AI Core DX-M1 device status. Source: `dx_rt/cli/dxrt-cli.cpp`.
```bash
dxrt-cli
@@ -456,66 +363,49 @@ save file : result.jpg
segmentation result
-#### Image Detection and Segmentation
+#### Object Detection + Segmentation
@@ -526,12 +416,22 @@ save file : result.jpg
## Python Application Demos
:::tip
-Before using Python for model inference, please ensure you have entered the virtual environment where the [**dx-engine Python library**](dx-rt#install-dx-rt-python-library) is installed
+Before running model inference in Python, make sure you have entered the virtual environment where the [**dx-engine Python library**](dx-rt#install-the-dx-rt-python-package) is installed.
:::
+Install Python dependencies
+
+
+
+```bash
+pip install -r ./templates/python/requirements.txt
+```
+
+
+
### ImageNet Python Example
-Source code location: `./templates/python/imageNet_example.py`
+Source file: `./templates/python/imageNet_example.py`
@@ -542,28 +442,18 @@ python3 ./templates/python/imageNet_example.py --config example/run_classifier/i
```bash
-(.venv) rock@rock-5b-plus:~/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-runtime/dx_app$ python3 ./templates/python/imageNet_example.py --config example/run_classifier/imagenet_example.json
-DXRT v2.9.5
-[ ] -> npu_0 -> [ ]
- InferenceEngine /mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/workspace/res/models/models-1_60_1/EfficientNetB0_4.dxnn
- Task[0] npu_0, NPU, NPU memory usage 21199872 bytes (input 157696 bytes, output 2 bytes)
- inputs
- data, UINT8, [1, 224, 704 ], 0
- outputs
- argmax_output, UINT16, [1 ], 0
-
-
+(.venv) rock@rock-5b-plus:~/ssd/deepx/v2.1.0/dx_app$ python3 ./templates/python/imageNet_example.py --config example/run_classifier/imagenet_example.json
[DX-APP Notify] single run inference using run
-[sample/ILSVRC2012/0.jpeg] Top1 Result : class 831 (studio couch, day bed)
+[sample/ILSVRC2012/0.jpeg] Top1 Result : class 905 (window shade)
[sample/ILSVRC2012/1.jpeg] Top1 Result : class 321 (admiral)
[sample/ILSVRC2012/2.jpeg] Top1 Result : class 846 (table lamp)
[sample/ILSVRC2012/3.jpeg] Top1 Result : class 794 (shower curtain)
```
-### Yolov5s Python Example
+### YOLOv5s Python Example
-Source code location: `templates/python/yolov5s_example.py`
+Source file: `templates/python/yolov5s_example.py`
@@ -574,60 +464,68 @@ python3 templates/python/yolov5s_example.py
```bash
-(.venv) rock@rock-5b-plus:~/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-runtime/dx_app$ python3 templates/python/yolov5s_example.py
-DXRT v2.9.5
-[ ] -> npu_0 -> [ ]
- InferenceEngine /mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/workspace/res/models/models-1_60_1/YOLOV5S_3.dxnn
- Task[0] npu_0, NPU, NPU memory usage 50736064 bytes (input 786432 bytes, output 131072 bytes)
- inputs
- images, UINT8, [1, 512, 1536 ], 0
- outputs
- BBOX, BBOX, [1, unknown ], 0
-
-
+(.venv) rock@rock-5b-plus:~/ssd/deepx/v2.1.0/dx_app$ python3 templates/python/yolov5s_example.py
dxrt inference Done!
-/mnt/ssd/deepx/dx_rt_SDK_v2.9.5/dx-all-suite/dx-runtime/dx_app/templates/python/yolov5s_example.py:110: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
- tensor[4] = score
decoding output Done!
-[Result] Detected 7 Boxes.
-[0] conf, classID, x1, y1, x2, y2, : 0.7701, person(0), 307, 140, 400, 363
-[1] conf, classID, x1, y1, x2, y2, : 0.6203, bowl(45), 25, 359, 80, 393
-[2] conf, classID, x1, y1, x2, y2, : 0.6127, bowl(45), 46, 316, 107, 347
-[3] conf, classID, x1, y1, x2, y2, : 0.4740, oven(69), 0, 227, 154, 324
-[4] conf, classID, x1, y1, x2, y2, : 0.4365, bowl(45), 0, 329, 69, 380
-[5] conf, classID, x1, y1, x2, y2, : 0.3829, oven(69), 391, 246, 495, 358
-[6] conf, classID, x1, y1, x2, y2, : 0.3426, person(0), 0, 296, 47, 333
+[Result] Detected 11 Boxes.
+[0] conf, classID, x1, y1, x2, y2, : 0.8739, person(0), 383, 66, 501, 348
+[1] conf, classID, x1, y1, x2, y2, : 0.7595, bowl(45), 57, 288, 133, 327
+[2] conf, classID, x1, y1, x2, y2, : 0.7423, bowl(45), 31, 342, 98, 385
+[3] conf, classID, x1, y1, x2, y2, : 0.6719, oven(69), 0, 170, 191, 300
+[4] conf, classID, x1, y1, x2, y2, : 0.5923, bowl(45), 0, 303, 86, 368
+[5] conf, classID, x1, y1, x2, y2, : 0.5887, oven(69), 487, 201, 621, 342
+[6] conf, classID, x1, y1, x2, y2, : 0.5675, person(0), 0, 263, 60, 307
+[7] conf, classID, x1, y1, x2, y2, : 0.4376, potted plant(58), 0, 0, 62, 151
+[8] conf, classID, x1, y1, x2, y2, : 0.3855, bottle(39), 215, 231, 251, 297
+[9] conf, classID, x1, y1, x2, y2, : 0.3226, cup(41), 147, 268, 171, 302
+[10] conf, classID, x1, y1, x2, y2, : 0.3183, bowl(45), 155, 167, 181, 185
save file : yolov5s_1.jpg
dxrt inference Done!
decoding output Done!
-[Result] Detected 6 Boxes.
-[0] conf, classID, x1, y1, x2, y2, : 0.7209, tv(62), 2, 218, 124, 296
-[1] conf, classID, x1, y1, x2, y2, : 0.6937, chair(56), 234, 259, 284, 339
-[2] conf, classID, x1, y1, x2, y2, : 0.6208, chair(56), 289, 258, 346, 336
-[3] conf, classID, x1, y1, x2, y2, : 0.4169, person(0), 337, 212, 371, 323
-[4] conf, classID, x1, y1, x2, y2, : 0.4062, tv(62), 443, 252, 512, 316
-[5] conf, classID, x1, y1, x2, y2, : 0.4052, vase(75), 439, 326, 469, 404
+[Result] Detected 9 Boxes.
+[0] conf, classID, x1, y1, x2, y2, : 0.8261, chair(56), 291, 215, 355, 318
+[1] conf, classID, x1, y1, x2, y2, : 0.7895, tv(62), 2, 164, 155, 266
+[2] conf, classID, x1, y1, x2, y2, : 0.7772, chair(56), 361, 215, 433, 311
+[3] conf, classID, x1, y1, x2, y2, : 0.6629, person(0), 422, 158, 463, 296
+[4] conf, classID, x1, y1, x2, y2, : 0.6128, vase(75), 548, 299, 586, 399
+[5] conf, classID, x1, y1, x2, y2, : 0.5882, tv(62), 553, 208, 640, 286
+[6] conf, classID, x1, y1, x2, y2, : 0.5303, person(0), 385, 171, 403, 211
+[7] conf, classID, x1, y1, x2, y2, : 0.4672, dining table(60), 470, 360, 637, 423
+[8] conf, classID, x1, y1, x2, y2, : 0.4226, refrigerator(72), 451, 168, 510, 290
save file : yolov5s_2.jpg
dxrt inference Done!
decoding output Done!
-[Result] Detected 4 Boxes.
-[0] conf, classID, x1, y1, x2, y2, : 0.7521, person(0), 6, 238, 85, 398
-[1] conf, classID, x1, y1, x2, y2, : 0.7475, person(0), 113, 252, 163, 401
-[2] conf, classID, x1, y1, x2, y2, : 0.5038, person(0), 266, 315, 348, 396
-[3] conf, classID, x1, y1, x2, y2, : 0.3259, person(0), 166, 333, 213, 400
+[Result] Detected 17 Boxes.
+[0] conf, classID, x1, y1, x2, y2, : 0.8743, person(0), 18, 476, 318, 1068
+[1] conf, classID, x1, y1, x2, y2, : 0.8742, person(0), 420, 528, 615, 1080
+[2] conf, classID, x1, y1, x2, y2, : 0.7187, person(0), 993, 753, 1308, 1068
+[3] conf, classID, x1, y1, x2, y2, : 0.6268, person(0), 622, 832, 795, 1080
+[4] conf, classID, x1, y1, x2, y2, : 0.5648, person(0), 300, 870, 487, 1076
+[5] conf, classID, x1, y1, x2, y2, : 0.5136, person(0), 1785, 791, 1920, 1076
+[6] conf, classID, x1, y1, x2, y2, : 0.5084, backpack(24), 802, 881, 903, 1061
+[7] conf, classID, x1, y1, x2, y2, : 0.4755, backpack(24), 1083, 963, 1290, 1080
+[8] conf, classID, x1, y1, x2, y2, : 0.4404, person(0), 1732, 723, 1841, 982
+[9] conf, classID, x1, y1, x2, y2, : 0.4285, person(0), 1657, 630, 1725, 855
+[10] conf, classID, x1, y1, x2, y2, : 0.4195, person(0), 1597, 750, 1710, 967
+[11] conf, classID, x1, y1, x2, y2, : 0.4082, person(0), 885, 787, 1012, 1061
+[12] conf, classID, x1, y1, x2, y2, : 0.4081, person(0), 236, 693, 393, 1057
+[13] conf, classID, x1, y1, x2, y2, : 0.4052, person(0), 1466, 742, 1578, 922
+[14] conf, classID, x1, y1, x2, y2, : 0.3827, person(0), 817, 570, 903, 840
+[15] conf, classID, x1, y1, x2, y2, : 0.3682, person(0), 1338, 791, 1511, 1076
+[16] conf, classID, x1, y1, x2, y2, : 0.3127, person(0), 690, 588, 746, 765
save file : yolov5s_3.jpg
dxrt inference Done!
decoding output Done!
[Result] Detected 1 Boxes.
-[0] conf, classID, x1, y1, x2, y2, : 0.7188, dog(16), 88, 104, 455, 409
+[0] conf, classID, x1, y1, x2, y2, : 0.8201, dog(16), 187, 91, 1073, 808
save file : yolov5s_4.jpg
dxrt inference Done!
decoding output Done!
[Result] Detected 4 Boxes.
-[0] conf, classID, x1, y1, x2, y2, : 0.8054, horse(17), 3, 212, 209, 359
-[1] conf, classID, x1, y1, x2, y2, : 0.6319, cow(19), 289, 226, 395, 316
-[2] conf, classID, x1, y1, x2, y2, : 0.5420, horse(17), 0, 208, 101, 273
-[3] conf, classID, x1, y1, x2, y2, : 0.5380, horse(17), 162, 217, 283, 329
+[0] conf, classID, x1, y1, x2, y2, : 0.8946, horse(17), 1, 190, 315, 409
+[1] conf, classID, x1, y1, x2, y2, : 0.7735, cow(19), 436, 212, 597, 346
+[2] conf, classID, x1, y1, x2, y2, : 0.7386, horse(17), 0, 184, 150, 280
+[3] conf, classID, x1, y1, x2, y2, : 0.7217, horse(17), 244, 197, 428, 366
save file : yolov5s_5.jpg
```
@@ -635,3 +533,44 @@ save file : yolov5s_5.jpg
yolov5s objection detection result
+
+## Build DX-APP Documentation
+
+:::tip
+For more details on how to use DX-APP, please build the full documentation and refer to it.
+:::
+
+### Install MkDocs
+
+
yolov5s objection detection result
**`default`** mode options: "NEAREST", "LINEAR"(default), "AREA", "CUBIC", "LANCZOS4"
**`torchvision`** mode options:
"NEAREST", "LANCZOS", "BILINEAR"(default), "BICUBIC", "BOX", "HAMMING" | `LINEAR`, `BILINEAR` | -| `centercrop` | `width`, `height` | Crop specified size from image center | `width: 224`, `height: 224` | -| `transpose` | `axis` | Dimension transformation order | `[2, 0, 1]` | -| `normalize (*)` | `mean`, `std` | Normalization processing, supports single value or array | `[0.5,0.5,0.5]` | -| `mul (*)` | `x` | Multiply all pixels by a value | `x: 255` | -| `add (*)` | `x` | Add a value to all pixels | `x: 0.1` | -| `subtract (*)` | `x` | Subtract a value from all pixels | `x: 0.1` | -| `div (*)` | `x` | Divide all pixels by a value | `x: 255.0` | -| `expandDim` | `axis` | Expand a dimension at specified position | `axis: 0` | +Input Image Preprocessing Configuration + +The preprocessing pipeline required before feeding images into the ONNX model. These configurations are used to preprocess the calibration dataset during compilation. +Items marked with (\*) will be executed on the NPU side to reduce CPU workload. Therefore, preprocessing steps moved to the NPU should be omitted in runtime software. If no preprocessing is required, you can skip this section. + +| Preprocessing | Parameters / Options | Description | Example | +| --------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | +| `convertColor` | `RGB2BGR`, `BGR2RGB`, `BGR2GRAY`, `BGR2YCrCb` | Convert color channel order | `"RGB2BGR"` | +| `resize` | `mode` | `default` (opencv) or `torchvision` | `"default"` / `"torchvision"` | +| | `width`, `height` | Target image size (default mode only) | `width: 224`, `height: 224` | +| | `size` | Target short-side size (torchvision mode only) | `size: 256` | +| | `interpolation` | Interpolation (optional)
Options for **`default`**: "NEAREST", "LINEAR"(default), "AREA", "CUBIC", "LANCZOS4"
Options for **`torchvision`**:
"NEAREST", "LANCZOS", "BILINEAR"(default), "BICUBIC", "BOX", "HAMMING" | `LINEAR`, `BILINEAR` | +| `centercrop` | `width`, `height` | Crop the given size from the image center | `width: 224`, `height: 224` | +| `transpose` | `axis` | Dimension order (permute) | `[2, 0, 1]` | +| `normalize (*)` | `mean`, `std` | Normalization; supports scalar or array | `[0.5,0.5,0.5]` | +| `mul (*)` | `x` | Multiply all pixels by a value | `x: 255` | +| `add (*)` | `x` | Add a value to all pixels | `x: 0.1` | +| `subtract (*)` | `x` | Subtract a value from all pixels | `x: 0.1` | +| `div (*)` | `x` | Divide all pixels by a value | `x: 255.0` | +| `expandDim` | `axis` | Expand one dimension at the specified axis | `axis: 0` | :::info -Items marked with (\*) will be executed on the NPU side +Items marked with (\*) will be executed on the NPU side. ::: #### Model Configuration Example @@ -212,31 +212,31 @@ Items marked with (\*) will be executed on the NPU side ] ``` -#### Accuracy Enhancement Scheme Configuration +#### Enhanced Accuracy Scheme Configuration -To further improve model accuracy, you can choose to use the accuracy enhancement scheme (Enhanced Scheme), with scheme names **DXQ-P0 ~ DXQ-P5**. +To further improve model accuracy, you can enable the Enhanced Scheme options named **DXQ-P0 \~ DXQ-P5**. :::tip -- Each enhancement scheme is independent, meaning multiple schemes can be enabled simultaneously. -- Using these schemes **does not always bring accuracy improvement**, it is recommended to evaluate the effect based on actual situations; -- Enabling these schemes in **CPU-only environments will significantly increase compilation time**. +- Each scheme is independent, so you can enable multiple schemes at the same time. +- These schemes **do not always improve accuracy**; evaluate the results based on your use case. +- Enabling these schemes in a **CPU-only environment can significantly increase compilation time**. ::: -| Scheme ID | Name | Parameter Description | -| ---------- | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **DXQ-P0** | Operation Fusion (Fusing) | `alpha`: Fusion ratio, value range `0 < alpha < 1.0`, **recommended value: 0.5** | -| **DXQ-P1** | Find Optimal Parameters | `true/false`, enabling will significantly increase CPU compilation time | -| **DXQ-P2** | Relax Calibration Range to Find Optimal Parameters | `alpha`: Minimum value ratio of calibration range, `0 < alpha < beta`, recommended value: **0.1**, `beta`: Maximum value ratio of calibration range, `alpha < beta < ∞`, recommended value: **1.0** | -| **DXQ-P3** | Parameter Fine-tuning (Method 1) | `num_samples`: Number of data samples used for calibration, `0 < num_samples < ∞`, **recommended value: 1024** | -| **DXQ-P4** | Parameter Fine-tuning (Method 2) | Same as DXQ-P3 | -| **DXQ-P5** | Parameter Fine-tuning (Method 3) | Same as DXQ-P3 | +| Scheme ID | Name | Parameter Description | +| ---------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| **DXQ-P0** | Fusing | `alpha`: fusing ratio, range `0 < alpha < 1.0`, **recommended: 0.5** | +| **DXQ-P1** | Search Best Parameters | `true/false`; enabling it significantly increases CPU compilation time | +| **DXQ-P2** | Relax Range Search | `alpha`: min ratio of calibration range, `0 < alpha < beta`, recommended: **0.1**; `beta`: max ratio, `alpha < beta < ∞`, recommended: **1.0** | +| **DXQ-P3** | Fine-tuning (Method 1) | `num_samples`: number of samples for calibration, `0 < num_samples < ∞`, **recommended: 1024** | +| **DXQ-P4** | Fine-tuning (Method 2) | Same as DXQ-P3 | +| **DXQ-P5** | Fine-tuning (Method 3) | Same as DXQ-P3 | :::tip -Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different fine-tuning methods**. +Although DXQ-P3, DXQ-P4, and DXQ-P5 share the same parameters, they use **different fine-tuning methods**. ::: -**Accuracy Enhancement Configuration Example** +**Enhanced scheme configuration example** ```json ''' @@ -263,7 +263,7 @@ Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different f } ``` -**Accuracy Enhancement Single Scheme Control Configuration Example** +**Enhanced scheme enable/disable example** ```json ''' @@ -278,18 +278,136 @@ Although DXQ-P3, DXQ-P4, DXQ-P5 have the same parameters, they use **different f ''' ``` -| Name | Compilation Speed | Accuracy Improvement Tendency | -| ------ | ----------------- | ----------------------------- | -| DXQ-P0 | Very Fast | Low | -| DXQ-P1 | Fast | Low-Medium | -| DXQ-P2 | Slower | Medium-High | -| DXQ-P3 | Very Slow | High | -| DXQ-P4 | Very Slow | High | -| DXQ-P5 | Very Slow | High | +| Name | Compile Speed | Accuracy Improvement Tendency | +| ------ | ------------- | ----------------------------- | +| DXQ-P0 | Very fast | Low | +| DXQ-P1 | Fast | Low-Medium | +| DXQ-P2 | Slow | Medium-High | +| DXQ-P3 | Very slow | High | +| DXQ-P4 | Very slow | High | +| DXQ-P5 | Very slow | High | + +### PPU Post-processing Acceleration + +PPU (Post-Processing Unit) provides hardware-accelerated post-processing for object detection models. If your target model uses a YOLO-family architecture, you can add PPU configuration parameters in the model compilation config file. + +#### Supported PPU Types + +There are two supported object detection model architectures for PPU: + +- **Type 0 (Anchor-Based YOLO)** + + Designed for anchor-based models, such as YOLOv3, YOLOv4, YOLOv5, and YOLOv7. + +- **Type 1 (Anchor-Free YOLO)** + + Designed for anchor-free models, such as YOLOX. + +#### Identify ONNX Nodes That Can Use PPU + +
+ 
+ Yolov5s PPU Conv node
+
+
+ ```json
+ '''
+ "ppu": {
+ "type":0,
+ "conf_thres":0.25,
+ "activation": "Sigmoid",
+ "num_classes": 80,
+ "layer": {
+ "Conv_196": {
+ "num_anchors": 3
+ },
+ "Conv_308": {
+ "num_anchors": 3
+ },
+ "Conv_420": {
+ "num_anchors": 3
+ }
+ }
+ },
+ '''
+ ```
+
+
+ Yolov5s PPU Conv node
+
+ 
+ Yolov5s PPU Conv node
+
+
+ ```json
+ '''
+ "ppu": {
+ "type":1,
+ "conf_thres":0.25,
+ "num_classes": 80,
+ "layer": [
+ {
+ "bbox": "Conv_261",
+ "obj_conf": "Conv_262",
+ "cls_conf": "Conv_254"
+ },
+ {
+ "bbox": "Conv_282",
+ "obj_conf": "Conv_283",
+ "cls_conf": "Conv_275"
+ },
+ {
+ "bbox": "Conv_303",
+ "obj_conf": "Conv_304",
+ "cls_conf": "Conv_296"
+ }
+ ]
+ },
+ '''
+ ```
+
+
+ Yolov5s PPU Conv node
+ **Note: fixed at compile time and cannot be changed at runtime.** | +| `num_classes` | int | Number of classes supported by the model. | +| `layer` | list | Detection head configuration list. Each element corresponds to one head and contains the following fields: | +| `layer.bbox` | string | Layer name that outputs **Bounding Box** coordinates. | +| `layer.obj_conf` | string | Layer name that outputs **Object Confidence**. | +| `layer.cls_conf` | string | Layer name that outputs **Class Confidence**. | + +### Run Model Compilation + +After finishing the model configuration file, compile the model using the following command.
@@ -18,28 +18,34 @@ Link: [**DX Model Zoo**](https://developer.deepx.ai/wp-content/modelzoo/model_zo
## Installation
-### Clone DX-ALL-SUITE Repository
+:::info
+DX-Model-Zoo is only supported on x86 Ubuntu 20.04 LTS or 22.04 LTS.
+:::
+
+### Clone the DX-ALL-SUITE Repository
:::tip
-Please follow [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite) to clone the specified version of the DX-ALL-SUITE repository
+Clone the specified DX-ALL-SUITE version following [DX-ALL-SUITE](./dx-sdk-introduction#dx-all-suite).
:::
### Install DXMZ (Deepx Model Zoo)
-Navigate to the `dx-all-suite/dx-modelzoo` directory
+#### Manual Installation
-