文章目录
YOLOX-TensorRT in Python
基础环境
# 切换 我的 Cuda为 cuda_11
. .bashrcCuda11
# 查看 cuda 版本
nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
..
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0
conda activate tf25
pip install pycocotools
# nvidia-tensorrt 版本需要是 7.*
pip install nvidia-tensorrt==7.2.* --index-url https://pypi.ngc.nvidia.com
YOLOX-TensorRT in Python – 参考教程
torch2trt 安装
git clone https://github.com/NVIDIA-AI-IOT/torch2trt
cd torch2trt
python setup.py install
模型转换【yolox_s.pth 转 tensorRT】
Cuda10.0 和 Cuda_11.2 均转换成功
cd yoloDir/YOLOX
python tools/trt.py -n yolox-s -c preModels/yolox_s.pth
- 正确运行输出如下
2021-08-27 04:09:41.895 | INFO | __main__:main:57 - loaded checkpoint done.
[TensorRT] WARNING: Tensor DataType is determined at build time for tensors not marked as input or output.
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 1 output network tensors.
2021-08-27 04:17:33.106 | INFO | __main__:main:70 - Converted TensorRT model done.
2021-08-27 04:17:33.286 | INFO | __main__:main:78 - Converted TensorRT model engine file is saved for C++ inference.
# 转换过程中，我的显卡占用
`0 N/A N/A 7491 C python 6491MiB `
10分钟左右，模型转换才完成
转换得到的 tensorRT 模型默认路径如下
ll YOLOX_outputs/yolox_s
total 52384
22082073 Aug 27 04:31 model_trt.engine
31539695 Aug 27 04:31 model_trt.pth
报错问题记录
python tools/trt.py -n yolox-s -c preModels/yolox_s.pth运行遇到如下报错
AttributeError: 'tensorrt.tensorrt.Builder' object has no attribute 'max_workspace_size'
tensorRT 模型 Python 推理测试
python tools/demo.py image -n yolox-s --trt --save_result
# 或者
python tools/demo.py image -f exps/default/yolox_s.py --trt --save_result
输出如下
2021-08-27 06:23:15.638 | INFO | __main__:main:249 - Args: Namespace(camid=0, ckpt=None, conf=0.3, demo='image', device='gpu', exp_file=None, experiment_name='yolox_s', fp16=False, fuse=False, legacy=False, name='yolox-s', nms=0.3, path='./assets/dog.jpg', save_result=True, trt=True, tsize=None)
2021-08-27 06:23:15.912 | INFO | __main__:main:259 - Model Summary: Params: 8.97M, Gflops: 26.81
loading checkpoint , trt_file = ./YOLOX_outputs/yolox_s/model_trt.pth
2021-08-27 06:23:18.343 | INFO | __main__:main:289 - Using TensorRT to inference
2021-08-27 06:23:20.761 | INFO | __main__:inference:159 - Infer time: 0.0064s
2021-08-27 06:23:20.763 | INFO | __main__:image_demo:196 - Saving detection result in ./YOLOX_outputs/yolox_s/vis_res/2021_08_27_06_23_20/dog.jpg
YOLOX-TensorRT in C++
参考链接如下：
https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/TensorRT/cpp
Step 1: Prepare serialized engine file
上面 模型转换【yolox_s.pth 转 tensorRT】 步骤已经给出
Step 2: build the demo【 tensorRT 模型 C++ 推理测试】
vim demo/TensorRT/cpp/CMakeLists.txt
make -j可能遇到的报错
遇到如下类似报错，是因为 Cuda 和 tensorRT 没有正确配置
解决方法，正确配置 Cuda 和 tensorRT ，cmake .. make -j是可以顺利执行的
In file included from /home/xx/project/project21Next/yoloDir/YOLOX/demo/TensorRT/cpp/yolox.cpp:11:0:
/home/xx/project/project21Next/yoloDir/YOLOX/demo/TensorRT/cpp/logging.h:239:10: error: looser throw specifier for ‘virtual void Logger::log(nvinfer1::ILogger::Severity, const char*)’
void log(Severity severity, const char* msg) override
# 再或者
/usr/bin/ld: warning: libnvrtc.so.11.1, needed by /home/moli/project/project21/modelTrans/tensorRT/tensorRT7/TensorRT-7.2.3.4/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
/home/moli/project/project21/modelTrans/tensorRT/tensorRT7/TensorRT-7.2.3.4/lib/libnvinfer.so: undefined reference to `nvrtcDestroyProgram@libnvrtc.so.11.1'
make -j 顺利执行之后生成得到 可执行文件yolox* ，我们需要
ldd yolox检查该文件依赖
可以发现类似依赖缺失如下
libcublas.so.11 => not found
libcudnn.so.8 => not found
libcublas.so.11 => not found
libcublasLt.so.11 => not found
libnvrtc.so.11.1 => not found
解决思路，就是找到 这些 so 库存在的位置，加入 LD_LIBRARY_PATH 变量中
export LD_LIBRARY_PATH=/home/墨理/usr/mycuda/targets/x86_64-linux/lib:/home/墨理/anaconda3/envs/SampleSolution/lib:$LD_LIBARARY_PATH
./yolox ../model_trt.engine -i ../../../../assets/dog.jpg正确 运行输出如下：
blob image
6ms
num of boxes before nms: 44
num of boxes: 4
1 = 0.95411 at 124.35 119.06 436.04 x 302.04
16 = 0.91279 at 134.33 223.34 175.87 x 326.01
7 = 0.61163 at 463.02 76.87 231.18 x 94.88
58 = 0.43789 at 684.24 110.90 32.13 x 42.94
# 默认保存到当前目录
save vis file
# vim yolox.cpp draw_objects()方法中，把弹框修改为了保存图片
cv::imwrite("det_res.jpg", image);
fprintf(stderr, "save vis file\n");
/* cv::imshow("image", image); */
/* cv::waitKey(0); */
我这边编译成功的正确版本配置如下
cuda_11.2.r11.2 + TensorRT-7.2.3.4
# OpenCV
set(OpenCV_DIR /home/墨理/project/project21Next/modelTrans/openCV/opencv-4.5.3/build)
find_package(OpenCV)
# cuda 自己安装的 cuda_11.2
include_directories(/home/墨理/usr/mycuda/include)
link_directories(/home/墨理/usr/mycuda/lib64)
# tensorRT
include_directories(/home/墨理/project/project21/modelTrans/tensorRT/tensorRT7/TensorRT-7.2.3.4/include/)
link_directories(/home/墨理/project/project21/modelTrans/tensorRT/tensorRT7/TensorRT-7.2.3.4/lib/)
# cudnn 配置看情况即可
因此整个过程需要，依次执行的命令如下
cd YOLOX/demo/TensorRT/cpp
vim CMakeLists.txt
mkdir build
cd build
cmake ..
make -j
# 检查依赖是否缺失
ldd yolox
export LD_LIBRARY_PATH=/home/moli/usr/mycuda/targets/x86_64-linux/lib:/home/moli/anaconda3/envs/SampleSolution/lib:$LD_LIBARARY_PATH
./yolox ../model_trt.engine -i ../../../../assets/dog.jpg
溜了溜了
