Tutorial: Real-Time Object Detection with DeepStream on Nvidia Jetson AGX Orin

Last month, Nvidia unleashed the next-generation edge computing hardware device branded as Jetson AGX Orin at GTC. Courtesy of Nvidia, I was fortunate enough to get a Jetson AGX Orin Developer Kit to evaluate and experiment with it.

The Jetson AGX Orin Developer Kit has everything you need to run AI inference at the edge with ultra-low latency and high throughput. As a successor to the most powerful Jetson AGX Xavier, AGX Orin packs a punch.

Below are the specifications of the Jetson AGX Orin compute module:

👁 Image

The developer kit comes with a carrier board that makes it easy to connect various peripherals.

👁 Image

The Jetson AGX Orin Developer Kit comes with a preview of JetPack SDK 5.0, which is based on the Ubuntu 20.04 root filesystem and Linux Kernel 5.10. It comes preloaded with CUDA 11.4, cuDNN 8.3.2, TensorRT 8.4.0, and DeepStream 6.0.

This tutorial will walk you through the steps involved in performing real-time object detection with DeepStream SDK running on Jetson AGX Orin.

Step 1 – Install TensorFlow on JetPack 5.0

Since we use a pre-trained TensorFlow model, let’s get the runtime installed.

sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v50 'tensorflow<2'

This installs TensorFlow 1.15 which works best for this tutorial. The --extra-index-url points to a wheel optimized for JetPack 5.0.

👁 Image

Check if the GPU is accessible from TensorFlow.

👁 Image

Step 2 – Download Pre-trained TensorFlow Inception Model

We will use the TensorFlow Inception V2 model trained on the COCO dataset.

wget -qO- http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2017_11_17.tar.gz | tar xvz -C /tmp

Step 3 – Converting the TensorFlow Model to TensorRT

This step optimizes the model by converting the TensorFlow frozen model to a serialized TensorRT UFF MetaGraph model.

python3 /usr/lib/python3.8/dist-packages/uff/bin/convert_to_uff.py \ /tmp/ssd_inception_v2_coco_2017_11_17/frozen_inference_graph.pb -O NMS \
-p /usr/src/tensorrt/samples/sampleUffSSD/config.py \
-o /tmp/sample_ssd_relu6.uff

Step 4 – Compiling the Custom Object Detector Sample Application

DeepStream SDK comes with a sample application that can be integrated with various models. We will now compile the application to use our Inception V2 model.

cd /opt/nvidia/deepstream/deepstream/sources/objectDetector_SSD
export CUDA_VER=11.4
export LD_LIBRARY_PATH=/usr/local/cuda
sudo -E make -C nvdsinfer_custom_impl_ssd

👁 Image

Step 5 – Edit DeepStream Application Configuration File

Open /opt/nvidia/deepstream/deepstream/sources/objectDetector_SSD/deepstream_app_config_ssd.txt and replace the [source0] section with the below contents:

[source0]
enable=1
type=1
camera-width=640
camera-height=480
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=0

This essentially configures the USB webcam as the input source. Once you connect the camera, make sure it’s visible to JetPack.

ls -la /dev/video*

👁 Image

Step 6 – Run the DeepStream Inference Pipeline

Finally, run the inference to perform object detection.

deepstream-app -c deepstream_app_config_ssd_USB.txt

👁 Image

We can increase the inference performance and the number of frames per second by switching to the inbuilt Camera Serial Interface (CSI) of the Jetson AGX Orin Developer Kit.