TensorRT Samples

Contents

1. "Hello World" Samples

Sample	Language	Format	Description
sampleOnnxMNIST	C++	ONNX	“Hello World” For TensorRT With ONNX
network_api_pytorch_mnist	Python	INetwork	“Hello World” For TensorRT Using Pytorch

2. TensorRT API Samples

Sample	Language	Format	Description
sampleCudla	C++	INetwork	Using The CuDLA API To Run A TensorRT Engine (aarch64 only)
sampleDynamicReshape	C++	ONNX	Digit Recognition With Dynamic Shapes In TensorRT
sampleEditableTimingCache	C++	INetwork	Create a deterministic build using editable timing cache
sampleNamedDimensions	C++	ONNX	Working with named input dimensions
sampleNonZeroPlugin	C++	INetwork	Adding plugin with data-dependent output shapes
sampleIOFormats	C++	ONNX	Specifying TensorRT I/O Formats
sampleProgressMonitor	C++	ONNX	Progress Monitor API usage
engine_refit_onnx_bidaf	Python	ONNX	refitting an engine built from an ONNX model via parsers.
introductory_parser_samples	Python	ONNX	Introduction To Importing Models Using TensorRT Parsers
onnx_packnet	Python	ONNX	TensorRT Inference Of ONNX Models With Custom Layers
simpleProgressMonitor	Python	ONNX	Progress Monitor API usage
python_plugin	Python	INetwork/ONNX	Python-based TRT plugins
non_zero_plugin	Python	INetwork/ONNX	Python-based TRT plugin for NonZero op
sample_plugin_v2_to_v3_migration	Python	INetwork	Migrating a custom plugin from IPluginV2DynamicExt to IPluginV3
cute_dsl_plugin	Python	INetwork	Python-based TRT plugin for RMSNorm with a CuteDSL kernel
attention_mdtrt	Python	ONNX	Multi-device attention inference with MPI and NCCL

3. Safety Samples

Sample	Language	Format	Description
sampleSafeMNIST	C++	ONNX	Build a Safety Engine for MNIST
sampleSafePluginV3	C++	ONNX	Use Safety-Supported Plugins With Safety Engines

Preparing sample data

Many samples require the TensorRT sample data package. If not already mounted under /usr/src/tensorrt/data (NVIDIA NGC containers), download and extract it:

1. Download the current TensorRT sample data package. Sample data is updated only when needed, so the package may be hosted under an earlier TensorRT release.

   wget https://github.com/NVIDIA/TensorRT/releases/download/v11.0/tensorrt_sample_data_20260602.zip

2. Extract and set up the data:

   unzip tensorrt_sample_data_*.zip
   mkdir -p /usr/src/tensorrt/data
   cp -r tensorrt_sample_data_*/* /usr/src/tensorrt/data/
   export TRT_DATADIR=/usr/src/tensorrt/data

After extraction, the data directory structure should be:

$TRT_DATADIR/
├── int8_api/
├── mnist/
└── resnet50/