README.md

libtensorrt - A C++ library

NOTE:
This package is not open source and does not contain any source code. Instead, in order to "build" the exported target(s) it downloads (potentially large) pre-built binaries provided by NVIDIA for the target platform.

CI for this package is disabled due to the above.
Supported platforms/compilers are Windows/MSVC and Linux/GCC or Clang (x86_64).

NVIDIA® TensorRT™ is a C++ library that facilitates high-performance inference on NVIDIA GPUs. TensorRT takes a trained network, which consists of a network definition and a set of trained parameters, and produces a highly optimized runtime engine that performs inference for that network. TensorRT provides APIs using C++ and Python that help to express deep learning models using the Network Definition API or load a pre-defined model using the parsers that allow TensorRT to optimize and run them on an NVIDIA GPU. TensorRT applies graph optimizations, layer fusion, among other optimizations, while also finding the fastest implementation of that model leveraging a diverse collection of highly optimized kernels. TensorRT also supplies a runtime that you can use to execute this network on NVIDIA’s GPUs.

For more information about TensorRT, visit https://developer.nvidia.com/tensorrt.

Usage

To start using libtensorrt in your project, add the following depends value to your manifest, adjusting the version constraint as appropriate:

depends: libtensorrt ^10.16.1

Then import the library in your buildfile:

import libs = libtensorrt%lib{nvinfer}

Importable targets

This package provides the following importable targets:

lib{nvinfer}            # core builder and runtime (main TensorRT API)
lib{nvinfer_plugin}     # standard built-in op plugins (batch norm, RNN cells, etc.)
lib{nvinfer_lean}       # runtime-only nvinfer without the builder, smaller footprint
lib{nvinfer_dispatch}   # forward-compatible dispatch shim, delegates to the installed lean runtime
lib{nvinfer_vc_plugin}  # version-compatible plugin library, pairs with nvinfer_dispatch
lib{nvonnxparser}       # ONNX parser, converts .onnx graphs into TensorRT networks
lib{tensorrt_shim}      # capture shim, records engine-build API calls for replay

lib{tensorrt_shim} is only available on Linux.

It also provides the following importable executables:

exe{trtexec}          # command line engine builder and inference benchmark
exe{tensorrt_player}  # rebuilds an engine from captured TensorRT API calls

exe{tensorrt_player} is only available on Linux.

Configuration variables

This package provides the following configuration variables:

[dir_path] config.libtensorrt.cache ?= $out_root

The directory used to cache downloaded binary archives between builds. If config.tensorrt.cache is set (a shared project-wide cache directory), it takes precedence over the per-package default of $out_root.