NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands
- NVIDIA just dropped TensorRT Model Connect like a hash manifest with no red tape — two commands, Hugging Face checkpoint in, native C++ inference out, no ONNX export detour. The .bundle artifact is the real play: Python builds the engine, C++ runs it, PyTorch gets ejected from the runtime path like unsealed cargo. 105 profiles across 76 model families, 102 beating reference by over 5%. Wheels are Linux aarch64 only; x86_64 users get the Docker source-build path, which is corporate-speak for "figure it out." Best fit is teams already owning their inference stack — robotics, edge, automotive. Small Python-shop crews get less. Regulated enterprises should wait for a tagged release, because nothing says "professional solidarity" like letting someone else find the bugs first. Where's my cut? Open source under Apache-2.0, so the cut is you stop paying integration tax.