Overview
Better Stack tests OpenCV 5's brand-new DNN engine — the first major release in 8 years. The headline: ONNX operator coverage jumped from 22% to 80%, enabling modern AI models (YOLO, stable diffusion, vision language models) to run natively on CPU with zero extra dependencies. Four hands-on demos show colorization, real-time object detection, latent diffusion inpainting, and VLM image captioning — all running inside plain OpenCV with no PyTorch, no ONNX Runtime, and no GPU.
1 Why This Update Is a Big Deal
OpenCV is everywhere: 86K+ GitHub stars, 1M+ installs per day, the foundation for robotics, AR, medical imaging, and industrial inspection for two decades. Everyone has been building on the same version 4 line for the last 8 years.
2 The New DNN Engine Explained
The headline feature: a complete rewrite of the DNN (Deep Neural Network) module (the part that runs neural networks).
What Is ONNX?
ONNX (Open Neural Network Exchange) is a universal format for AI models — think of it as the "PDF of machine learning." When you train a model in PyTorch, TensorFlow, or any other framework, you can export it to ONNX format so it can run anywhere else — on a different framework, a different device, or a different operating system — without needing the original training environment.
ONNX defines a standard set of operators (mathematical operations like convolution, matrix multiply, attention, normalization, etc.) that make up neural networks. The more operators a runtime supports, the more models it can run. An "ONNX operator" is essentially one building block of a neural network — modern architectures like transformers use dozens of these blocks together.
The Key Number
- OpenCV 4: supported only ~22% of ONNX operators → grab a modern model, try to load it, hit a wall because some operator wasn't supported
- OpenCV 5: supports 80% of ONNX operators → most models run out of the box, including modern transformers that were impossible before
Why the Jump?
- Old engine: processed networks layer by layer
- New engine: built around a typed operation graph — looks at the whole network as a graph first, does shape inference, constant folding, and operator fusion before executing
- In plain terms: it understands the entire model before running it
- Can now handle dynamic shapes and modern transformer architectures that the old engine couldn't
Other Improvements
- Native FP16 and BF16 data types
- Real n-dimensional array support in
cv::Mat - Python-first core
- Legacy C API removed — C++17 baseline
3 Faster Than ONNX Runtime?
OpenCV benchmarked the new engine against Microsoft's ONNX Runtime on CPU:
- YOLOv8: 11.5% faster than ONNX Runtime
- OWLv2: almost 37% faster
- X-FEaT: 30% faster
4 One Catch: CPU Only (For Now)
Important caveat: the new engine is CPU only at launch. GPU support is coming later in the v5 cycle. If you need GPU inference today, fall back to the classic engine (which still has CUDA and OpenVINO support).
Everything demonstrated in the video runs entirely on CPU.
5 Demo 1: Image Colorization
Taking a black-and-white frame from Spider Noir and running OpenCV's colorization example:
- Speed: extremely fast — near-instant processing
- Quality: not perfect (older colorization model with muted tones), but sky was correctly colored
- The point: this image was produced using the native OpenCV neural network with zero extra dependencies
6 Demo 2: Real-Time Object Detection
Running YOLO object detection on a Spider Noir video clip:
- Real-time detection running on CPU through the new DNN engine
- This is the model type where OpenCV is faster than ONNX Runtime
- No PyTorch, no separate runtime, no GPU required — all plain OpenCV
7 Demo 3: Latent Diffusion Inpainting
OpenCV 5 ships a latent diffusion inpainting example — paint over something in an image and the diffusion model fills in the blank:
- Slower than classic inpainting — classic uses a single forward pass (instant), LDM uses iterative denoising (multiple steps)
- Even slower on CPU — diffusion is better suited for GPU
- Result: decent job with visible diffusion artifacts — solvable by increasing step count or using a different model
The significance: stable diffusion running natively inside OpenCV — no external framework needed.
8 Demo 4: Vision Language Model Captioning
Using PaliGemma for image captioning natively inside OpenCV:
- Painfully slow on CPU — definitely not practical for production use
- Better alternatives exist (e.g., Gemma 4 12B on OMLX is 100× faster)
- The real point: OpenCV now has all the pieces for LLM inference — tokenizer, attention layers, KV cache — all built in
9 Final Thoughts
One library, running on CPU, zero extra dependencies — used for colorization, real-time object detection, diffusion inpainting, and image captioning.
- You still train models on PyTorch — that's not OpenCV's job
- But for running models inside a vision pipeline, OpenCV 5 is a massive leap
- Once GPU support ships, it will be even more compelling
🎯 Key Takeaways
🔑 Key Takeaways
- First major release in 8 years — OpenCV 5 is a genuine milestone for the most widely-used computer vision library
- ONNX coverage jumped from 22% to 80% — most modern models now run out of the box
- Graph-based DNN engine — understands the entire model before executing, handles dynamic shapes and transformers
- Matches or beats ONNX Runtime on CPU — 11-37% faster on tested models (self-reported)
- CPU only for now — GPU support coming later in the v5 cycle
- Zero-dependency inference — no PyTorch, no ONNX Runtime, no separate frameworks needed
- YOLO runs in real-time on CPU — practical for production deployments
- Stable diffusion works natively — slow on CPU but proves the engine can handle iterative models
- VLM infrastructure built in — tokenizer, attention, KV cache signal where OpenCV is heading
- C++17 baseline, Python-first core — legacy C API removed, modern data types (FP16, BF16)
🔗 Resources & Links
- OpenCV 5 Official Page — release announcement and overview
- OpenCV GitHub Repository — source code and examples