AMD Vitis™ Video Analytics SDK

Turning vision into powerful insights

Overview

The Vitis™ Video Analytics SDK (VVAS) is a complete software stack to build AI-powered intelligent video analytics solutions on AMD platforms. It takes input from USB/MIPI cameras, video from files, or streams over RTSP, and uses Vitis AI to generate insights from pixels for various use cases such as understanding traffic and pedestrians in smart cities, health and safety monitoring in hospitals, self-checkout, and analytics in retail, detecting component defects at a manufacturing facility, and others.

The VVAS SDK is based on the popular Gstreamer framework, and the core SDK consists of several hardware accelerator plug-ins that use various accelerators such as video encoder, decoder, multiscaler (for resize and color space conversion), deep learning processing unit (DPU) for AI inference, etc. By performing all the compute-heavy operations in dedicated accelerators, it can achieve the highest performance for video analytics applications. It offers extensive AI models support for popular object detection, segmentation, and classification tasks.

For the developer community, Vitis Video also provides a framework in the form of generic Infrastructure plugins, software acceleration libraries, and a simplified interface for users to develop their own acceleration library to control a custom hardware accelerator. With this framework, users can easily integrate their custom accelerators/kernels into Vitis Video Analytics SDK. It builds on top of Xilinx Run Time (XRT), Vitis, and Vitis AI and abstracts these complex interfaces, making it easier for developers to build video analytics applications.

The Vitis Video Analytics SDK supports deployment on Zynq™ UltraScale+™ MPSoC-based embedded platforms such as Kria™ SoM and ZCU104 evaluation kit, as well as larger edge or data center platforms.

Benefits

VVAS is the complete software stack to build and deploy AI-based intelligent video analytics applications. 

Graph Architecture

The Vitis Video Analytics SDK is an optimized graph architecture built using the open-source GStreamer framework. The graph below shows a typical video analytic application starting from input video to output metadata. All the individual blocks are various plug-ins that are used. At the bottom are the different hardware engines used throughout the application. Optimum memory management with zero-memory copy between plug-ins and the use of various accelerators ensure the highest performance.

Custom Plug-ins
Highly optimized GStreamer plug-ins developed to provide very specific functionality using optimized kernels and Ips on AMD platforms.

Infrastructure Plug-ins
These are generic infrastructure Gstreamer plug-ins being developed to help users integrate their kernels into the Gstreamer framework.

Acceleration Software libs
These are optimized acceleration s/w libs developed to manage the state machine of the acceleration kernels/Ips and expose the interface so that these Acceleration s/w libs can be hooked into VVAS generic infrastructure plug-ins. These can be used as a reference to develop a new acceleration s/w lib based on the VVAS framework.

Acceleration Hardware (Kernels/Ips)
These are highly optimized kernels being developed by AMD.

Reference Platforms and Applications
VVAS provides several reference platforms catering to different applications/solutions needs.

Development Flows

1. Download prebuilt images of Multichannel ML and Smart Model Select applications
2. Exercise the existing functionalities/pipelines of tutorial and applications
   
3. Develop Vitis™ Video Analytics SDK acceleration software library for your custom logic/kernel and integrate it with Vitis Video Analytics SDK infrastructure plugin and verify the functional correctness.
   
4. Vitis Video Analytics SDK ships 16 AI models with the current release. If you would like to use a different AI model, please either leverage your own model or download it from AMD Model Zoo and compile it using Vitis AI
   
5. Run your pipeline/application with a newly built AI model.

Deployment Options

Get Started

Partner Solutions