mipi-2

MIPI Connectivity for Imaging

High performance, low power

Overview

Embedded vision systems are the keystones to many advanced capabilities in modern applications including augmented vision, automotive, machine vision, medical, and Smart City cameras. Modern vision systems require high resolution, high frame rates, and increasing pixel depth, presenting significant data rate and memory bandwidth challenges. Xilinx offers both cost-optimized and high-performance MIPI-based solutions for camera sensor capture and display, supporting  D-PHY, CSI-2, and DSI protocols. Additionally, Xilinx offers Image Signal Processing IP required by many image sensor applications for color conversion, correction, balancing, and more. Every design has unique requirements, the flexibility of Xilinx devices enables developers to meet those needs as well as differentiate their solutions.

Xilinx MIPI IP Cores

input

CSI-2

Receiver and Transmitter Controller Subsystems

  • 1-4 PPI Lane Support
  • Multiple data type support (RAW, RGB, YUV)
  • AXI IIC support for CCI interface
  • Filtering based on Virtual Channel ID (VC)
  • Single, Dual, Quad pixel support output
  • AXI4-compliant interface with support for 4K resolution imagers and processors
  • Small resource count
output

DSI

Transmitter Controller Subsystem

  • 1-4 Lane Support
  • Supports all mandatory data types
  • Virtual channel (1 to 4) Programmable EoTp generation support
  • Low Power (LP) and Ultra Low Power (ULP) mode generation
  • Multilane interoperability
  • ECC generation for packet header
  • CRC generation for data bytes (can be optional)
  • Pixel byte conversion based on data format
  • AXI4-Lite interface to access more registers
  • AXI4-compliant for input video stream  

Xilinx devices support multiple MIPI configurations, from single to multiple links, different data rates, and data types, and can mix receive and transmit datapaths on a single device. Xilinx devices are also expandable, limited only by the number of I/Os. To further accelerate or streamline solutions, programmable logic enables integration of the entire image signal processing (ISP) chain, while remaining scalable and field upgradable as ISP algorithms improve.

MIPI Support across Xilinx Products

Cost-Optimized Portfolio High-Performance Devices
  • Low-cost 7 series devices that can implement D-PHY compatibility using a network of external resistors
  • D-PHY data rates up to 800Mb/s per lane up to 8MP image resolution
  • Higher data rates require an external D-PHY
  • Featured devices:
  • High-performance devices have built-in support for the D-PHY physical layer
    • Zynq UltraScale+™ MPSoC devices have data rates up to 2500Mb/s per lane up to 16MP image resolution
    • Versal™ ACAPs have data rates up to 3200Mb/s per lane up to 32MP image resolution
  • These devices also have dedicated engines for real-time processing, graphics processing, video encoding and decoding, waveform, packet processing, and Machine Learning /Artificial Intelligence engines
  • Featured devices:
Applications

Applications

Xilinx offers MIPI-based solutions for camera sensor capture and display through cost-optimized and high-performance devices that enable edge-type applications across a broad set of markets


Automotive

The Xilinx Automotive (XA) portfolio solves industry challenges within ADAS, Automated Driving, In-Vehicle Infotainment & Driver Information, and Electrification & Networking. Adaptable architecture allows customers to achieve low-latency compute and the ability to connect a wide array of interfaces – including MIPI. As enhancements to the MIPI protocol are released in the toolchain, Xilinx's flexible silicon can adapt – letting customers design their systems with the latest interfaces for the fastest data transmission possible.

automotive-tile

industrial-tile

Industrial

Smart Cities and Smart Retail require a complex mix of compute requirements spanning from the cloud to the edge. Xilinx Industrial Vision solutions deliver camera and image processing platforms with optimal performance, power, and cost – while also offering low latency and high reliability. Xilinx offerings include heterogeneous embedded processing, I/O flexibility, hardware-based deterministic control, and comprehensive solutions able to handle harsh environments over industrial life cycles. The Vitis™ unified software platform and Vitis™ AI development environment simplify development of accelerated applications and AI inference across Xilinx hardware platforms, enabling video security equipment manufacturers and AI developers to rapidly create scalable and differentiated products.


Consumer Electronics

Consumer products implement machine learning to improve autonomy, user interaction, and situational awareness. Traditional applications such as DSLR cameras, printers, and video conferencing are now being accompanied by camera-enabled robots and drones, TVs, virtual reality headsets, and smart appliances. Xilinx offers high-resolution MIPI interfaces along with machine learning, video compression, and a wide variety of AV interfaces to achieve adaptable, low-cost, single-chip solutions. This high level of integration is enabling the next wave of immersive consumer video and imaging products with increasing levels of autonomy and analytics.

consumer-tile

aerospace-tile

Aerospace & Defense

With 30+ continuous years of heritage and support, Xilinx delivers the broadest product portfolio for enabling the next generation of advancements in ground-based, airborne, and space systems. Commercial-grade and Defense-grade variants of Zynq® UltraScale+™ MPSoC devices satisfy requirements for MIPI sensor imaging and connectivity. Various platforms are available including the UAV and Robotics Platform, which is built on the ZU7EV device with support for dual 4Kp60 MIPI connectivity in a software programmable platform.


Broadcast & Pro A/V

Whether the application is conferencing and collaboration, digital cinema, or traditional broadcast, MIPI allows for the integration of the latest sensor technology for broadcast and professional video capture applications. MIPI IP, combined with any-to-any media connectivity, a Video Codec Unit (VCU), and 8K capable video processing elements, enables the latest developments in applications such as intelligent digital signage, 4K UHD broadcast, machine learning collaboration, and high quality, low bit rate live streaming.

broadcast-tile
Evaluation Kits

Evaluation Kits

Xilinx offers high-performance, low-power consumption MIPI-based solutions for camera sensor capture and display, supporting D-PHY, CSI-2, and DSI protocols.


Cost-Optimized Portfolio Platforms

A broad range of Industrial and Healthcare IoT applications can leverage MIPI support on our Cost-Optimized Portfolio platforms. These include vision-guided robotics, security cameras, Smart Cities, smart retail, endoscopy, and machine vision.  If you are looking to design with our Spartan-7 FPGA  or Zynq-7000 SoC families, start with these kits.

sprtan-board

Spartan-7 SP701 Evaluation Kit

The SP701 Evaluation Kit, equipped with the best-in-class performance-per-watt Spartan®-7 FPGA, is built for designs requiring sensor fusion such as industrial networking, embedded vision, and automotive applications. The SP701 features video interfaces including MIPI CSI, DSI, and HDMI. It also features high I/O availability and I/O expansion capability via Pmods and FMC connectors making it the largest IP development canvas for Spartan-7 FPGA users.
Learn more >

Featuring the Spartan-7 XC7S100 FPGA

System Logic Cells (K) 102
Memory 1,100Kb DRAM, 4,320Kb Block RAM
DSP Slices 160
Maximum I/O Pins 400
Additional Features 8 clock management tiles (1MMCM + 1 PLL)
1 analog to digital converter (XADC)
1 configuration AES / HMC Block
zybo-board

Zybo Z7: Zynq-7000 ARM/FPGA SoC Development Kit

The Zybo Z7 from Digilent is built around the Zynq-7000 device, integrating dual Arm® Cortex®-A9 processors with 7 series programmable logic. The Zybo Z7 kit offers a video-capable feature set that includes a MIPI CSI-2 compatible Pcam connector, HDMI input and output, high DDR3L bandwidth, and other multimedia and connectivity peripherals. This feature set provides an affordable solution for complex embedded vision applications.
Learn More >

Featuring the Zynq-7000 XCZ7010 SoC or XC7Z020 SoC

Device XCZ7010 XC7Z7020
System Logic Cells (K) 28 85
Memory 1,024MB DRAM
2.1Mb Block RAM
1,024MB DRAM
4.9Mb Block RAM
DSP Slices 80 220
Maximum I/O Pins 100 125
Additional Features 2 clock management tiles (1MMCM + 1 PLL) 4 clock management tiles (1MMCM+1PLL)
1 analog to digital converter (XADC) 1 analog to digital converter (XADC)
1 AES and SHA block 1 AES and SHA block

High-Performance Device Platforms

For AI or video focused industrial, medical imaging, endoscopy, healthcare edge AI and more, our High-Performance device platforms open up added performance and features.  If your application includes a need for video encoding or machine learning, check out our high-performance line based on the Zynq UltraScale+ MPSoCs.

mpsoc-board

Zynq UltraScale+ MPSoC ZCU104 Evaluation Kit

The ZCU104 Evaluation Kit enables designers to jumpstart designs for embedded vision applications such as surveillance, Advanced Driver Assisted Systems (ADAS), machine vision, Augmented Reality (AR), drones, and medical imaging. This kit features a Zynq® UltraScale+™ MPSoC EV device with integrated video codec and supports many common peripherals and interfaces for embedded vision applications. The included ZU7EV device is equipped with a quad-core Arm® Cortex®-A53 applications processor, dual-core Cortex-R5F real-time processor, Mali™-400 MP2 graphic processing unit, 4KP60 capable H.264/H.265 video codec, and 16nm FinFET+ programmable logic.
Learn More >

Featuring the Zynq UltraScale+ XCZU7EV-2FFVC1156 MPSoC

System Logic Cells (K) 504
Memory 38 Mb
DSP Slices 1,728
Video Codec Unit
1
Maximum I/O Pins
464
ultra96-board

Ultra96-V2 Zynq UltraScale+ ZU3EG

Ultra96-V2, is an Arm-based development board built around the Zynq UltraScale+™ MPSoC. Available in commercial and industrial temperature grades, Ultra96-V2 has been designed with a certified radio module from Microchip. Additional power control and monitoring is possible with the included Infineon Pmics. Engineers have options of connecting to Ultra96-V2 through a Webserver using integrated wireless access point capability or to use the provided Linux Matchbox windows environment that can be viewed on the integrated Mini DisplayPort video output. Multiple application examples and onboard development options are provided as examples.
Learn More >

Featuring the Zynq UltraScale+ MPSoC ZU3EG A484

System Logic Cells (K) 154
Memory 7.6 Mb
DSP Slices 360
Video Codec Unit
None
Maximum I/O Pins
252

Targeted Platform

Platform designed specifically to meet the extreme vision and sensor processing requirements of UAV and Robotics development.

uav-board

Autonomous Control & Robotics

The UAV and Robotics Platform (URP) meets the typical processing requirements encountered when designing robotics and UAV applications. It is a Zynq UltraScale+ MPSoC based platform, providing a broad spectrum of processing capabilities. The small form factor board integrates sensors-on-board, software, and FPGA logic. With its motor control units, position sensors, obstacle detection interfaces, and communication resources, the URP can be used for many applications.
Learn More >

Key Features and Benefits

  • Fully integrated high-performance navigation sensors
  • Multi-camera and sensor fusion
  • High-bandwidth connectivity
  • Onboard hosting of autopilot on real-time processing cores
  • Reliable high-speed motor control solution
  • High-performance infrastructure for video processing
  • High-level-synthesis supports rapid algorithm development and implementation
  • H.265 codec VCU supporting 4K60fps
  • Dimensions: 135 mm x 68.4 mm x 10 mm
  • Onboard high-efficiency power supplies
  • Support for NVMe SSD, PCIe, Gigabit Ethernet, DisplayPort, HDMI
  • Advanced debug support
  • Industrial temperature (-40°C +85°C)

What’s Included

  • Dual 4K camera interfaces
  • High-performance DDR4 memory 64bits + 8bits ECC
  • Navigation sensors
  • Positional sensors
  • Quad motion control interfaces
  • Support for NVMe SSD, PCIe, Ethernet, DisplayPort, HDMI
System Logic Cells (K) 504
Memory 38 Kb
DSP Slices 1.728
Video Codec Unit 1
Maximum I/O Pins 464
Documentation

Documentation

Default Default 标题 文件类型 日期
Get Started

Get Started

Design Example Description Device Support

Enlarge

Spartan-7 SP701 Application Example
This example showcases a CSI-2 Subsystem IP with a PCAM camera module (Digilent), which is a popular interface used by MIPI CSI camera modules, and a DSI Subsystem IP with a display. These are both controlled by a MicroBlaze™ soft processor and all the IP is integrated in Vivado®. The D-PHY is implemented on the SP701 evaluation kit using passive components.
Spartan-7

Enlarge

Zynq UltraScale+ MPSoC ZCU102 Application Example
This design demonstrates the use of the MIPI CSI-2 RX (decodes and processes video data) and MIPI DSI TX subsystems on the ZCU102 board. The system receives images captured by the IMX274 image sensor. Processed images are then displayed on either an HDMI monitor or a MIPI DSI display.
Zynq UltraScale+ MPSoC

Adaptable Advantage Blog