Vision processing unit
A vision processing unit (VPU) is (as of 2023) an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.[1][2]
Overview
Vision processing units are distinct from graphics processing units (which are specialised for video encoding and decoding) in their suitability for running machine vision algorithms such as CNN (convolutional neural networks), SIFT (scale-invariant feature transform) and similar.
They may include
image processing
.
Contrast with GPUs
They are distinct from
frame buffers, with random access patterns
). VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.
Target markets are
mobile devices
.
Examples
- Movidius Myriad X, which is the third-generation vision processing unit in the Myriad VPU line from Intel Corporation.[3]
- Pixel Visual Core (PVC), which is a fully programmable Image, Vision and AI processor for mobile devices
- Microsoft HoloLens, which includes an accelerator referred to as a holographic processing unit (complementary to its CPU and GPU), aimed at interpreting camera inputs, to accelerate environment tracking and vision for augmented reality applications.[6]
- MIT intended for running convolutional neural networks.[7]
- convolutions, using a dataflow architecture.
- Mobileye EyeQ, by Mobileye
- Programmable Vision Accelerator (PVA), a 7-way VLIW Vision Processor designed by Nvidia.
Broader category
Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of
AI accelerators
(to which VPUs may also belong), however as of 2016 there is no consensus on the name:
- neuromorphic processor aimed at similar sensor data pattern recognitionand intelligence tasks, including video/audio.
- Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.[8]
- All models of Intel
See also
- Adapteva Epiphany, a manycore processor with similar emphasis on on-chip dataflow, focussed on 32-bit floating point performance
- CELL, a multicore processor with features fairly consistent with vision processing units (SIMD instructions & datatypes suitable for video, and on-chip DMA between scratchpad memories)
- Coprocessor
- Graphics processing unit, also commonly used to run vision algorithms. NVidia's Pascal architecture includes FP16 support, to provide a better precision/cost tradeoff for AI workloads
- MPSoC
- OpenCL
- OpenVX
- Physics processing unit, a past attempt to complement the CPU and GPU with a high throughput accelerator
- Tensor Processing Unit, a chip used internally by Google for accelerating AI calculations
References
- ^ Seth Colaner; Matthew Humrick (January 3, 2016). "A third type of processor for AR/VR: Movidius' Myriad 2 VPU". Tom's Hardware.
- ^ Prasid Banerje (March 28, 2016). "The rise of VPUs: Giving Eyes to Machines". Digit.in.
- ^ "Intel® Movidius™ Vision Processing Units (VPUs)". Intel.
- ^ Weckler, Adrian. "Dublin tech firm Movidius to power Google's new virtual reality headset". Independent.ie. Retrieved 15 March 2016.
- ^ "DJI Brings Two New Flagship Drones to Lineup Featuring Myriad 2 VPUs - Machine Vision Technology - Movidius". www.movidius.com.
- ^ Fred O'Connor (May 1, 2015). "Microsoft dives deeper into HoloLens details: 'Holographic processor' role revealed". PCWorld.
- ^ Chen, Yu-Hsin; Krishna, Tushar; Emer, Joel & Sze, Vivienne (2016). "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks". IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers. pp. 262–263.
- ^ "Introducing Qualcomm Zeroth Processors: Brain-Inspired Computing". Qualcomm. October 10, 2013.
- ^ "Intel to Bring a 'VPU' Processor Unit to 14th Gen Meteor Lake Chips". PCMAG.