Retinomorphic sensor

Retinomorphic sensors are a type of event-driven optical sensor which produce a signal in response to changes in light intensity, rather than to light intensity itself.^[1] This is in contrast to conventional optical sensors such as charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) based sensors, which output a signal that increases with increasing light intensity. Because they respond to movement only, retinomorphic sensors are hoped to enable faster tracking of moving objects than conventional image sensors, and have potential applications in autonomous vehicles, robotics, and neuromorphic engineering.^[2][3][4][5]

Naming and history

The first so-called artificial retina were reported in the late 1980's by Carver Mead and his doctoral students Misha Mahowald, and Tobias Delbrück.^[6][7] These silicon-based sensors were based on small circuits involving differential amplifiers, capacitors, and resistors. The sensors produced a spike and subsequent decay in output voltage in response to a step-change in illumination intensity. This response is analogous to that of animal retinal cells, which in the 1920's were observed to fire more frequently when the intensity of light was changed than when it was constant.^[8] The name silicon retina has hence been used to describe these sensors.^[9]

The term retinomorphic was first used in a conference paper by Lex Akers in 1990.^[10] The term received wider use by Stanford Professor of Engineering Kwabena Boahen, and has since been applied to a wide range of event-driven sensing strategies.^[11] The word is analogous to neuromorphic, which is applied to hardware elements (such as processors) designed to replicate the way the brain processes information.

Operating principles

There are several retinomorphic sensor designs which yield a similar response. The first designs employed a differential amplifier which compared the input signal from of a conventional sensor (e.g. a phototransistor) to a filtered version of the output,^[6] resulting in a gradual decay if the input was constant. Since the 1980's these sensors have evolved into much more complex and robust circuits.^[1]

A more compact design of retinomorphic sensor consists of just a photosensitive capacitor and a resistor in series.^[12] The output voltage of these retinomorphic sensors, ${\displaystyle V_{out}}$, is defined as voltage dropped across the resistor. The photosensitive capacitor is designed to have a capacitance which is a function of incident light intensity. If a constant voltage ${\displaystyle V_{in}}$, is applied across this RC circuit it will act as a passive high-pass filter and all voltage will be dropped across the capacitor (i.e. ${\displaystyle V_{out}=0}$). After a sufficient amount of time, the plates of the capacitor will be fully charged with a charge ${\displaystyle Q=\pm C_{dark}(V_{in}-V_{out})}$ on each plate, where ${\displaystyle C_{dark}}$ is the capacitance in the dark. Since ${\displaystyle V_{out}=0}$ under constant illumination, this can be simplified to ${\displaystyle Q=\pm C_{dark}V_{in}}$.

If light is then applied to the capacitor it will change capacitance to a new value: ${\displaystyle C_{light}}$. The charge that the plates can accommodate will therefore change to ${\displaystyle Q=\pm C_{light}(V_{in}-V_{out})}$, leaving a surplus / deficit of charge on each plate. The excess charge will be forced to leave the plates, flowing either to ground or the input voltage terminal. The rate of charge flow is determined by the resistance of the resistor ${\displaystyle R}$, and the capacitance of the capacitor. This charge flow will lead to a non-zero voltage being dropped across the resistor and hence a non-zero ${\displaystyle V_{out}}$. After the charge stops flowing the system returns to steady-state, all the voltage is once again dropped across the capacitor, and ${\displaystyle V_{out}=0}$ again.

For a capacitor to change its capacitance under illumination, the dielectric constant of the insulator between the plates,^[13] or the effective dimensions of the capacitor, must be illumination-dependent. The effective dimensions can be changed by using a bilayer material between the plates, consisting of an insulator and a semiconductor. Under appropriate illumination conditions the semiconductor will increase its conductivity when exposed to light, emulating the process of moving the plates of the capacitor closer together, and therefore increasing capacitance. For this to be possible, the semiconductor must have a low electrical conductivity in the dark, and have an appropriate band gap to enable charge generation under illumination. The device must also allow optical access to the semiconductor, through a transparent plate (e.g. using a transparent conducting oxide).

Applications

Conventional cameras capture every part of an image, regardless of whether it is relevant to the task. Because every pixel is measured, conventional image sensors are only able to sample the visual field at relatively low frame rates, typically 30 - 240 frames per second. Even in professional high speed cameras used for motion picture, the frame rate is limited to a few 10's of thousands of frames per second for a full resolution image. This limitation could represent a performance bottleneck in the identification of high speed moving objects. This is particularly critical in applications where rapid identification of movement is critical, such as in autonomous vehicles.

By contrast, retinomorphic sensors identify movement by design. This means that they do not have a frame rate and instead are event-driven, responding only when needed. For this reason, retinomorphic sensors are hoped to enable identification of moving objects much more quickly than conventional real-time image analysis strategies.^[4] Retinomorphic sensors are therefore hoped to have applications in autonomous vehicles,^[14][15] robotics,^[16] and neuromorphic engineering.^[17]

Theory

Retinomorphic sensor operation can be quantified using similar techniques to simple RC circuits, the only difference being that capacitance is not constant as a function of time in a retinomorphic sensor.^[18] If the input voltage is defined as ${\displaystyle V_{in}}$, the voltage dropped across the resistor as ${\displaystyle V_{out}}$, and the voltage dropped across the capacitor as ${\displaystyle V_{C}}$, we can use

${\displaystyle V_{in}=V_{C}+V_{out}}$

Defining the current flowing through the resistor as ${\displaystyle I}$, we can use Ohm's Law to write:

${\displaystyle V_{in}=V_{C}+IR}$

From the definition of current, we can then write this in terms of charge, ${\displaystyle Q}$, flowing off the bottom plate:

${\displaystyle V_{in}=V_{C}+R{\frac {dQ}{dt}}}$

where ${\displaystyle t}$ is time. Charge on the capacitor plates is defined by the product of capacitance, ${\displaystyle C}$, and the voltage across the capacitor, ${\displaystyle V_{C}}$, we can hence say:

${\displaystyle V_{in}=V_{C}+R{\frac {d}{dt}}(CV_{C})}$

Because capacitance in retinomorphic sensors is a function of time, ${\displaystyle C}$ cannot be taken out of the derivative as a constant. Using the product rule, we get the following general equation of retinomorphic sensor response:

${\displaystyle V_{in}=V_{C}+R{\bigg [}C{\frac {dV_{C}}{dt}}+V_{C}{\frac {dC}{dt}}{\bigg ]}}$

or, in terms of the output voltage:

${\displaystyle V_{out}=R{\bigg [}C{\frac {dV_{C}}{dt}}+V_{C}{\frac {dC}{dt}}{\bigg ]}}$

Response to a step-change in intensity

While the equation above is valid for any form of ${\displaystyle C(t)}$, it cannot be solved analytically unless the input form of the optical stimulus is known. The simplest form of optical stimulus would be a step function going from zero to some finite optical power density ${\displaystyle P}$ at a time ${\displaystyle t=t_{0}}$. While real-world applications of retinomorphic sensors are unlikely to be accurately described by such events, it is a useful way to understand and benchmark the performance of retinomorphic sensors. In particular, we are primarily concerned with the maximum height of the ${\displaystyle V_{out}}$ immediately after the light has been turned on.

In this case the capacitance could be described by:

${\displaystyle C(t)={\begin{cases}C_{dark},&{\text{for }}t<t_{0}\\C_{light},&{\text{for }}t\geq t_{0}\end{cases}}}$

The capacitance under illumination will depend on ${\displaystyle P}$. Semiconductors are known^[19] to have a conductance, ${\displaystyle G}$, which increases with a power-law dependence on incident optical power density: ${\displaystyle G\propto P^{\gamma }}$, where ${\displaystyle \gamma }$ is a dimensionless exponent. Since ${\displaystyle G}$ is linearly proportional to charge density, and capacitance is linearly proportional to charges on the plates for a given voltage, the capacitance of a retinomorphic sensor also has a power-law dependence on ${\displaystyle P}$. The capacitance as a function of time in response to a step function, can therefore be written as:

${\displaystyle C(t)={\begin{cases}C_{dark},&{\text{for }}t<t_{0}\\C_{dark}+\alpha _{0}P^{\gamma },&{\text{for }}t\geq t_{0}\end{cases}}}$

where ${\displaystyle \alpha _{0}}$ is the capacitance prefactor. For a step function we can re-write our differential equation for ${\displaystyle V_{in}}$ as a difference equation:

${\displaystyle V_{in}=V_{C}+R{\bigg [}C{\frac {\Delta V_{C}}{\Delta t}}+V_{C}{\frac {\Delta C}{\Delta t}}{\bigg ]}}$

where ${\displaystyle \Delta V_{C}}$ is the change in voltage dropped across the capacitor as a result of turning on the light, ${\displaystyle \Delta C}$ is the change in capacitance as a result of turning on the light, and ${\displaystyle \Delta t}$ is the time taken for the light to turn on. The variables ${\displaystyle V_{C}}$ and ${\displaystyle C}$ are defined as the voltage dropped across the capacitor and the capacitance, respectively, immediately after the light has been turned on. I.e. ${\displaystyle V_{C}}$ is henceforth shorthand for ${\displaystyle V_{C}(t_{0})}$, and ${\displaystyle C}$ is henceforth shorthand for ${\displaystyle C(t_{0})}$. Assuming the sensor has been held in the dark for sufficiently long before the light is turned on, the change in ${\displaystyle V_{C}}$ can hence be written as:

${\displaystyle \Delta V_{C}=V_{C}-V_{in}}$

Similarly, the change in ${\displaystyle C}$ can be written as

${\displaystyle \Delta C=\alpha _{0}P^{\gamma }}$

Putting these into the difference equation for ${\displaystyle V_{in}}$:

${\displaystyle V_{in}=V_{C}+R{\bigg [}C{\frac {V_{C}-V_{in}}{\Delta t}}+V_{C}{\frac {\alpha _{0}P^{\gamma }}{\Delta t}}{\bigg ]}}$

Multiplying this out:

${\displaystyle V_{in}\Delta t=V_{C}\Delta t+R{\bigg [}C(V_{C}-V_{in})+V_{C}\alpha _{0}P^{\gamma }{\bigg ]}}$

Since we are assuming the light turns on very quickly we can approximate ${\displaystyle \Delta t\approx 0}$. This leads to the following:

${\displaystyle V_{C}=V_{in}{\frac {C_{dark}+\alpha _{0}P^{\gamma }}{C_{dark}+2\alpha _{0}P^{\gamma }}}}$

Using the relationship ${\displaystyle V_{in}=V_{C}+V_{out}}$, this can then be written in terms of the output voltage:

${\displaystyle V_{max}=V_{in}{\frac {\alpha _{0}P^{\gamma }}{C_{dark}+2\alpha _{0}P^{\gamma }}}}$

Where we have defined the peak height as ${\displaystyle V_{max}=V_{out}(t_{0})}$, since he peak occurs immediately after the light has been turned on.

The retinomorphic figure of merit, ${\displaystyle \Lambda }$, is defined as the ratio of the capacitance prefactor and the capacitance of the retinomorphic sensor in the dark:^[18]

${\displaystyle \Lambda ={\frac {\alpha _{0}}{C_{dark}}}}$

With this parameter, the inverse ratio of peak height to input voltage can be written as follows:

${\displaystyle {\frac {V_{in}}{V_{max}}}=2+{\frac {1}{\Lambda P^{\gamma }}}}$

The value of ${\displaystyle \gamma }$ will depend on the nature of recombination in the semiconductor,^[20] but if band-to-band recombination dominates and the charge density of electrons and holes are equal, ${\displaystyle \gamma =0.5}$. For systems where this is approximately true^[21] the following simplification to the above equation an be made:

${\displaystyle {\frac {V_{in}}{V_{max}}}=2+{\frac {1}{\Lambda P^{1/2}}}}$

This equation provides a simple method for evaluating the retinomorphic figure of merit from experimental data. This can be carried out by measuring the peak height, ${\displaystyle V_{max}}$, of a retinomorphic sensor in response to a step change in light intensity from 0 to ${\displaystyle P}$, for a range of values ${\displaystyle P}$. Plotting ${\displaystyle V_{in}/V_{max}}$ as a function of ${\displaystyle P^{-1/2}}$ should yield a straight line with a gradient of ${\displaystyle 1/\Lambda }$. This approach assumes that ${\displaystyle V_{max}}$ is linearly proportional to ${\displaystyle V_{in}}$.

See also

• Active-pixel sensor
• Charge coupled device
• Event camera
• Neuromorphic engineering
• Optical sensor
• Photodiode

References