Maximally informative dimensions

Maximally informative dimensions is a

Neural stimulus-response functions are typically given as the probability of a neuron generating an action potential, or spike, in response to a stimulus ${\displaystyle \mathbf {s} }$. The goal of maximally informative dimensions is to find a small relevant subspace of the much larger stimulus space that accurately captures the salient features of ${\displaystyle \mathbf {s} }$. Let ${\displaystyle D}$ denote the dimensionality of the entire stimulus space and ${\displaystyle K}$ denote the dimensionality of the relevant subspace, such that ${\displaystyle K\ll D}$. We let ${\displaystyle \{\mathbf {v} ^{K}\}}$ denote the basis of the relevant subspace, and ${\displaystyle \mathbf {s} ^{K}}$ the projection of ${\displaystyle \mathbf {s} }$ onto ${\displaystyle \{\mathbf {v} ^{K}\}}$. Using Bayes' theorem we can write out the probability of a spike given a stimulus:

${\displaystyle P(spike|\mathbf {s} ^{K})=P(spike)f(\mathbf {s} ^{K})}$

where

${\displaystyle f(\mathbf {s} ^{K})={\frac {P(\mathbf {s} ^{K}|spike)}{P(\mathbf {s} ^{K})}}}$

is some nonlinear function of the projected stimulus.

In order to choose the optimal ${\displaystyle \{\mathbf {v} ^{K}\}}$, we compare the prior stimulus distribution ${\displaystyle P(\mathbf {s} )}$ with the spike-triggered stimulus distribution ${\displaystyle P(\mathbf {s} |spike)}$ using the Shannon information. The average information (averaged across all presented stimuli) per spike is given by

${\displaystyle I_{spike}=\sum _{\mathbf {s} }P(\mathbf {s} |spike)log_{2}[P(\mathbf {s} |spike)/P(\mathbf {s} )]}$.^[3]

Now consider a ${\displaystyle K=1}$ dimensional subspace defined by a single direction ${\displaystyle \mathbf {v} }$. The average information conveyed by a single spike about the projection ${\displaystyle x=\mathbf {s} \cdot \mathbf {v} }$ is

${\displaystyle I(\mathbf {v} )=\int dxP_{\mathbf {v} }(x|spike)log2[P_{\mathbf {v} }(x|spike)/P_{\mathbf {v} }(x)]}$,

where the probability distributions are approximated by a measured data set via ${\displaystyle P_{\mathbf {v} }(x|spike)=\langle \delta (x-\mathbf {s} \cdot \mathbf {v} )|spike\rangle _{\mathbf {s} }}$ and ${\displaystyle P_{\mathbf {v} }(x)=\langle \delta (x-\mathbf {s} \cdot \mathbf {v} )\rangle _{\mathbf {s} }}$, i.e., each presented stimulus is represented by a scaled Dirac delta function and the probability distributions are created by averaging over all spike-eliciting stimuli, in the former case, or the entire presented stimulus set, in the latter case. For a given dataset, the average information is a function only of the direction ${\displaystyle \mathbf {v} }$. Under this formulation, the relevant subspace of dimension ${\displaystyle K=1}$ would be defined by the direction ${\displaystyle \mathbf {v} }$ that maximizes the average information ${\displaystyle I(\mathbf {v} )}$.

This procedure can readily be extended to a relevant subspace of dimension ${\displaystyle K>1}$ by defining

${\displaystyle P_{\mathbf {v} ^{K}}(\mathbf {x} |spike)=\langle \prod _{i=1}^{K}\delta (x_{i}-\mathbf {s} \cdot \mathbf {v} _{i})|spike\rangle _{\mathbf {s} }}$

and

${\displaystyle P_{\mathbf {v} ^{K}}(\mathbf {x} )=\langle \prod _{i=1}^{K}\delta (x_{i}-\mathbf {s} \cdot \mathbf {v} _{i})\rangle _{\mathbf {s} }}$

and maximizing ${\displaystyle I({\mathbf {v} ^{K}})}$.

Importance

Maximally informative dimensions does not make any assumptions about the Gaussianity of the stimulus set, which is important, because naturalistic stimuli tend to have non-Gaussian statistics. In this way the technique is more robust than other dimensionality reduction techniques such as spike-triggered covariance analyses.

References