Neural Computation

From Mind-Brain.org

Main :: Neural Computation

The brain, often considered as a dynamical system, may also be considered as a computational system that computes representations and their transformations. While consciousness itself is not dependent on external inputs, our everyday world of waking consciousness is dependent on inputs from receptor arrays, including photoreceptors, gustatory receptors, touch receptors, etc, which convey information about an external environment.

1 Quantifying Information
2 How Neurons Encode Information
3 Cross-correlation
4 Joint Cumulants

[edit] Quantifying Information

Information may be encoded using several neuronal or network variables and is often quantified using Information Theory.

[edit] How Neurons Encode Information

mean firing rate - appears common in the peripheral nervous system.
cross-correlations - synchrony is included under this term.
higher-order correlations
synfire chains
spike-timing dependent firing

[edit] Cross-correlation

For two random variables, X and Y, the cross-correlation is given by the following:

$\rho_{X,Y}=\frac{E(XY)-E(X)E(Y)}{\sqrt{E(X^2)-E^2(X)}~\sqrt{E(Y^2)-E^2(Y)}}$

where E() denotes the expectation value (or mean).

For the special case of two binary vectors, x and y, the cross-correlation is given by the following:

$q(x,y)=\frac{\sum P(xy = 1) - \sum P(x = 1)\sum P(y = 1)}{\sqrt{\sum P(x = 1) - \big (\sum P(x = 1)\big )^2} \sqrt{\sum P(y = 1) - \sum \big (P(y = 1)\big )^2}}$

where P() denotes the probability of the argument. The correlation is defined only if both of the standard deviations are finite and both of them are nonzero. It is a corollary of the Cauchy-Schwarz inequality that the correlation cannot exceed 1 in absolute value.

The correlation is 1 in the case of an increasing linear relationship, −1 in the case of a decreasing linear relationship, and some value in between in all other cases, indicating the degree of linear dependence between the variables. The closer the coefficient is to either −1 or 1, the stronger the correlation between the variables.

If the variables are statistically independent, then the correlation is 0, but the converse is not true because the correlation coefficient detects only linear dependencies between two variables. Here is an example: Suppose the random variable X is uniformly distributed on the interval from −1 to 1, and Y = X². Then Y is completely determined by X, so that X and Y are dependent, but their correlation is zero; they are uncorrelated. However, in the special case when X and Y are jointly normal, independence is equivalent to uncorrelatedness.

[edit] Joint Cumulants

The joint cumulant of several random variables X₁, ..., X_n is

$\kappa(X_1,\dots,X_n) =\sum_\pi (|\pi|-1)!(-1)^{|\pi|-1}\prod_{B\in\pi}E\left(\prod_{i\in B}X_i\right)$

where π runs through the list of all partitions of { 1, ..., n }, and B runs through the list of all blocks of the partition π. For example,

$\kappa(X,Y,Z)=E(XYZ)-E(XY)E(Z)-E(XZ)E(Y)\,$

$-E(YZ)E(X)+2E(X)E(Y)E(Z)\,$

The joint cumulant of just one random variable is its expected value, and that of two random variables is their covariance. If some of the random variables are independent of all of the others, then the joint cumulant is zero. If all n random variables are the same, then the joint cumulant is the nth ordinary cumulant.

The combinatorial meaning of the expression of moments in terms of cumulants is easier to understand than that of cumulants in terms of moments:

$E(X_1\cdots X_n)=\sum_\pi\prod_{B\in\pi}\kappa(X_i : i \in B)$