**PSC 128/NPB 163
Linear Algebra (a primer) **

**Why linear algebra?**

- What if more than one value is going in and coming out of
a system? For example, we have many inputs coming into the retina (photoreceptors)
and many outputs (retinal ganglion cells). Any single input affects many
different outputs, and any given output is influenced by many different inputs.
How do we even begin to understand such a system?

- Linear algebra provides a useful tool for characterizing
the behavior of systems where many values must be represented simultaneously.
- e.g.,
*the brain*.

- A vector is simply a list of numbers. A vector is usually
denoted in boldface, or underlined:

or .

An “*n*-dimensional vector” has*n*elements. One can think of an*n*-dimensional vector as either a point in an*n*-dimensional space, or as an arrow drawn from the origin to the point with coordinates .

- The addition of two vectors is simply the addition of their
elements:

,

Or, we could write the elements of**z**as . Geometrically, vectors are added by placing them end to end. The vector from the origin to the tip of the last vector is the sum vector.

- The
*inner-product*(or*dot**product*) of two vectors takes the sum of products of the elements of each vector: . This provides a measure of the similarity of two vectors (provided you know the length of each vector). If you divide the inner product by the length of each vector, it gives you the cosine of the angle between them

where is the angle between the two vectors, and . An alternative geometric interpretation of the inner product is that gives you the length of after it has been projected onto**y.**Thus, orthogonal vectors have an inner product of zero.The inner product is oftentimes also denoted , where the superscript*T*denotes “transpose.” The transpose of a vector simply tilts it on its side so it is written as a row of numbers, .

- A matrix is just a 2D array of numbers. It is used to map
a vector
**x**into a new vector**y**via the relation . It takes the place of the constant, k, in a simple one-dimensional (scalar) linear system. An*m*x*n*matrix**M**has elements

and we can write the resulting vector,**y**, elementwise as . In this case the vector**y**will be*m*-dimensional.

- Analysis/synthesis: the
*row*vectors of a matrix tell you what part of the input space (vectors on the right side of the matrix) the matrix*analyzes*. The*column*vectors tell you what part of the output space (i.e., vectors coming out the left side) the matrix can*synthesize*from any input. In each case, it is the number of*linearly independent*vectors that define these spaces.

- Just as a linear scalar system can produce only a limited
set of remappings of scalar values, so too is a matrix operation limited
in the set of remappings it can produce. What do these remappings look like?
- rotations (orthonormal matrices) and scalings (diagonal matrices) and
combinations thereof.

- If
**y**=**Mx**and**z**=**Ay**, then**z**=**AMx**=**Cx**, where**C**=**AM**is just another matrix. Thus, the concatenation of two matrix operations is just another matrix operation. The combined matrix**C**is obtained via matrix multiplication:

Note however that matrix multiplication is not commutative - i.e, .

- Any matrix
**M**can be decomposed in terms of a rotation, a scaling, and another rotation. i.e.,