## Of vector and matrix products

Listed below are definitions of some vector or matrix products. I will be needing these definitions in the succeeding posts. Thanks to Jetpack‘s Latex support, I can now include equations in my post.

### Outer Product

Given a row vector $\mathbf{u} = \begin{bmatrix} u_1 & u_2 & \ldots & u_n\end{bmatrix}$ with n elements and another vector $\mathbf{v} = \begin{bmatrix} v_1 & v_2 & \ldots & v_m\end{bmatrix}$ with m elements, the outer product of u and v is given by

$\mathbf{u} \otimes \mathbf{v} = \begin{bmatrix} u_1v_1 & u_1v_2 & \ldots & u_1v_m \\ u_2v_1 & u_2v_2 & \ldots & u_2v_m \\ \vdots & \vdots & \ddots & \vdots \\ u_nv_1 & u_nv_2 & \ldots & u_nv_m \end{bmatrix}.$

The outer product as defined above can also be written as a matrix multiplication:

$\mathbf{u} \otimes \mathbf{v} = \mathbf{u}^T \mathbf{v} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} \begin{bmatrix} v_1 & v_2 & \ldots & v_m \end{bmatrix} = \begin{bmatrix} u_1 v_1 & u_1 v_2 & \ldots & u_1 v_m \\ u_2 v_1 & u_2 v_2 & \ldots & u_2 v_m \\ \vdots & \vdots & \ddots & \vdots \\ u_nv_1 & u_nv_2 & \ldots & u_n v_m \end{bmatrix}.$

### Kronecker Product

Given two matrices A which is m-by-n and B, which is q-by-p, the Kronecker product C of A and B is given by:

$\mathbf{C} = \mathbf{A} \otimes \mathbf{B} = \begin{bmatrix} a_{11}\mathbf{B} & a_{12}\mathbf{B} & \ldots & a_{1n}\mathbf{B} \\ a_{21}\mathbf{B} & a_{22}\mathbf{B} & \ldots & a_{2n}\mathbf{B} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1}\mathbf{B} & a_{m2}\mathbf{B} & \ldots & a_{mn} \mathbf{B} \end{bmatrix},$

where C is an mp-by-nq matrix.

### Khatri-Rao Product

Using the above definition, we can now define the Khatri-Rao product C of two matrices A and B as a column-wise Kronecker product of the two matrices. First, rewrite the matrices A and B as follows:

$\mathbf{A} = \left[\; \mathbf{a}_1 \mid \mathbf{a}_2 \mid \ldots \mid \mathbf{a}_n \; \right]$

and

$\mathbf{B} = \left[\; \mathbf{b}_1 \mid \mathbf{b}_2 \mid \ldots \mid \mathbf{b}_n \;\right]$

where $\mathbf{a}_i, \mathbf{b}_i$ represent the columns of A and B, respectively. The Khatri-Rao product is given by

$\mathbf{C} = \mathbf{A} \odot \mathbf{B} = \left[\; \mathbf{a}_1 \otimes \mathbf{b}_1 \mid \mathbf{a}_2 \otimes \mathbf{b}_2 \mid \ldots \mid \mathbf{a}_n \otimes \mathbf{b}_n \right]$

Whew! I thought including equations was easy. Just hover your mouse over the equations above and you’ll see how complicated the code is to write such simple equations. There should be a better way.  Anyone?