Of vector and matrix products


Listed below are definitions of some vector or matrix products. I will be needing these definitions in the succeeding posts. Thanks to Jetpack‘s Latex support, I can now include equations in my post.

Outer Product

Given a row vector \mathbf{u} = \begin{bmatrix} u_1 & u_2 & \ldots & u_n\end{bmatrix}  with n elements and another vector \mathbf{v} = \begin{bmatrix} v_1 & v_2 & \ldots & v_m\end{bmatrix}  with m elements, the outer product of u and v is given by

    \mathbf{u} \otimes \mathbf{v} =  \begin{bmatrix}  u_1v_1 & u_1v_2 & \ldots & u_1v_m \\    u_2v_1 & u_2v_2 & \ldots & u_2v_m \\  \vdots & \vdots & \ddots & \vdots \\  u_nv_1 & u_nv_2 & \ldots & u_nv_m  \end{bmatrix}.

The outer product as defined above can also be written as a matrix multiplication:

    \mathbf{u} \otimes \mathbf{v} = \mathbf{u}^T \mathbf{v} =    \begin{bmatrix}  u_1 \\ u_2 \\ \vdots \\ u_n  \end{bmatrix}  \begin{bmatrix}  v_1 & v_2 & \ldots & v_m  \end{bmatrix}    = \begin{bmatrix}  u_1 v_1 & u_1 v_2 & \ldots & u_1 v_m \\  u_2 v_1 & u_2 v_2 & \ldots & u_2 v_m \\  \vdots & \vdots & \ddots & \vdots \\  u_nv_1 & u_nv_2 & \ldots & u_n v_m  \end{bmatrix}.

Kronecker Product

Given two matrices A which is m-by-n and B, which is q-by-p, the Kronecker product C of A and B is given by:

    \mathbf{C} = \mathbf{A} \otimes \mathbf{B} = \begin{bmatrix}  a_{11}\mathbf{B} & a_{12}\mathbf{B} & \ldots & a_{1n}\mathbf{B} \\  a_{21}\mathbf{B} & a_{22}\mathbf{B} & \ldots & a_{2n}\mathbf{B} \\  \vdots & \vdots & \ddots & \vdots \\  a_{m1}\mathbf{B} & a_{m2}\mathbf{B} & \ldots & a_{mn} \mathbf{B}  \end{bmatrix},

where C is an mp-by-nq matrix.

Khatri-Rao Product

Using the above definition, we can now define the Khatri-Rao product C of two matrices A and B as a column-wise Kronecker product of the two matrices. First, rewrite the matrices A and B as follows:

\mathbf{A} = \left[\; \mathbf{a}_1 \mid \mathbf{a}_2 \mid \ldots \mid \mathbf{a}_n \; \right]


\mathbf{B} = \left[\; \mathbf{b}_1 \mid \mathbf{b}_2 \mid \ldots \mid \mathbf{b}_n \;\right]

where \mathbf{a}_i, \mathbf{b}_i represent the columns of A and B, respectively. The Khatri-Rao product is given by

    \mathbf{C} = \mathbf{A} \odot \mathbf{B} = \left[\; \mathbf{a}_1 \otimes \mathbf{b}_1 \mid \mathbf{a}_2 \otimes \mathbf{b}_2 \mid \ldots \mid \mathbf{a}_n \otimes \mathbf{b}_n \right]

Whew! I thought including equations was easy. Just hover your mouse over the equations above and you’ll see how complicated the code is to write such simple equations. There should be a better way.  Anyone?

Leave a Reply