Overview
Calculus
Computing Systems
Data
Graphs
Linear Algebra
Models and Modeling
Organization
Probability
Programming Constructs
- Programming Constructs Overview
- Abstraction
- Array
- Attribute
- Best-first Search
- Binary Search
- Block
- Branch
- Callback
- Class
- Conditional
- Constructor
- Container/Collection
- Dynamic Array
- Encapsulation
- Exception
- Expression
- Function
- Garbage Collection
- Greedy Algorithms
- Hash
- HTTP Request
- Identifier
- Inheritance
- Inner Class
- Instance
- Iterator
- Keyword
- Lambda
- Libraries
- List
- Linked List
- Literal
- Metaclass
- Method
- Mixin
- Object
- Operator
- Overloading
- Overriding
- Package
- Parameter
- Polymorphism
- Primitive
- Programming Process
- Recursion
- Reflection
- Regular Expression
- Reserved Word
- Return
- Sort
- Statement
- Switch
- Table
- This/Self
- Token
- Type
- Variable
Statistics
Trigonometry

Glossary and Index
Mathematical Symbols
Applications
Search
Blog
About the Author
Contact

Menu

The Science of Machine Learning & AI

Mathematics - Data Science - Computer Science

Overview
Calculus
Computing Systems
Data
Graphs
Linear Algebra
Models and Modeling
Organization
Probability
Programming Constructs
- Programming Constructs Overview
- Abstraction
- Array
- Attribute
- Best-first Search
- Binary Search
- Block
- Branch
- Callback
- Class
- Conditional
- Constructor
- Container/Collection
- Dynamic Array
- Encapsulation
- Exception
- Expression
- Function
- Garbage Collection
- Greedy Algorithms
- Hash
- HTTP Request
- Identifier
- Inheritance
- Inner Class
- Instance
- Iterator
- Keyword
- Lambda
- Libraries
- List
- Linked List
- Literal
- Metaclass
- Method
- Mixin
- Object
- Operator
- Overloading
- Overriding
- Package
- Parameter
- Polymorphism
- Primitive
- Programming Process
- Recursion
- Reflection
- Regular Expression
- Reserved Word
- Return
- Sort
- Statement
- Switch
- Table
- This/Self
- Token
- Type
- Variable
Statistics
Trigonometry

Glossary and Index
Mathematical Symbols
Applications
Search
Blog
About the Author
Contact

Mathematical Notation Powered by CodeCogs

This work is licensed under a Creative Commons Attribution 4.0 International License.

Blog RSS

Matrices

A matrix (plural matrices) is an array of numbers, symbols, or expressions, arranged in rows and columns. A general form is:

$\begin{bmatrix} A_{11} &A_{12} &\cdots &A_{1n} \\ A_{21}&A_{22} & \cdots &A_{2n} \\ \vdots &\vdots & \ddots & \vdots\\ A_{m1} & A_{m2} & \cdots & A_{mn} \end{bmatrix}$

Matrices and matrix mathematics is important in Machine Learning for a number of reasons:

Data Cluster Manipulation

Machine Learning operations often involve retrieving, using and storing clusters of data points. Matrices are an efficient way to handle this type of data.

Mathematical Formulas and Program Code

Matrices can be represented in compact mathematical formulas which can be programmed for easy access.

Custom Computing Hardware

Custom computing hardware such as GPUs and TPUs are constructed for the efficient and rapid processing of data.

Matrix Addition

The formula for the sum of two matrices A+B to form a new matrix C is:

$\\C_{i,j}=A_{i,j}+B_{i,j} \\\\\textup{so for example:} \\\\\begin{bmatrix} 2 & 5 & 2\\ 5 & 6 & 9 \end{bmatrix} = \begin{bmatrix} 1 & 3 & 2 \\ 0 & 4 & 6 \end{bmatrix} + \begin{bmatrix} 1 & 2 & 0\\ 5 & 2 & 3 \end{bmatrix}$

Matrix Scalar Multiplication

The formula for multiplying a matrix A by a scalar value s to form a new matrix C is:

$\\C_{i,j}=s\cdot A_{i,j} \\\\\textup{so for example:} \\\\\begin{bmatrix} 2 & -4 & 0\\ 6 & 8 & 2 \end{bmatrix} = 2 \cdot \begin{bmatrix} 1 & -2 & 0\\ 3 & 4 & 1 \end{bmatrix}$

Matrix Full Multiplication

Matrix multiplication is performed under these conditions:

the number of columns in the first matrix must be equal to the number of rows in the second matrix
result matrix, known as the matrix product, has the number of rows of the first and the number of columns of the second matrix

The formula for the sum of two matrices AxB to form a new matrix C is:

$\\for \,\,\,i=1,\cdots ,m\,\,\,and\,\,\,j=1,\cdots ,p: \\\\\textup{ }\;\;\;\;\;\; c_{ij}=a_{i1}b_{1j}\,+\,\cdots\,+a_{in}b_{nj}=\sum_{k=1}^{n}a_{ik}b_{kj} \\\\\textup{where:} \\\\a\,\,\,\,\,\,\textup{left matrix} \\b\,\,\,\,\,\,\,\textup{right matrix} \\c\,\,\,\,\,\,\,\textup{result matrix} \\n\,\,\,\,\,\,\textup{number of columns in the left matrix and rows in the right matrix} \\m\,\,\,\,\textup{number of rows in the left matrix} \\p\,\,\,\,\,\,\textup{number of columns right matrix} \\k\,\,\,\,\,\,\textup{counter for summation over n} \\i\,\,\,\,\,\,\,\textup{index over m} \\j\,\,\,\,\,\,\textup{index over p}$

As shown below, matrix multiplication uses vector dot products to calculate result cell values:

$\\ \textup{each cell of the result matrix is the vector dot product of} \\\textup{a row from the left matrix A and column from the right matrix B:} \\\\C_{ij}=\begin{bmatrix} A_{i1}&A_{i2} \end{bmatrix}\bullet \begin{bmatrix} B_{1j} & B_{2j} \end{bmatrix} \\\\\textup{for example:} \\\\C_{12}=\begin{bmatrix} A_{11}&A_{12} \end{bmatrix}\bullet \begin{bmatrix} B_{12} & B_{22} \end{bmatrix} \\\\\textup{which can be represented within the respective matrices as:} \\\\\begin{bmatrix} \bullet & C_{12} &\bullet \\ \bullet & \bullet &\bullet \\ \bullet &\bullet &\bullet \\ \bullet &\bullet & \bullet \end{bmatrix} = \begin{bmatrix} A_{11} &A_{12} \\ \cdot &\cdot \\ \cdot &\cdot \\ \cdot & \cdot \end{bmatrix} \times \begin{bmatrix} \cdot &B_{12} &\cdot \\ \cdot &B_{22} & \cdot \end{bmatrix}$

Matrix Concatenation

Matrix concatenation is the process of joining multiple matrices along the vertical axis, horizontal axis, or as a single row to create a new matrix:

$\\A=\begin{bmatrix} 1 &2 \\ 3 &4 \end{bmatrix} \\\\\\B=\begin{bmatrix} 5 & 6 \end{bmatrix} \\\\\\A\,\triangledown B=\begin{bmatrix} 1 &2 \\ 3 &4 \\ 5 &6 \end{bmatrix} \\\\\\A\triangleright B=\begin{bmatrix} 1 &2 &5 \\ 3 &4 &6 \end{bmatrix} \\\\\\A\,\square \, B=\begin{bmatrix} 1 &2 &3 &4 &5 &6 \end{bmatrix} \\\\\\\mathrm{\textup{where:}} \\\\A\,\,\,\,\mathrm{\textup{matrix}} \\B\,\,\,\,\mathrm{\textup{matrix}} \\\triangledown\,\,\,\,\mathrm{\textup{concatenation along the vertical axis}} \\\triangleright\,\,\,\,\mathrm{\textup{concatenation along the horizontal axis}} \\\square\,\,\,\,\mathrm{\textup{concatenation along a single row}}$

References

Web Search
Wikipedia Matrix Mathematics Article
YouTube Search