Dimensionality Reduction

Dimensionality reduction is the process of reducing the number of random variables under consideration for training a model.

Processes

Generalized Dimensionality Reduction processes can be expressed as follows:

$\\\begin{Bmatrix}\left \{d_{1},d_{2},...,d_{n} \right \}\\ \left \{f_{1},f_{2},...,f_{a} \right \}\end{Bmatrix}\Rightarrow A\left ( d,f,p,m \right ) \Rightarrow S\left ( d,f,p,m \right )\Rightarrow \left \{f_{1},f_{2},...,f_{s} \right \} \\\\\\\mathrm{\textup{where:}} \\\\d\,\,\,\,\,\,\mathrm{\textup{data}} \\n\,\,\,\,\,\,\mathrm{\textup{number of data elements}} \\f\,\,\,\,\,\,\mathrm{\textup{features}} \\a\,\,\,\,\,\,\mathrm{\textup{number of available features}} \\p\,\,\,\,\,\,\mathrm{\textup{prediction objectives}} \\m\,\,\,\,\mathrm{\textup{models under consideration}} \\s\,\,\,\,\,\,\mathrm{\textup{number of selected features}} \\A\,\,\,\,\mathrm{\textup{analysis functions}} \\S\,\,\,\,\mathrm{\textup{selection functions}}$

Analysis Functions

Functions for dimensionality reduction analysis include:

Factor Analysis: describes variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors
Principal Component Analysis: method to determine the correlation of data points

Selection Functions

Functions for dimensionality reduction selection include:

Feature Selection: method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables

Dimensionality Reduction

Processes

Analysis Functions

Selection Functions

References