Feature Selection

Feature Selection chooses a set of variables that will be present in the data records used for model training, testing and prediction processing.

Factors

relevance: features should be correlated with the predictive objectives of the model
redundancy: redundancy of features should be minimized
dimensionality: manages the number of features
availability: features should be available in a sufficient number of model training records
accuracy: features should provide data that is as accurate as possible

Processes

Generalized Data Visualization processes can be expressed as follows:

$\small \\\begin{Bmatrix}\left \{d_{1},d_{2},...,d_{n} \right \}\\ \left \{f_{1},f_{2},...,f_{a} \right \}\end{Bmatrix}\Rightarrow F\begin{pmatrix}N\left ( d,f,p \right )\\ M\left (d,f,p,m \right )\\ A\left (d,f,p,m \right )\end{pmatrix}\Rightarrow \left \{f_{1},f_{2},...,f_{s} \right \} \\\\\mathrm{\textup{where:}} \\\\d\,\,\,\,\,\,\mathrm{\textup{data}} \\n\,\,\,\,\,\,\mathrm{\textup{number of data elements}} \\f\,\,\,\,\,\,\mathrm{\textup{features}} \\a\,\,\,\,\,\,\mathrm{\textup{number of available features}} \\p\,\,\,\,\,\,\mathrm{\textup{prediction objectives}} \\m\,\,\,\,\mathrm{\textup{models}} \\s\,\,\,\,\,\,\mathrm{\textup{number of selected features}} \\F\,\,\,\,\,\mathrm{\textup{feature selection functions}} \\N\,\,\,\,\mathrm{\textup{mining functions}} \\M\,\,\,\,\mathrm{\textup{modeling functions}} \\A\,\,\,\,\,\mathrm{\textup{analysis functions}}$

Notes:

data: data can be from sources such as databases, files, commercial repositories
prediction objectives: can include factors such as applications, accuracy
feature selection functions: is an iterative process of mining, modeling, and analysis which is carried out until the prediction objectives are met
mining functions: includes functions such as data discovery, data cleaning, data normalization, data ETL
modeling functions: includes functions such as model training, feature selection, model selection, hyperparameter tuning
analysis functions: includes functions such as probability and statistical analysis, business analysis, model accuracy analysis, prediction results analysis

Feature Selection

Factors

Processes

References