• Overview
  • Calculus
    • Calculus Overview
    • Activation Functions
    • Differential Calculus
    • Euler's Number
    • Gradients
    • Integral Calculus
    • Logarithms
    • Rectifier Activation Function
    • Sigmoid Activation Function
    • Stochastic Gradient Descent
    • Tanh Activation Function
  • Computing Systems
    • Computing Systems Overview
    • Application Programming Interface
    • Big O Notation
    • Client-Server Architecture
    • Cloud Computing
    • DOM
    • Exponential Growth
    • Graphics Processing Units
    • HTML iframe
    • Hybrid Cloud Computing
    • Internet Protocol Suite
    • Machine Learning & AI Platforms
    • P Versus NP Complexity
    • Quantum Computing
    • Server
    • Software Containers
    • System Scaling
    • Web Crawler
  • Data
    • Data Overview
    • Columnar Databases
    • CSV Data
    • Data Cleaning
    • Data Discovery
    • Data ETL
    • Data Flow
    • Data Lake
    • Data Lakehouse
    • Data Pipeline
    • Data Visualization
    • Data Warehouse
    • Dimensionality Reduction
    • Document Databases
    • Extrapolation
    • Factor Analysis
    • Graph Databases
    • Interpolation
    • JSON
    • Large Data Querying
    • Normalization
    • Outliers
    • Principal Components Analysis
    • Relational Databases
    • Sampling
    • Signal Processing
    • Synthetic Data
    • Vector Databases
  • Linear Algebra
    • Linear Algebra Overview
    • Concatenation
    • Convolution
    • Eigenvalues and Eigenvectors
    • Linear Equations
    • Linear Vector Projection
    • Masking
    • Matrices
    • Pooling
    • Scalars
    • Softmax Function
    • Vectors
  • Models and Modeling
    • Models and Modeling Overview
    • AI Agents
    • Algorithm Libraries
    • Artificial General Intelligence
    • Artificial Narrow Intelligence
    • Artificial Neural Networks
    • Artificial Superintelligence
    • Artificial Universal Intelligence
    • Attention
    • Automated Machine Learning
    • Backpropagation
    • Causal Embedding
    • Classification
    • Cluster Analysis
    • Collaborative Filtering
    • Convolutional Neural Networks
    • Cross Decomposition
    • Curve Fitting
    • Decision Trees
    • Deep Learning
    • Deep Reasoning
    • Diffusion Models
    • Ensemble Learning
    • Explainability
    • Feature Selection
    • Fourier Analysis
    • Foundation Models
    • Gaussian Analysis
    • Generative Adversarial Networks
    • Generative AI
    • Gradient Boosting
    • Graphs
    • Histogram of Oriented Gradients
    • Image Processing
    • K-Means Clustering
    • Large Language Models
    • Linear Regression
    • Logistic Regression
    • Long Short-term Memory
    • Markov Chains
    • Model Alignment
    • Model Categories
    • Model Self Improvement
    • Modeling Process
    • Naive Bayes
    • Nearest Neighbors
    • Probabilistic Graphical Models
    • Prompts and Prompting
    • Random Forest
    • Recurrent Neural Networks
    • Regression Analysis
    • Regularization
    • Reinforcement Learning
    • Retrieval Augmented Generation
    • Supervised Learning
    • Support Vector Machines
    • Transformer Neural Networks
    • Unsupervised Learning
    • Word Embedding
  • Organization
    • Organization Overview
    • Agile Processes
    • Application Selection Process
    • Business Model Components
    • Chief AI Officer
    • Coding
    • Functional Groups
    • Governance
    • Implementation
    • Individuals
    • Research
    • Risks
    • Staying Current
  • Probability
    • Probability Overview
    • Central Limit Theorem
    • Cross Entropy Loss
    • Entropy
    • Independent Events
    • Law of Large Numbers
    • Mutually Exclusive Events
    • Normal Distribution
    • Poisson Distribution
    • Probability Density Function
    • Probability Measure
    • P-Value
  • Programming Constructs
    • Programming Constructs Overview
    • Abstraction
    • Array
    • Attribute
    • Best-first Search
    • Binary Search
    • Block
    • Branch
    • Callback
    • Class
    • Conditional
    • Constructor
    • Container/Collection
    • Dynamic Array
    • Dynamic Programming
    • Encapsulation
    • Exception
    • Expression
    • Function
    • Garbage Collection
    • Greedy Algorithms
    • Hash
    • HTTP Request
    • Identifier
    • Inheritance
    • Inner Class
    • Instance
    • Iterator
    • Keyword
    • Lambda
    • Libraries
    • List
    • Linked List
    • Literal
    • Metaclass
    • Method
    • Mixin
    • Object
    • Operator
    • Overloading
    • Overriding
    • Package
    • Parameter
    • Polymorphism
    • Primitive
    • Programming Process
    • Recursion
    • Reflection
    • Regular Expression
    • Reserved Word
    • Return
    • Sort
    • Statement
    • Switch
    • Table
    • This/Self
    • Token
    • Type
    • Variable
  • Statistics
    • Statistics Overview
    • Accuracy
    • A/B Testing
    • Bias
    • Bias-Variance Tradeoff
    • Confidence
    • Correlation
    • Confusion Matrix
    • Deviation
    • Dispersion
    • Estimator
    • Fairness
    • Loss (Cost) Function
    • Mean Squared Error
    • Hypothesis
    • Prediction and Inference
    • Repeatability
    • Standard Deviation
    • Statistical Power of a Test
    • Variance
  • Trigonometry
    • Trigonometry Overview
    • Cosine Similarity
    • Periodic Functions
    • Trigonometric Functions
  • Glossary and Index
  • Mathematical Symbols
  • Applications
  • Search
  • Blog
  • About the Author
  • Contact
  • Menu

The Science of Machine Learning & AI

Mathematics - Data Science - Computer Science
  • Overview
  • Calculus
    • Calculus Overview
    • Activation Functions
    • Differential Calculus
    • Euler's Number
    • Gradients
    • Integral Calculus
    • Logarithms
    • Rectifier Activation Function
    • Sigmoid Activation Function
    • Stochastic Gradient Descent
    • Tanh Activation Function
  • Computing Systems
    • Computing Systems Overview
    • Application Programming Interface
    • Big O Notation
    • Client-Server Architecture
    • Cloud Computing
    • DOM
    • Exponential Growth
    • Graphics Processing Units
    • HTML iframe
    • Hybrid Cloud Computing
    • Internet Protocol Suite
    • Machine Learning & AI Platforms
    • P Versus NP Complexity
    • Quantum Computing
    • Server
    • Software Containers
    • System Scaling
    • Web Crawler
  • Data
    • Data Overview
    • Columnar Databases
    • CSV Data
    • Data Cleaning
    • Data Discovery
    • Data ETL
    • Data Flow
    • Data Lake
    • Data Lakehouse
    • Data Pipeline
    • Data Visualization
    • Data Warehouse
    • Dimensionality Reduction
    • Document Databases
    • Extrapolation
    • Factor Analysis
    • Graph Databases
    • Interpolation
    • JSON
    • Large Data Querying
    • Normalization
    • Outliers
    • Principal Components Analysis
    • Relational Databases
    • Sampling
    • Signal Processing
    • Synthetic Data
    • Vector Databases
  • Linear Algebra
    • Linear Algebra Overview
    • Concatenation
    • Convolution
    • Eigenvalues and Eigenvectors
    • Linear Equations
    • Linear Vector Projection
    • Masking
    • Matrices
    • Pooling
    • Scalars
    • Softmax Function
    • Vectors
  • Models and Modeling
    • Models and Modeling Overview
    • AI Agents
    • Algorithm Libraries
    • Artificial General Intelligence
    • Artificial Narrow Intelligence
    • Artificial Neural Networks
    • Artificial Superintelligence
    • Artificial Universal Intelligence
    • Attention
    • Automated Machine Learning
    • Backpropagation
    • Causal Embedding
    • Classification
    • Cluster Analysis
    • Collaborative Filtering
    • Convolutional Neural Networks
    • Cross Decomposition
    • Curve Fitting
    • Decision Trees
    • Deep Learning
    • Deep Reasoning
    • Diffusion Models
    • Ensemble Learning
    • Explainability
    • Feature Selection
    • Fourier Analysis
    • Foundation Models
    • Gaussian Analysis
    • Generative Adversarial Networks
    • Generative AI
    • Gradient Boosting
    • Graphs
    • Histogram of Oriented Gradients
    • Image Processing
    • K-Means Clustering
    • Large Language Models
    • Linear Regression
    • Logistic Regression
    • Long Short-term Memory
    • Markov Chains
    • Model Alignment
    • Model Categories
    • Model Self Improvement
    • Modeling Process
    • Naive Bayes
    • Nearest Neighbors
    • Probabilistic Graphical Models
    • Prompts and Prompting
    • Random Forest
    • Recurrent Neural Networks
    • Regression Analysis
    • Regularization
    • Reinforcement Learning
    • Retrieval Augmented Generation
    • Supervised Learning
    • Support Vector Machines
    • Transformer Neural Networks
    • Unsupervised Learning
    • Word Embedding
  • Organization
    • Organization Overview
    • Agile Processes
    • Application Selection Process
    • Business Model Components
    • Chief AI Officer
    • Coding
    • Functional Groups
    • Governance
    • Implementation
    • Individuals
    • Research
    • Risks
    • Staying Current
  • Probability
    • Probability Overview
    • Central Limit Theorem
    • Cross Entropy Loss
    • Entropy
    • Independent Events
    • Law of Large Numbers
    • Mutually Exclusive Events
    • Normal Distribution
    • Poisson Distribution
    • Probability Density Function
    • Probability Measure
    • P-Value
  • Programming Constructs
    • Programming Constructs Overview
    • Abstraction
    • Array
    • Attribute
    • Best-first Search
    • Binary Search
    • Block
    • Branch
    • Callback
    • Class
    • Conditional
    • Constructor
    • Container/Collection
    • Dynamic Array
    • Dynamic Programming
    • Encapsulation
    • Exception
    • Expression
    • Function
    • Garbage Collection
    • Greedy Algorithms
    • Hash
    • HTTP Request
    • Identifier
    • Inheritance
    • Inner Class
    • Instance
    • Iterator
    • Keyword
    • Lambda
    • Libraries
    • List
    • Linked List
    • Literal
    • Metaclass
    • Method
    • Mixin
    • Object
    • Operator
    • Overloading
    • Overriding
    • Package
    • Parameter
    • Polymorphism
    • Primitive
    • Programming Process
    • Recursion
    • Reflection
    • Regular Expression
    • Reserved Word
    • Return
    • Sort
    • Statement
    • Switch
    • Table
    • This/Self
    • Token
    • Type
    • Variable
  • Statistics
    • Statistics Overview
    • Accuracy
    • A/B Testing
    • Bias
    • Bias-Variance Tradeoff
    • Confidence
    • Correlation
    • Confusion Matrix
    • Deviation
    • Dispersion
    • Estimator
    • Fairness
    • Loss (Cost) Function
    • Mean Squared Error
    • Hypothesis
    • Prediction and Inference
    • Repeatability
    • Standard Deviation
    • Statistical Power of a Test
    • Variance
  • Trigonometry
    • Trigonometry Overview
    • Cosine Similarity
    • Periodic Functions
    • Trigonometric Functions
  • Glossary and Index
  • Mathematical Symbols
  • Applications
  • Search
  • Blog
  • About the Author
  • Contact

Blog Special: The Accelerating Evolution of Artificial Intelligence

Copyright © 2016-2025 Don Cowan All Rights Reserved

Mathematical Notation Powered by CodeCogs

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Blog RSS

About the Author


Artificial Neural Networks Progress and Popularity

October 09, 2020 in Artificial Neural Network

Artificial Neural Networks (ANNs) have become a very popular Machine Learning model architecture; www.marketsandmarkets.com predicts a 20.5% Compound Annual Growth Rate (CAGR) over the coming years:

Factors contributing to ANN growth include:

  • Big Data - ANNs have the ability to use very large input data sets for model training and the amount of data available is growing rapidly

  • Research and Development - the number of ANN published research papers is growing rapidly as are new applications

  • Model Variations - there are a large number of ANN variations available, including: Convolutional Neural Networks, Deep Learning, Generative Adversarial Networks, Long Short-Term Memory, Recurrent Neural Networks, Reinforcement Learning, Transformer Neural Networks

  • Flexibility - ANNs can be configured using a number of modeling process hyperparameters, including: data features, network graph architectures, training learning rate, model training times to control factors such as the Bias-Variance Tradeoff

  • Hardware Processing Acceleration - technologies such as Graphics Processing Units, Tensor Processing Units, and other AI Accelerators can be used to reduce model training times

In effect, ANNs have become a virtual ‘Swiss Army Knife’ of Machine Learning.

Tags: Artificial Neural Networks
1 Comment

A Note from the Author

October 09, 2020 in Machine Learning

The Science of Machine Learning started as a single blog post on my personal website www.donkcowan.com. It grew to the point where editing and managing it as a single post became impractical. I considered converting the content to a book, however creating a separate website seemed like the better approach as it offers easy content creation, updating and linking.

The website provides an overview of major Machine Learning science concepts, techniques, approaches and more. Each topic is covered on a separate web page and includes references for additional information.

Machine Learning is an amazing combination of Mathematics, Data Science, and Computer Science, including:

  • Calculus

  • Computing Systems

  • Data

  • Graphs

  • Linear Algebra

  • Models and Modeling

  • Organization

  • Probability

  • Programming Constructs

  • Statistics

  • Trigonometry

I’d love to see your reactions to the website. Feel free to contact me with your observations, opinions, suggestions, etc.

Don

1 Comment
Prev / Next

Blog


Featured Posts

Featured
AI Agents.png
Apr 29, 2025
Developments in AI Agents: Q1 2025 Landscape Analysis
Apr 29, 2025
Apr 29, 2025
AI in 2025.png
Apr 1, 2025
The Technical Evolution of AI in 2025
Apr 1, 2025
Apr 1, 2025
Executive Discussing AI.png
Feb 26, 2025
The Hurdles of AI Implementation: Navigating the Challenges for Enterprises
Feb 26, 2025
Feb 26, 2025
CAIO at work.png
Feb 13, 2025
The Chief AI Officer: Driving Enterprise Value in the Age of Artificial Intelligence
Feb 13, 2025
Feb 13, 2025
Worker with Robot.png
Jan 2, 2025
Thriving in the Age of Superintelligence: A Guide to the Professions of the Future
Jan 2, 2025
Jan 2, 2025
Use of AI in Medicine.jpg
Dec 20, 2024
AI in Medicine: Revolutionizing Healthcare
Dec 20, 2024
Dec 20, 2024
Model Fine Tuning.png
Nov 18, 2024
Recent Work on Large Language Model Fine Tuning
Nov 18, 2024
Nov 18, 2024
AI Spring.png
Nov 7, 2024
The New AI Spring: Why an AI Winter is Unlikely This Time
Nov 7, 2024
Nov 7, 2024
Extending Life Expectancy with AI.png
Oct 26, 2024
How AI Can Help Extend Life Expectancy
Oct 26, 2024
Oct 26, 2024
Living and Working with AI.png
Oct 25, 2024
How AI Will Change the Way We Live and Work
Oct 25, 2024
Oct 25, 2024