Learn Machine Learning Resources

This is a subject break down for every topic in machine learning that I’ve studied extensively and a link and short description to the resource that I found most helpful to get either a quick overview, good understanding or deep understanding with intuition.

Some of the resources are abbreviated, you can click on them individually or check the bottom of the page for a full reference list.

The resources are ordered by how helpful I found them for the given topic. Most helpful does not mean easiest to me. If a resources is ordered above another, it means that I have gained deeper knowledge and understanding from it. Next to each resource are onions indicating the theoretical depth and difficulty of that resource on that topic. From one onion (easy, high-level) to three onions (hard, in-depth, theoretical), depending on how deep you want to understand a machine learning algorithm or technique or how well you already know it, you may want to go through them in different order.

I have categorized the topics in the way I think is most straightforward. Please leave a comment or write me on how to improve it or if something doesn’t make sense. I didn’t follow any particular outline.

If a resource is in the main category, it means that it covers topics in all the sub categories. If I mention it in the main category and in the sub category. I want to emphasize how helpful it is and recommend those parts above all other resources in terms of helpfulness.

Supervised Learning

  • Hands on ML 🧅

Regression

  • Linear Regression
    • ISLR 🧅🧅
    • UW Regression 🧅
    • ML Refined 🧅🧅
  • Logistic Regression
    • ISLR 🧅🧅
    • Andrew NG ML 🧅🧅
    • UW Regression 🧅
  • Polynomial Regression
    • ISLR 🧅🧅
    • Hands on ML 🧅
  • Ridge, Lasso, ElasticNet
    • Hands on ML 🧅
  • Line Search

MLE and MAP

  • Probability by Hossein Pishro-Nik 🧅
  • OCW Introduction to Probability 🧅🧅

Decision Trees

  • Hands On ML 🧅
  • Berkeley ML Book 🧅🧅
  • Statquest Decision Trees 🧅

Random Forest

see Ensembling

Bias and Variance

  • ISRL 🧅
  • Bishop, PRML 🧅🧅
  • Hands On ML 🧅
  • UW Regression 🧅🧅

Naive Bayes

Support Vector Machines

  • Hands On ML 🧅
  • My Github Notes 🧅🧅
  • ML Refined 🧅🧅
  • Learning with Kernels 🧅🧅🧅

Ensembling

Bagging and Boosting

Gradient Boosting

  • ML Refined 🧅🧅
  • Statquest Gradient Boosting 🧅

Random Forest

Supervised Learning Metrics

Validation

Feature Engineering

  • Feature Engineering and Selection 🧅🧅
  • Kaggle FE 🧅
  • ML Refined 🧅🧅

Unsupervised Learning

Clustering

  • SKLearn Clustering

Kmeans

  • Hands on ML 🧅
  • UW Clustering 🧅

Hierarchical Clustering

DBSCAN

  • My DBSCAN Tutorial
  • SKLearn DBSCAN
  • Affinity Propagation

Gaussian Mixture Models

PCA

ICA

Recommenders

  • Andrew Ng Recommender Systems🧅🧅

Deep Learning

Neural Networks

  • Andrew Ng ML 🧅🧅
  • Deeplearning.ai Neural Networks and Deep Learning 🧅🧅
  • Hands on ML 🧅🧅

Convolutional Neural Networks

  • Convolutional Neural Networks 🧅
  • DL without a PhD

Transformers

  • Natural Language Processing with Transformers 🧅
  • Deep Learning Magnus Ekman 🧅🧅
  • D2L.ai 🧅🧅
  • AI Coffee Break with Letitia 🧅🧅
Attention Specific
  • Umar Jamil’s Video is Comprehensive and yet easy 🧅🧅
  • Batool Haider’s Youtube🧅🧅
  • Hands on ML 🧅🧅
  • Deep Learning Magnus Ekman 🧅🧅
  • Luis Serrano’s Youtube 🧅

Recurrent Neural Networks

  • Sequence Models Andrew Ng 🧅
  • Deep Learning Magnus Ekman 🧅🧅
  • MLPocket Deep Learning 🧅
  • Andrej Karpathy’s Blog Post 🧅
  • Hands on ML 🧅🧅
  • Deep Learning by Yann Le Cunn 🧅🧅🧅
  • D2L.ai’s section (is very comprehensive but I feel lacks depth in places where explanations should be expanded) 🧅🧅

Generative Deep Learning

  • Hands on ML 🧅
  • David Foster Generative Deep Learning 🧅
  • Ermon Deep Generative Models 🧅🧅🧅
  • Bishop Deep Learning 🧅🧅
  • Deep Learning Book 🧅🧅🧅

Generative Adversarial Networks

  • Generative Adversarial Networks Coursera🧅🧅
  • Luis Serrano GAN🧅

Reinforcement Learning

If you want a good overview of the field, I can recommend just going through Sutton & Barto back to back. It is one of the most beautiful textbooks I’ve ever worked through that solidly builds your understanding of reinforcement learning algorithms and concepts.

Monte Carlo

Temporal Difference

Deep Reinforcement Learning

  • CS285 UC Berkeley DRL 🧅🧅🧅 This course is long, but working through it has deepened my understanding and I had countless “aha!” moments. Sergey Levine is a superb teacher.
  • Deepmind YT DRL Lectures 🧅
  • Deep Reinforcement Learning Hands-On 🧅🧅 There are many good Packt Books, but the ratio of helpful to useless is very low. This one is extremely good. A little light but great explanations, and solid introduction to theory with applications.
  • Foundations of Deep Reinforcement Learning 🧅🧅 This book is one of the best for an introduction into the fundamentals of some DRL Algorithms. However, the further you go, the less useful it becomes unless you want to learn the Authors DRL library which they will end up using more and more. Which again, wouldn’t be a problem if it was a well readable library, but it is not.

Bayesian Methods

  • Pattern Recognition and Machine Learning 🧅🧅
  • Barber Bayesian reasoning and machine learning 🧅🧅🧅 This is one of my favorite books, but it’s very technical and requires a lot of fundamental knowledge and I think one of the few that you really have to read from start to finish. I.e. most textbooks have standalone chapters, you can open it at any one point and easily skip all that has come before. I think this isn’t possible with this book.

Bayesian Inference

  • Probability by Hossein Pishro-Nik 🧅

Gaussian Processes

  • Surrogates 🧅🧅
  • Gaussian Processes Distill.pub 🧅🧅
  • Probability by Hossein Pishro-Nik 🧅

Markov Chain Monte Carlo

  • Blitzstein & Hwang Intro to Probability 🧅
  • YT Ritvikmath MCMC 🧅
  • OCW Introduction to Probability 🧅🧅
  • Ermon CS228 Notes 🧅🧅🧅

Semi-Supervised Learning

Self-Supervised Learning

Active Learning

Time Series

  • Short Course on Time Series Analysis 🧅
  • Forecasting Principles and Practice 🧅
  • Practical Time Series Coursera 🧅🧅🧅
  • Shumway Time Series 🧅🧅🧅
  • RitvikMath TS Playlist 🧅🧅 The AR, MA etc Videos are fantastic, they are somewhat oversimplified though and I recommend reading through one of the books or watching some of the courses above first. Ritvik has a gift of making things easy to understand by simplifying it down to the core concepts. I gave it two onions, because I feel like without theory, unfortunately some of the videos in this playlist will give you a misunderstanding.

Bayesian Deep Learning

  • Probability by Hossein Pishro-Nik 🧅 (MLE and MAP again)
  • Hands-On Bayesian Neural Networks 🧅🧅🧅

NLP

Jurafsky is by far the best resource. For now, it’s free. It’s comprehensive, it builds on foundations given that you have some basic understanding of Probability and Linear Algebra, but even there it explains them.

It goes very far and in the end the concepts become very complex and I felt Jurafsky intended this to be read and understood in sequence. So it’s not one I’d recommend getting a quick overview of one topic (though there are some that work well as standalone resource) within NLP. However, if you have the time and motivation. Use this and supplement it with the other resources below when you get stuck and need another perspective.

Basic Probability Theory & Linear Algebra

  • Probability by Hossein Pishro-Nik 🧅
  • Essential Math for AI 🧅🧅
  • Mutual Information Video by Stats Quest 🧅

Logistic Regression & Naive Bayes

  • see section above

Tokenization & Embeddings

Learn about Tokenization, Skipgram, GloVe, Matrix Factorization, negative Sampling, Embedding, Vector Spaces (overview), Fast Text

  • Sequence Models Andrew Ng 🧅
  • D2L.ai Beam Search Section🧅🧅
  • Natural Language Processing with Transformers 🧅
  • Jurafsky Speech and Language Processing 🧅🧅🧅
  • Chris McCormick Word2Vec🧅
  • Essential Math for AI 🧅🧅
  • TF-IDF Video in UW’s Coursera Course

Beam Search

  • Sequence Models Andrew Ng 🧅
  • Jurafsky Speech and Language Processing 🧅🧅🧅
  • Eisenstein NLP 🧅🧅🧅
  • Hands on ML 🧅

Backpropagation through Time

  • Sequence Models Andrew Ng 🧅

Tasks

NER, POS, Classification, QA, Metrics

  • NLP by Deeplearning.ai 🧅
  • Natural Language Processing with Transformers 🧅
  • Jurafsky Speech and Language Processing 🧅🧅🧅 (Really the best and most comprehensive if you want to learn the meta concepts and understand them in depth)

Transformers

  • see Section above

Recurrent Neural Networks

  • see section above

Fine-tuning LLM’s

Graph Neural Networks

Probability and Statistics

Causality

Computer Vision

Recommenders

Links to the resources

OCW Introduction to Probability YT Ritvikmath MCMC Ermon CS228 Notes Ermon Deep Generative Models Short Course on Time Series Analysis Practical Time Series Coursera Forecasting Principles and Practice [Shumway Time Series][YT Ritvikmath TS Playlist](https://www.youtube.com/playlist?list=PLvcbYUQ5t0UHOLnBzl46_Q6QKtFgfMGc3) Bayesian Neural Networks Distill Draft Hands On Bayesian Neural Networks Bayesian Logistic Regression Statquest Decision Trees Statquest Gradient Boosting Gaussian Processes Distill.pub YT Luis Serrano GAN CS285 UC Berkely DRL [YT Deepmind RL][Deep Reinforcement Learning Hands-On] DL without a PhD SKLearn Clustering SKLearn DBSCAN