Machine Learning

Instructor

Fabio A. González
Profesor Titular
Depto de Ingeniería de Sistemas e Industrial
Universidad Nacional de Colombia

Course goal

The main goal of Machine Learning (ML) is the development of systems that are able to autonomously change their behavior based on experience. ML offers some of the more effective techniques for knowledge discovery in large data sets. ML has played a fundamental role in areas such as bioinformatics, information retrieval, business intelligence and autonomous vehicle development.

The main goal of this course is to study the computational, mathematical and statistical foundations of ML, which are essential for the theoretical analysis of existing learning algorithms, the development of new algorithms and the well-founded application of ML to solve real-world problems.

1 Learning Foundations

1.1 Introduction

1.2 Bayesian decision theory

1.3 Estimation

1.4 Linear models

1.5 Design and analysis of ML experiments

2 Kernel Methods

2.1 Kernel methods basics

2.2 Support vector learning

3 Neural Networks

3.1 Neural networks basics

3.2 Deep learning

3.3 Convolutional neural networks

3.4 Recurrent neural networks

3.5 Deep generative models

3.6 Transformer networks

4 Probabilistic Programming

4.1 Bayesian Methods

4.2 Monte Carlo inference

4.3 Variational Bayes

5 Quantum Machine Learning

5.1 Learning with Density Matrices and Random Fourier Features

• Participation 10%
• Assignments 20%
• Quizzes 20%
• Exam 30%
• Final project 20%

Grades

References

• [Alp14] Alpaydin, E. Introduction to Machine Learning, 3Ed. The MIT Press, 2014
• [Mur12] Murphy, Kevin P. Machine learning: a probabilistic perspective. The MIT Press, 2012.
• [Barber2013] Barber, David, Bayesian Reasoning and Machine Learning, Cambridge University Press, 2013.
• [Bis06] Bishop, C. Pattern Recognition and Machine Learning. Springer-Verlag, 2006
• [HTF09] Hastie, T. and Tibshirani, R. and Friedman. The elements of statistical learning: data mining, inference, and prediction, Springer, 2009
• [GBC2016] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
• [Mit97] Mitchell, T. M. 1997 Machine Learning. 1st. McGraw-Hill Higher Education.
• [DHS00] Duda, R. O., Hart, P. E., and Stork, D. G. 2000 Pattern Classification (2nd Edition). Wiley-Interscience.
• [SC04] Shawe-Taylor, J. and Cristianini, N. 2004 Kernel Methods for Pattern Analysis. Cambridge University Press.
• [SS02] Scholkopf, B. and Smola, A.J., 2002, Learning with kernels, MIT Press.
• [OCW-ML] 6.867 Machine Learning, Fall 2006, MIT OpenCourseWare.
• [STANFD-ML] Andrew Ng, CS229 Machine Learning, Stanford University
• [FundDL] Raúl Ramos, Fundamentos de Deep Learning, Universidad de Antioquia, 2021

Additional resources

• SciPy: scientific, mathematical, and engineering package for Python
• scikit-learn: machine learning Scipy add-on
• Kaggle: datascience competition, many interesting data sets and different competitions with prizes.
• Coursera Machine Learning Course: one of the first (and still one of the best) machine learning MOOCs taught by Andrew Ng.
• Stanford Statistical Learning Course: an introductory course with focus in supervised learning and taught by Trevor Hastie and Rob Tibshirani.

Week	Topic	Material	Assignments
Mar 7-14	1.1 Introduction	Asynchronous Class: Brief Introduction to ML (slides)(video 1, video 2, video 3) Linear Algebra and Probability Review (part 1 Linear Algebra, part 2 Probability) [Alp14] Chap 1 (slides)	Practice problems 1 Practice problems 2
Mar 21	1.2 Bayesian decision theory	Asynchronous Class (video 1) (video 2) [Alp14] Chap 3 (slides) (annotated slides)	Practice problems 3
Mar 28	1.3 Estimation	Asynchronous Class (video 1) (video 2) [Alp14] Chap 4, 5 (slides) Bias and variance (Jupyter notebook)	Practice problems 4
Apr 4	1.5 Design and analysis of ML experiments	Asynchronous class (video 1) (video 2) [Alp14] Chap 19 (slides) Additional video: Hypothesis testing (video)	Assignment 1
Apr 18	2.1 Kernel methods basics	Asynchronous class (video 1) (video 2) Introduction to kernel methods (slides) [Alp14] Chap 13 (slides) [SC04] Chap 1 and 2	Practice problems 5
Apr 25	2.2 Support vector learning	Asynchronous class (video 1) (video 2) [Alp14] Chap 13 (slides) An introduction to ML (Lecture 4, pp 146), Smola Support Vector Machine Tutorial, Weston Máquinas de vectores de soporte y selección de modelos (Jupyter Notebook)	Practice problems 6
May 2	3.1 Neural network basics	Asynchronous class (video) Neural networks, Representation Learning and Deep Learning (slides) [Alp14] Chap 11 (slides) Quick and dirty introduction to neural networks (Jupyter notebook)	Assignment 2
May 9	3.1 Neural network basics	Asynchronous class (video) [Alp14] Chap 11 (slides) Backpropagation derivation handout	Practice problems 7
May 16-23	3.2 Deep learning	Asynchronous class (video 1, video 2) Representation Learning and Deep Learning (slides) [GBC2016] Chap 6 Deep learning frameworks (slides) Introduction to TensorFlow (Jupyter notebook) Neural Networks in Keras (Jupyter notebook)	Practice problems 8
May 30	3.3 Convolutional neural networks	Asynchronous class (video 1, video 2) CNN for image classification in Keras (Jupyter notebook) ConvNetJS demos Feature visualization	Practice problems 9
Jun 6	3.4 Recurrent neural networks	Asynchronous class (video 1, video 2) CNN for text classification handout LSTM language model handout [FunDL] Sect 5.0-5.3	Project proposal: 3 persons per group, maximum 2 pages describing problem, objectives and method (12/06/22)
Jun 13	3.5 Transformer networks	[FunDL] Sect 5.8 Videos: [FunDL] 15. Self-attention [FunDL] 16. Bert	Assigment 3: (25/06/22) [FunDL] LAB 5.1 - Time series prediction [FunDL] LAB 5.3 - Transformer - Bert
Jun 20	3.5 Deep generative models	Alexander Amini, Deep generative models (slides, video) (from MIT 6.S191) Deep generative models (Jupyter notebook)
July 6	Final project		Final project