Machine Learning

Course Description

Objective

The main goal of Machine Learning (ML) is the development of systems that are able to autonomously change their behavior based on experience. ML offers some of the more effective techniques for knowledge discovery in large data sets. ML has played a fundamental role in areas such as bioinformatics, information retrieval, business intelligence and autonomous vehicle development.

The main goal of this course is to study the computational, mathematical and statistical foundations of ML, which are essential for the theoretical analysis of existing learning algorithms, the development of new algorithms and the well-founded application of ML to solve real-world problems.

Methodology

Professor's lectures on fundamental topics

Practical assignments and exercises to be solved by students

Technical papers' review and presentation by students

Final project

Written and practical tests. Students must show a good grasp of concepts and skills covered in the course.

Contents

Topic	Material	Assignments	Presentations
1. Introduction	Brief Introduction to ML (notes) [Mit97] Cap 1 [Alp04] Cap 1,2 [DHS00] A.1, A.2	Assignment 1	Videos: The great robot race Introduction to ML (notes)
2. Bayesian decision theory 2.1 A review of probability theory 2.2 Classification 2.3 Lost and risk 2.4 Naive Bayes classifier 2.5 Bayesian Networks 2.6 Maximum likelihood estimation 2.7 Bayesian estimation 2.8 Parametric Classification 2.9 Expectation Maximization	[Alp04] Chap 3 (notes) Chap 4 (notes) Chap 7 (Sect. 7.4) (EM notes) (Parametric notes) [DHS00] Chap 3 [Tenenbaum06]	Assignment 2 Assignment 3	Video: Graphical models (notes) Paper presentation: [Friedman99] Alejandro Riveros [Goldenberg05] Camilo López [Dietterich02] Jeison Gutierres
3. Kernel methods 3.1 The kernel trick 3.2 Kernel ridge regression 3.3 Kernel functions 3.4 Other kernel Algorithms 3.5 Kernels in complex structured data	[SC04] Chap 2 Introd. to kernel methods (notes) Matlab examples (.pdf,.m)		Video: Support Vector Classification (part 4) Paper presentation: [Grauman05] María E. Rojas [Lodhi02] David Becerra [Leslie02] Daniel Restrepo
4. Regularization and model complexity 4.1 Bias vs variance tradeoff 4.2 Risk and empirical risk 4.3 Complexity measures	Chap 4 (Sect. 4.3, 4.7, 4.8) (bias/var notes)	Assignment 4	Video: Iterative Regularization Scheme and Early Stopping in Learning from Examples Paper presentation: [Roberts00] José Luis Morales [Lawrence96] Javier Vargas [Mehta95] Alexander Cerón
5. Performance evaluation 5.1 Performance evaluation in supervised learning 5.2 Performance evaluation in unsupervised learning 5.3 Hypthesis testing	[Alp04] Cap 14 [TSK05] Chap 8 (Sect. 8.5)	Assignment 5	Video: An Empirical Comparison of Learning Methods and Metrics Paper presentation: [Hand01] Carlos Garzón [Domingos99] Jimmy Cifuentes [Japkowicz02] Omar Erazo
6. Combining multiple classifiers 6.1 Voting 6.2 Error correcting codes 6.3 Bagging 6.4 Boosting	[Alp04] Cap 15		[Dietterich00] Sandra Tocarruncho [Oza05] Emir Cortés [Mason00] Edwin Niño
7. Clustering and density estimation			Miguel Dussan: Web Clustering Rolando Beltrán: Bi-Clustering Wilson Soto: Sequence Clustering
Final Exam (19/6/08)
Project (08/7/08)

Topic

Material

Assignments

Presentations

1. Introduction

Brief Introduction to ML (notes)
[Mit97] Cap 1
[Alp04] Cap 1,2
[DHS00] A.1, A.2

Assignment 1

Videos:
The great robot race
Introduction to ML (notes)

2. Bayesian decision theory
2.1 A review of probability theory
2.2 Classification
2.3 Lost and risk
2.4 Naive Bayes classifier
2.5 Bayesian Networks
2.6 Maximum likelihood estimation
2.7 Bayesian estimation
2.8 Parametric Classification
2.9 Expectation Maximization

[Alp04]
Chap 3 (notes)
Chap 4 (notes)
Chap 7 (Sect. 7.4) (EM notes)
(Parametric notes)
[DHS00] Chap 3
[Tenenbaum06]

Assignment 2
Assignment 3

Video:
Graphical models (notes)
Paper presentation:
[Friedman99] Alejandro Riveros
[Goldenberg05] Camilo López
[Dietterich02] Jeison Gutierres

3. Kernel methods
3.1 The kernel trick
3.2 Kernel ridge regression
3.3 Kernel functions
3.4 Other kernel Algorithms
3.5 Kernels in complex structured data

[SC04] Chap 2
Introd. to kernel methods
(notes)
Matlab examples (.pdf,.m)

Video:
Support Vector Classification (part 4)
Paper presentation:
[Grauman05] María E. Rojas
[Lodhi02] David Becerra
[Leslie02] Daniel Restrepo

4. Regularization and model complexity
4.1 Bias vs variance tradeoff
4.2 Risk and empirical risk
4.3 Complexity measures

Chap 4 (Sect. 4.3, 4.7, 4.8)
(bias/var notes)

Assignment 4

Video:
Iterative Regularization Scheme and Early Stopping in Learning from Examples
Paper presentation:
[Roberts00] José Luis Morales
[Lawrence96] Javier Vargas
[Mehta95] Alexander Cerón

5. Performance evaluation
5.1 Performance evaluation in supervised learning
5.2 Performance evaluation in unsupervised learning
5.3 Hypthesis testing

[Alp04] Cap 14
[TSK05] Chap 8 (Sect. 8.5)

Assignment 5

Video:
An Empirical Comparison of Learning Methods and Metrics
Paper presentation:
[Hand01] Carlos Garzón
[Domingos99] Jimmy Cifuentes
[Japkowicz02] Omar Erazo

6. Combining multiple classifiers
6.1 Voting
6.2 Error correcting codes
6.3 Bagging
6.4 Boosting

[Alp04] Cap 15

[Dietterich00] Sandra Tocarruncho
[Oza05] Emir Cortés
[Mason00] Edwin Niño

7. Clustering and density estimation

Miguel Dussan: Web Clustering
Rolando Beltrán: Bi-Clustering
Wilson Soto: Sequence Clustering

Final Exam (19/6/08)

Project (08/7/08)

Grading

Assignments 40%

Exams

Midterm 15%
Final 15%

Presentation 15%

Final project 15%

References

[Alp04] Alpaydin, E. 2004 Introduction to Machine Learning (Adaptive Computation and Machine Learning). The MIT Press.

[Mit97] Mitchell, T. M. 1997 Machine Learning. 1st. McGraw-Hill Higher Education.

[DHS00] Duda, R. O., Hart, P. E., and Stork, D. G. 2000 Pattern Classification (2nd Edition). Wiley-Interscience.

[HTF01] Hastie, T. and Tibshirani, R. and Friedman. 2001 The elements of statistical learning: data mining, inference, and prediction. Springer.

[SC04] Shawe-Taylor, J. and Cristianini, N. 2004 Kernel Methods for Pattern Analysis. Cambridge University Press.

[TSK05] Pang-Ning Tan, Michael Steinbach, Vipin Kumar, 2005, Introduction to Data Mining, Addison-Wesley.

[OCW-ML] 6.867 Machine Learning, Fall 2002, MIT OpenCourseWare.

Additional references

MacKay, "Information Theory, Inference, and Learning Algorithms", 2003. (on-line version)

Back

Machine Learning
Maestría en Ingeniería - Ingeniería de Sistemas y Computación

Contents

Course Description

Objective

Methodology

Contents

Grading

References

Additional references

Assignments

Resources

Machine Learning Maestría en Ingeniería - Ingeniería de Sistemas y Computación

Contents

Course Description

Objective

Methodology

Contents

Grading

References

Additional references

Assignments

Resources

Machine Learning
Maestría en Ingeniería - Ingeniería de Sistemas y Computación