CS182
Home

CS182: Introduction to Machine Learning

Course Descriptions

 

Machine learning (ML) is the science of making computer artifacts improve their performance without requiring humans to program their behavior explicitly. Machine learning has accomplished successes in a wide variety of challenging applications, ranging from computational molecular biology to computer vision to social web analysis.

CS182 Introduction to Machine Learning is an undergraduate-level introductory course in machine learning. It is not only suitable for students who want to apply principled machine learning techniques competently to their application-oriented research areas, but is also suitable for students pursuing or planning to pursue research in machine learning or other related areas that focus on model and algorithm development.

(Developed based on the course materials by Ethem Alpaydin.)

Announcements

  1. Piazza: https://piazza.com/shanghaitech.edu.cn/fall2021/cs182

  2. Gradescope: https://www.gradescope.com/courses/324113

Prerequisites

  1. Compulsory: Linear Algebra, Calculus, Probability and Statistics, Programming.

  2. Recommended Postrequisites: Matrix Analysis and Computations, Convex Optimization, Machine Learning.

Textbooks and Optional References

Textbooks

  1. Ethem Alpaydin, Introduction to Machine Learning (4th Edition), The MIT Press (ACML book series), 2020.

  2. Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2006.

  3. Richard O. Duda, Peter E. Hart, and David G. Stork, Pattern Classification (2nd Edition), Wiley, 2000.

References

  1. Aaron Courville, Ian Goodfellow, and Yoshua Bengio, Deep Learning, The MIT Press (ACML book series), 2016.

  2. Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction (2nd Edition), The MIT Press (ACML book series), 2018.

  3. Tom Mitchell, Machine Learning, McGraw Hill, 1997.

  4. Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd Edition), Springer, 2009.

  5. Zhihua Zhou, Machine Learning, Tsinghua University Press, 2016. (a ref. in Chinese)

  6. Hang Li, Statistical Learning Methods (2nd Edition), Tsinghua University Press, 2019. (a ref. in Chinese)

  7. Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.

  8. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning (2nd Edition), The MIT Press, 2018.

AI Labs

  1. DeepMind; OpenAI; MetaAI/FAIR; BlackRockAI; IBMAI.

ML Practice Platform

  1. Kaggle; Tianchi.

Schedule (Subject to Change)

Topics

  1. Topic 0: Overview

  2. Topic 1: ML Introduction; Mathematical Foundations of ML (Linear Algebra, Probability and Statistics, Optimization Theory, Information Theory, etc.)

  3. Topic 2: Bayesian Decision Theory

  4. Topic 3: Parameter Estimation for Generative Models

  5. Topic 4: Linear Discrimination Models

  6. Topic 5: Feedforward Neural Networks

  7. Topic 6: Support Vector Machines

  8. Topic 7: Model Assessment and Selection

  9. Topic 8: Deep Learning Models

  10. Topic 9: Recurrent Neural Networks

  11. Topic 10: Clustering and Mixture Models

  12. Topic 11: Nonparametric Methods

  13. Topic 12: Decision Trees

  14. Topic 13: Dimensionality Reduction

  15. Topic 14: Matrix Factorization

  16. Topic 15: Probabilistic Graphical Models

  17. Topic 16: Ensemble Learning

  18. Topic 17: Reinforcement Learning

  19. Guest Lecture: Deep Reinforcement Learning and Game AI [slides]

  20. Topic 18: Review

Selected Topics

  1. S-Topic 1: Probabilistic Graphical Models

  2. S-Topic 2: Probabilistic Topic Models

  3. S-Topic 3: Hidden Markov Models

  4. S-Topic 4: State Space Models

  5. S-Topic 5: Reinforcement Learning

  6. S-Topic 6: Numerical Optimization in ML

Note: All course materials are available on Piazza.com.

Assessment

30% assignments, 40% final exam, 30% final project.

Academic Integrity Policy

Group study and collaboration on problem sets are encouraged, as working together is a great way to understand new materials. Students are free to discuss the homework problems with anyone under the following conditions:

  • Students must write down their own solutions. Plagiarism is never allowed. Similar answers, MATLAB codes, etc., found in HWs will invite you into suspected plagiarism investigation.

  • Students must list the names of their collaborators (i.e., anyone with whom the assignment was discussed).

  • Students can not use old solution sets from other classes under any circumstances, unless the instructor grants special permission.

Students are encouraged to read the ShanghaiTech Policy on Academic Integrity.