Instructor-In-Charge: Chittaranjan Hota, Ph.D (hota[AT]hyderabad.bits-pilani.ac.in)
Teaching Assistants: Anish Shandilya, Paryetri Banerjee, Harsha Varun, Shashank S, Chaitra C
Schedule: T(4), Th(4,10), Venue: F-106
Scope and Objectives of the course: This course is an undergraduate course on Machine Learning. ML is the sub-field of Artificial Intelligence. It helps engineers build automated systems that learn from experiences or examples. It helps machines make data-driven decisions. For example, Google Maps for navigation uses the route network, real-time traffic characteristics, time of travel etc. to predict an appropriate path for you using ML algorithms. ML is a multi-disciplinary field, with roots in Computer science, and Mathematics. ML methods are best described using linear and matrix algebra and their behaviours are best understood using the tools of probability and statistics. According to the latest estimates, 328 million terabytes of data are created daily. With this increasing amounts of data, the need for automated methods for data analysis continues to grow. The goal of ML is to develop methods that can automatically detect patterns in data, and then use the uncovered patterns to predict the future outcomes of interest. This course will cover many ML models and algorithms, including linear models, multi-layer neural networks, support vector machines, density estimation methods, Bayesian belief networks, mixture models, clustering, ensemble methods, and reinforcement learning. The course objectives are the following:
- To select and apply an appropriate supervised learning algorithm for classification problems like Naïve Bayes, SVM, Logistic regression, Neural networks etc., To select and apply an appropriate supervised learning algorithm for regression problems like Linear regression, Ridge regression, Non-parametric kernel regression etc.
- To select and apply an appropriate un-supervised learning algorithm for clustering, linear and non-linear dimensionality reduction etc., To understand ML principles and techniques like Model selection, Under-fitting, Over-fitting, Cross-validation, Regularization etc.
- To test run appropriate ML algorithm on real and synthetic datasets and interpret their results.
Text Books: T1: Christopher Bishop: Pattern Recognition and Machine Learning, Springer-Verlag New York Inc., 2006; T2: Tom M. Mitchell: Machine Learning, The McGraw-Hill, Indian Edition, 2017; Reference Books: R1: Kevin Murphy: Machine Learning: A Probabilistic Perspective, MIT Press, 2012; R2: Shai Shalev-Shwartz and Shai Ben-David: Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014; R3: Ethem Alpaydin: Introduction to Machine Learning, 3rd Edition, MIT Press, 2014.
Lecture Plan:
Lect. # | Learning objectives | Topics to be covered | Chapter in the Text Book |
1 | Course administration | Course Administration, Motivation and ML Frameworks. | T2(1), Lecture Slides |
2 | Overview of ML | Supervised/Unsupervised/RL, Classification/ Regression, General Approach. | R3(1.1, 1.2) |
3 - 4 | Supervised Learning - I | Concept Learning: Version space and Candidate elimination algorithm. | T2(2.2, 2.5) |
5 - 6 | Supervised Learning - II | Decision Tree Learning: Tree Representation, Types of problems suitable for DT learning, Learning algorithm. | T2(3.2, 3.3, 3.4) |
7 - 8 | Evaluating a model | Bias, Cross-validation, Precision-Recall, ROC Curve. | T1(1.3), R3(19.6, 19.7) |
9 - 11 | Linear Models for Regression | Linear regression, Logistic regression, Gradient Descent, GD Analysis, SGD. | T1(3.1, 3.2), R1(8.1-3, 8.6) |
12 - 14 | Linear Models for Classification | Discriminant functions, least squares, Fisher’s Linear Discriminant. | T1(4.1) |
15 - 17 | Naïve Bayes | Generative Vs Discriminative models, Maximum A Posteriori (MAP) Vs Maximum Likelihood (ML) | T1(4.2, 4.3), T2(6.1-6.10) |
18 - 21 | Neural Networks-I | Perceptron Training, Multi-layer Perceptron(MLP): Components, Activations, Training: SGD, Computing Gradients, Error Backpropagation. | T1(5.1-5.4) |
22 - 25 | Neural Networks-II | Regularization, Data Augmentation, Convolutional Networks: CNNs, RNNs; Generative Models: Autoregressive, GANs. | T1(5.5), Lecture Slides |
26 – 29 | Instance-based Learning: Kernels & SVMs | k-Nearest Neighbour Learning, Constructing Kernels, Radial Basis Function Networks, Maximum margin classifiers. | T2(8.2), T1(6.1-6.3, 7.1) |
30 - 33 | Graphical Models | Bayesian Networks: Training, Structure learning, Inferences, Undirected models. | T1 (8.1, 8.4.1), T2(6) |
34 - 35 | Un-supervised Learning - I | Mixture Models and EM: K-means Clustering, Gaussian Mixture Models, EM for GMM. | T1 (9.1, 9.2) |
36 - 37 | Un-supervised Learning - II | Dimensionality Reduction, Principal Component Analysis (PCA). | T1(12.1) |
38 – 39 | Combining Models | Bayesian-model averaging, Boosting, Tree-based Models. | T1(14.1 -14.4) |
40 - 41 | Reinforcement Learning | Markov Decision Process, Value Iteration, Policy Iteration, Q-learning. | T1 (13.1), T2(13.3) |
Evaluation Scheme:
Component | Duration | Date & Time | Weightage | Nature of Component |
Mid-Semester Exam | 90 mins | 14/03/3024 (2:00 to 3:30pm) | 30% | Closed Book |
Home Assignments/ Projects (coding) | - | To be announced | 20% | Open Book |
Two announced quizzes | 30 mins each | Second week of Feb, and First week of April 2024. | 10% | Open Book |
Comprehensive Exam | 3 Hrs | 15/05/2024 (FN) | 40% | Closed Book |
Note: Minimum 40% of the evaluation component will be conducted before the mid semester grading; Chamber Consultation Hours: Monday (5 - 6 PM), H-137; Make-up Policy: Prior permission of the Instructor-In-Charge is required to get make-up on any evaluation component. Genuine requests will only be considered; Notices: All notices about the course will be put on google class page; Academic Honesty and Integrity Policy: Academic honesty and integrity are to be maintained by all the students throughout the semester and no type of academic dishonesty is acceptable.
Class Presentations:
- Course Administration and Motivation: Click here.
- Machine Learning Overview: Click here.
- Machine Learning Frameworks: Click here.
- Symbolic Learning (Version Space): Click here.
- Symbolic Learning (Decision Trees/ Random Forests): Click here.
- Model Evaluation (Bias, Variance, Cross-validation, Confusion Matrix, OOB metric etc.): Click here.
- Regression Models (Linear regression, Logistic Regression, Gradient Descent, Stochastic GD): Click here.
- Linear Discriminant Functions, Least Squares for Classification, Fisher's Discriminant Function: Click here.
- Probabilistic approach to Machine Learning (Bayesian Networks, Naive Bayes): Click here.
- Neural Networks-I (Connectionist Models: Perceptron, Multilayer Perceptron, Back Propagation): Click here.
- Neural Networks-II (Regularization, Data Augmentation, CNN, RNN, Autoregressive and GANs): Click here.
- Instance-based and Kernel-based Learning (k-NN, SVM): Click here.
- Un-supervised learning (K-Means, Gaussian Mixture Models, Principal Component Analysis): Click here.
Home Assignments:
- Data Exploration and Pre-Processing (submission deadline: 25th Jan 2024): Click here.
- TensorFlow's Decision Forests: Random Forests and Gradient Boosted Trees (submission deadline: 12th Feb 2024):Click here.
- Linear and Logistic Regression using TensorFlow (submission deadline: 1st March 2024): Click here.
- Gaussian Naïve Bayes Classifier using Scikit Learn (submission deadline: 5th April 2024): Click here.
- Back Propagation Neural Network for Regression Task using PyTorch (submission deadline: 12th April 2024): Click here.
- Mini-Project on Convolutional Neural Networks (CNNs) (Submission deadline: 30th April 2024): Click here.
--------------------------------------------
