Class Meetings: Not offered this semester.
Registration: Enrollment and registration for the course is handled by Christine Keat in the Department of Statistics and Operations Research. Ms. Keat can be reached by email at crikeat@email.unc.edu.
Instructor: Andrew B. Nobel, Department of Statistics and Operations Research
Office: Hanes 308 Email: nobel@email.unc.edu Phone: 919.962-1352.
Office Hours: TBA
Course Prerequisites: STOR 634, 654, and 664. It will be helpful, but not necessary, if students already have some familiarity with machine learning methods for classification and linear regression.
Overview: Generally speaking, machine learning is the study and development of methods that use existing data to make decisions or predictions about new data, or that identify meaningful structure in a data set. In most cases, machine learning methods are built around models that are not tailored to the specifics of the problem at hand. Machine learning draws on basic ideas from a number of disciplines, including statistics, computer science, and optimization, and has points of contact with data mining and the study of “big data”.
Audience and Goals: STOR 767 is an intermediate graduate level course in machine learning. The course is targeted towards PhD students in the Statistics and Operations Research (STOR) department, but may be appropriate for MS students in STOR, and for students in other departments with appropriate mathematical and statistical backgrounds. The goal of the course is to introduce students to, and provide them with working knowledge of, some of the core theory and methods in machine learning. We will discuss and explore key ideas of regularization, sparsity, and complexity, with an emphasis on topics that have a statistical component, or that are used in statistical applications. In some cases, we will look carefully at the theory supporting a method or family of methods, and provide rigorous proofs. In other cases, we will focus primarily on the method and state supporting theoretical results as needed. The coverage of classical subjects such as ridge regression and logistic regression that are covered in introductory courses will be brief, so that more time can be devoted to modern topics such as compressed sensing and graphical models.
Textbook: There is no official textbook for the course. Lectures will be based on material from a variety of sources, including on-line tutorials and surveys.
Grading: The course grade will be based on homework assignments, and a final project.
Homework Assignments | 50% |
Final Project | 50% |
Policy for Homework Assignments: Homework will be posted on the course web page, and will be due periodically througout the semester. Assignments will be collected at the beginning of class on the day that they are due, so please be prepared to turn in your homework at that time. Each assignment will be graded. Late/missed assignments will receive a grade of zero. In computing a student’s overall homework score the two lowest assignment scores will be dropped. Students are allowed to work on the homework assignments with other students, but each student should prepare the final version of his/her assignment on their own.
Classroom Protocol: Please show up on time, as late arrivals tend to disturb those already present. Reading of newspapers and the use of laptops, tablets, and phones, is not permitted during class.
Specific Prerequisites:
Statistics: Sample vs. population quantities. Basic understanding of statistical inference, in particular, maximum likelihood estimation and the method of moments.
Real Analysis: Infima and suprema, limits, continuous functions, integration and differentiation of functions of several variables, Taylor series, convexity.
Probability: Joint and conditional densities and probability mass functions. Definition and basic properties of expectations, variance, covariance, correlation. Basic discrete and continuous distributions and their properties. Inequalities: Jensen, Markov, Chebyshev, Hoeffding, and Bernstein.
Linear algebra: Vector spaces, dimension, subspaces, matrix addition and multiplication, determinant and trace, inverse, eigenvectors, eigenvalues, symmetric matrices, non-negative and positive definite matrices, rank.
Tentative Syllabus
Big Picture
Unsupervised vs. Supervised Learning
Close connections between learning methods and optimization problems
Importance and pervasiveness of convexity
Notions of complexity for a family of models,relations between complexity and inference
Sparsity and regularization
Primary Material
Overview of principal component analysis and singular value decomposition
Clustering and community detection: k-means, hierarchical clustering, spectral clustering
Convex sets and functions
Johnson-Lindenstrauss lemma
Classification: Bayes risk and Bayes rule, nearest neighbor rules, classification trees
Empirical risk minimization (ERM): Rademacher complexity, shatter coefficients and
VC dimension, error bounds
Support vector machines (SVM) and kernel machines
Boosting and bagging
Compressed sensing
Online learning and prediction: exponential weighting
Other Material
Graphical models
Variational inference
Penalized regression and the LASSO
First order methods for minimizing smooth convex functions
Stochastic approximation
Honor Code: Students are expected to adhere to the UNC honor code at all times. Violations of the honor code will be prosecuted.