Welcome to MATH7502. This course is part of the
Masters of Data Science program at the University of Queensland.
The prerequisite for the course is knowledge comparable to that of
MATH7501. This includes basic discrete mathematics, calculus and elementary manipulation of matrices. Feel free to use the
7501 - course reader to brush-up as needed.
The current course, MATH7502, is a linear algebra foundations course focusing on data-science applications. There is also a focus on numerical computation via software. It also contains a few basic elements of multi-variable calculus and continuous optimization At the end of the course students will possess a strong mathematical foundation allowing understanding and execution of activities such as these:
- Understanding how basic clustering algorithms work.
- Solving linear systems of equations.
- Using the Jacobians for solving smooth non-linear systems of equations by iteration.
- Formulating optimality conditions for unconstrained and constrained optimization of smooth multi-variable functions.
- Using and understanding least squares approximations and generalizations.
- Modelling evolution of linear systems over time and understanding the role of eigenvalues in such evolution.
- Understanding linear transformations of multi-variate normal distributions.
- Understanding the operation of Principal Component Analysis (PCA).
- Understanding the use of singular value decomposition as used for lossy data compression.
- Understanding the mathematics of gradient-descent, Gauss-Newton and the Levenberg-Marquardt, non-linear optimization methods.
As a motivating example for the use of linear algebra in data science and machine learning, see this
recent talk presented at UQ by Prof. Stephen Wright (UW-Madison).
In addition to the final exam, the course assessment includes 5 homework assignments.
Home assignments are to be submitted individually with each student submitting a unique assignment (copying assignments will not be tolerated). Nevertheless, students are encouraged to collaborate and discuss the homework assignments in an open and constructive manner. Sharing ideas, helping each other and jointly working towards a goal is great.
The course is coordinated by
Yoni Nazarathy (
y.nazarathy@uq.edu.au) and the tutor is
Maria Kleshnina (
m.kleshnina@uq.edu.au). The homework assignments were created with the help of Zhihao Qiao.
The course is mostly (but not soley) taught in "flipped mode". It relies on materials from the following:
- [VMLS] The book: Vectors Matrices and Least Squares (2018) by Stephen Boyd and Lieven Vandenberghe. You can use the free on-line version or you can order the book.
- [STRANG] The book: Introduction to Linear Algebra, Fifth Edition (2016) by Gilbert Strang. Here it is in the university book store. Selected sections are in the UQ Library. These are Sections 3.1, 5.1 and Chapter 7.
- [3B1B] The video series: Essence of linear algebra by 3Blue1Brown (Grant Sanderson).
- [LECT] Complementary lectures (presented in class by Yoni Nazarathy), including material (not found in 1-3) covering: Matrix Exponentials, Cramer's rule, Fourier Analysis, Principal Component Analysis, software packages and related terms.
- The Julia programming language. This is the recommended software for the course. However you can use alternatives if you wish (R, Python, Mathematica, Matlab,...). For basics, see the Julia linear algebra docs. Another resource for some code examples is this draft book co-authored with Hayden Klok [JSTAT]. Code examples from the book are available in this Github repo. There are several modes in which you can run Julia - the recommended one is Julia box. Also note that we are using Julia 0.6 (not far from 1.0). There was a significant change in Julia linear algebra between 0.5 and 0.6. For extra joy - see this video.
Material from [VMLS], [STRANG] and [3B1B] is used for learning in flipped mode (this means still attending class). Students cover this material independently out of class. Then it is discussed in class, with computational activities and exercises solved. Only a a few selected parts are in [LECT] mode, where a lecture is presented in class, not requiring prior preperation.