Communication dealing with technical matters is best achieved via the dedicated slack workspace where you can communicate with peer students and teaching staff (use this invite link to sign up). Formal messages and grades are broadcasted via blackboard.
The prerequisite for the course is knowledge comparable to that of MATH7501. This includes basic discrete mathematics, calculus and elementary manipulation of vectors and matrices. Feel free to use the 7501 - course reader to brush-up as needed.
The current course, MATH7502, is a linear algebra foundations course focusing on data-science applications. There is also some emphasis on numerical computation via software. It also contains a few basic elements of multi-variable calculus and continuous optimization. At the end of the course students will possess mathematical foundations allowing understanding and execution of activities such as these:
Understanding how basic clustering algorithms work.
Solving linear systems of equations with applications.
Solving smooth non-linear systems of equations by iteration.
Formulating optimality conditions for unconstrained and constrained optimization of smooth multi-variable functions.
Using and understanding least squares approximations and generalizations.
Modelling evolution of linear systems over time and understanding the role of eigenvalues in such evolution.
Understanding linear transformations of multi-variate normal distributions.
Understanding the operation of Principal Component Analysis (PCA).
Understanding the use of singular value decomposition as used for lossy data compression.
Understanding the mathematics of gradient-descent, Gauss-Newton and the Levenberg-Marquardt, non-linear optimization methods. Also with applications to Neural Networks.
Q: Should I take this course?
A: If you have done enough linear algebra and multi-variable calculus and you are able to independently realize how to apply it for machine learning and data science, then maybe there is no need. Otherwise, you should probably join. If you think you understand least squares, principal component analysis, gradient descent and clustering algorithms well then maybe there isn't a need. However, if you want to improve your mathematical understanding of such tools, please join.
Q: What if I haven't done MATH7501?
A: Depending on your background, you can perhaps make up (or review) the needed parts of MATH7501 independently. Consult with the teaching staff. You can also attend the "First Year Maths Support Tutes". They are run every afternoon, in Room 205 of building #41 between 2pm and 4pm.
Q: Is the format of the course similar to the previous offerings of MATH7501 or MATH7502?
A: No. The format is quite different both in comparison to MATH7501 and last year's MATH7502.
The course is mostly (but not solely) taught in "flipped mode". For this students are assigned reading of certain sections from two text books and are required to read prior to the lectures. Then in in the lectures, highlights from the reading are discussed and problems and examples are solved. One exception is the first week focusing on an introduction to the course as well as the Julia language. Note that tutorials are taking place in the first week in the form of a lecture.
[LALFD] The book: Linear Algebra and Learning from Data (2018) by Gilbert Strang. Up to 10% of the book will be supplied via the library. However, students need to obtain further sections of the book independently. Here it is in the university book store. Here are (2 x 24 hour loan) copies at UQ Library. The UQ Library has also scanned a few sections from the book here (these are VI.4, I.1, I.2, I.3, I.4).
The Julia programming language. This is the recommended software for the course. However you can use alternatives if you insist (R, Python, Mathematica, Matlab,...). For basics, see the Julia linear algebra docs. There are several modes in which you can run Julia. An easy option is Julia box. However installing Julia and Jupyter locally on your computer is recommended. See for example this explainer video.
Three homework assignments. These assignments are to be submitted individually with each student submitting a unique assignment (copying assignments will not be tolerated). Nevertheless, students are encouraged to collaborate and discuss the homework assignments in an open and constructive manner. Sharing ideas, helping each other and jointly working towards a goal is great. More details below.
A project report presented via a Jupyter notebook with an accompanying YouTube video.
These are group assignments covering additional material to the core material taught in the course. More details below.
Individual review of peer project reports.
This is an individual review of project reports (of other groups). More details below.
A final exam.
This is a (UQ central) final exam for the course. More details below.
Below is a detailed reading list. The semester has 13 weeks. The first week is an introductory lecture. Then lectures during weeks 2 to 11 require the students to read (and watch videos) prior to the lecture as per the schedule below. Minor refinements of the reading schedule will be communicated via blackboard. Week 13 is for wrap-up and exam review.
Lectures are recorded via UQ's blackboard system, however these recordings don't capture images of the whiteboard. For this, see the course's GitHub page, also containing Jupyter notebooks from class.
Unit 2: Vectors (week 2)
From [VMLS]: 1.1 Vectors, 1.2 Vector addition, 1.3 Scalar-vector Multiplication, 1.4 Inner Product, 1.5 Complexity of vector computations, 2.1 Linear Functions, 2.2 Taylor approximation, C.1.2 Scalar-valued function of a vector, C.1.3 Vector-valued function of a vector.
From [3B1B]:
Unit 3: Using Vectors (week 3)
From [VMLS]: 3.1 Norm, 3.2 Distance, 3.3 Standard deviation, 3.4 Angle, 4.1 Clustering, 4.2 A clustering objective, 4.3 The k-means Algorithm.
From [LALFD]: VI.4 Gradient Descent Toward the Minimum (only some parts of this section are covered).
From [3B1B]:
Unit 4: Matrices (weeks 4 and 5)
From [LALFD]: I.1 Multiplication Ax Using Columns of Ax, I.2 Matrix-Matrix Multiplication AB.
From [VMLS]: 5.1 Linear dependence, 5.2 Basis, 5.3 Orthonormal Vectors, 5.4 Gram-Schmidt Algorithm, 6.1 Matrices, 6.2 Zero and identity Matrices, 6.3 Transpose, addition and Norm, 6.4 Matrix-vector Multiplication. 7.1 Geometric transformations, 7.2 Selectors, 8.1 Linear and affine Functions, 8.2 Linear function models, 8.3 Systems of linear equations, 10.1 Matrix-matrix Multiplication, 10.2 Composition of linear Functions, 10.3 Matrix power. 10.4 QR factorization.
From [3B1B]:
Unit 5: Matrices and Vector Spaces (week 6)
From [VMLS]: 11.1 Left and right inverses, 11.2 inverse, 11.3 Solving linear equations, 11.5 Pseudo-inverse.
From [LALFD]: I.3 The Four Fundamental Subspaces, I.4 Elimination and A = LU, I.5 Orthogonal Matrices and Subspaces.
From [3B1B]:
Unit 6: Spectral Analysis (weeks 7 and 8)
From [LALFD]: I.6 Eigenvalues and Eigenvectors, I.7 Symmetric Positive Definite Matrices, I.8 Singular Values and Singular Vectors in the SVD, I.9 Principal Components and Best Low Rank Matrix, V.4 Covariance Matrices and Joint Probabilities. See also (extra): Chapter 6 from [ILA].
From [3B1B]:
Unit 7: Least Squares #1 (weeks 9 and 10)
From [VMLS]: 12.1 Least squares Problem, 12.2 Solution (to least squares problem), 12.3 Solving least squares Problems, 13.1 Least squares data fitting, 14.1 Classification, 14.2 Least squares classifier, 14.3 Multi-class classifiers.
From [LALFD]: II.2 Least Squares: Four Ways.
Optional video:
Unit 8: Least Squares #2 (weeks 11 and 12)
From [VMLS]: 15.1 Multi-objective least squares, 15.3 Estimation and inversion, 15.4 Regularized data fitting, 15.5 Complexity (regularized data fitting).
From [LALFD]: III.4 Split Algorithms for l^2 + l^1, V.5 Multivariate Gaussians and Weighted Least Squares.
Optional: Guest lecturer lecture notes by Phil Isaac.
Optional video:
Projects are to be carried out in groups of up to 5 people and no less than 3 people per group. Each group needs to choose one project topic from the topics below. A topic has associated reading from [VMLS], [LALFD] and in certain cases [SWJ]. The group then needs to study the material and present key ideas, principals and methods. The presentation is via a Julia Jupyter notebook with an accompanying YouTube video. Here are detailed instructions. Due 18/10/2019.
After projects are submitted. Individual peer reviews of projects will be carried out (you review projects of others). This review (summarized as a written document) is also part of the course assessment. The review questionare is here.
Topic 1: Constrained Optimization
From [VMLS]: 16.1 Constrained least squares problem, 16.2 Solution (to constrained least squares problem), 16.3 Solving constrained least squares problems, 17.2 Linear quadratic control, 17.3 Linear quadratic state estimation, C.3 Lagrange multipliers.
From [LALFD]: VI.2 Lagrange Multipliers = Derivatives of the Components.
From [VMLS]: 18.1 Nonlinear equations and least squares, 18.2 Gauss-Newton Algorithm, 18.3 Levenberg-Marquardt algorithm, 18.4 Nonlinear model fitting.
From [LALFD]: VI.1 Minimum Problems: Convexity and Newton's Method.
From [LALFD]: VI.5 Stochastic Gradient Descent and ADAM, VII.1 The Construction of Deep Neural Networks, VII.2 Convolutional Neural Networks, VII.3 Backpropogation and the Chain Rule.
