This course is designed to build understanding of programming, software architecture, and general software development in the context of mathematical, statistical, and machine learning applications. Students learn the basic elements of software development in the context of mathematics and data-science. The course teaches students how to implement algorithms from numerical mathematics, computer algebra systems, simulation, and machine learning. This is all done using solid software development practices that allow for organized maintainable and extendable software.
This is the second offering of this course. You may also see last year's course.
The course is aimed at students in their fourth semester (or above) of a mathematics degree. It assumes knowledge of calculus, linear algebra, elementary statistics, basic probability, and elementary concepts of discrete mathematics. It also assumes some prior experience in scientific/computational/statistical scripting, using languages such as R, MATLAB, Python, or similar. No specific detailed knowledge of such programming is assumed, however it is assumed that students have seen code, variables, loops, conditionals, and similar constructs previously.
The focus of the course is programming of systems that involve mathematics and mathematical modelling. This includes the interface of mathematical principles and a programming language as well as tooling and the experience of using solid basic practices that lead to organized and efficient code. It is not a course about a specific programming language. Nevertheless one language needs to be used and studied. The language for this course is the Julia language. Julia is a modern compiled programming language that in many ways is easy to work with like other scripting languages, however it also allows to create very efficient code. In general, the Julia language and the surrounding eco-system focuses on scientific computing and mathematics, and hence it is a good fit for this course. For reference, here are some courses at other universities that use Julia: Computational Thinking at MIT, Parallel Computing and Scientific Machine Learning at MIT, Introduction to Matrix Methods at Stanford, and Introduction to Machine Learning at Stanford among others. Note that these courses generally have goals that differ from the current course, yet their innovative use of Julia is of interest to us.
The course content is broken into 7 study units and 6 items of assessment, details of which are described below.
The course is delivered via weekly lectures which include three hours per week. The lectures involve both live demonstrations and theory. There are also 3 (guest) one hour perspective lectures. These perspective lectures are aimed at presenting students with further insights about mathematicians and statisticians working with software in industry.
In addition to the lectures there are also practicals aimed at smaller groups and operating both in flexible delivery and external delivery modes. The practicals are aimed to help the students prepare the assessment items and present the students with a chance to work live with help next to them. Practicals are scheduled on every week including the first week. Practicals involve both content delivery (see practical notebooks A-G). They are also a time for students to receive help from the tutors.
The use of a variety of features of a programing language (Julia in this case).
Technical tools such as: Unix, git, IDEs, Jupyter.
Mathematical and statistical algorithmic concepts, their theoretical analysis, and implementation.
Solid software development practices - with a view towards employability.
The lectures, perspective lectures, practicals, and online tutor support all aim to help the students build up these pillars of study, with the 6 assessment items serving as goals.
Ideally, after completion of this course a student will have the ability to continue self study of software and programming concepts after getting a 'jump-start' via this course. The student would ideally be able to work independently on projects for more advanced third and fourth year courses, and/or produce efficient code as part of Honours or higher degree research. Importantly, the student would have tools for contributing to open sourced projects, startup-teams, and be hirable in analytic software focused jobs in industry.
Clearly a one semester course cannot transform a mathematician into a software engineer, however it is hoped that through the course content, students will be able to further themselves on such a path if needed.
The course is composed of the following 8 study units. Each of the units feeds most of the pillars of study. The early units build up basic Julia, computer science, and tooling knowledge (mostly pillars 1 and 2), whereas the later units focus on deeper mathematical stories, mostly feeding pillars 3 and 4. Specifically with respect to pillar 3 (mathematical and statistical algorithmic concepts), there are four main concepts: numerical mathematics and ODEs (Unit 3), computer algebra systems (Unit 5), Monte Carlo and discrete event simulation (Unit 6), data processing (Unit 7), and machine learning (Unit 8). Clearly some of these concepts are often taught (often at greater depth) in other specialized courses. However in this course, the focus is software implementation.
Here is detail of the content of each of the Units:
Unit 1- Basics: Variables, arithmetic, logical statements, conditionals, iteration, generic functions, scope, arrays and similar structures, strings, input and output, the Julia language, Jupyter notebooks, REPL (command line), and markdown.
Unit 2 - On Algorithms and more: Sorting algorithms and their analysis. Quantifying performance via empirical measurement. Quantifying performance via mathematical analysis. Compilation steps. Memory organization. Representation of variables and quantities in memory. Additional tools: Unix command line, Git and GitHub like systems, IDEs (Integrated Development Environments).
Unit 3 - On data files, and basic numerics: Standard file formats (e.g. CSV, JSON), reading and writing to files, web input, basic plotting with Julia, basic descriptive statistics and statistical plotting with Julia, representation of floating point numbers, numerical inaccuracy issue (e.g. numerical derivatives), solutions of ODEs using standard methods, basic matrix operations and performance considerations, usage of third party packages and the Julia package manager. Usage of language features for dealing with special structures (e.g. sparsity). Further profiling and debugging tools and the basic usage of a debugger.
Unit 5 - Computer algebra systems and symbolic computation: Background from elementary number theory, p-addic lifting, Chinese remainder techniques, rational reconstruction, polynomial arithmetic, interpolation, GCD and Euclid's algorithm, factoring mod p, Zipple's algorithm, further symbolic computation applications.
Unit 6 - Monte Carlo and discrete event simulation: Pseudorandom number generators, from uniform distributions to any distribution, basic Monte Carlo based statistical analysis, discrete event simulation modelling, discrete event simulation engines. Modular software design, with modules and namespace control. Additional language features including meta-programming and further understanding of the compilation process. More on type inference and performance implications.
Unit 7 - Working with heterogenous datasets: Dataframes and data and more on memory management. Exploratory data analysis and visualization. Further language features. More on design principles of packages and interfaces.
Unit 8 - A view towards machine learning: Machine learning principles. Application of basic predictive models from machine learning libraries. Decision trees and the random forest algorithm as an example - including an implementation.
These are the 5 assessment items. The first 4 are due during semester and worked on progressively during the course. The last item is due during the final exam period. The course does not have a final exam. BigHW and Project 2 are to be worked on in pairs (or groups of 3 in special cases). The other items are individual.
BigHW (20%): Jupyter and REPL, basic Julia functionality and small programs – analysis of sorting, using an IDE, using Unix, using GitHub, file input output, and numerical mathematics. (HW in pairs).
Quiz (20%): Covering basics of Julia, representation of numbers, analysis of sorting performance.
Project 1 (20%): Symbolic computation. This projects builds on existing code supplied to the students.
Project 2 (20%): Discrete event simulation (project in pairs). This project requires the students to structure files and code independently.
Project 3 (20%): Machine learning (due during exam period). This project also involves preparing a written presentation based on data analysis findings.
The mark for the weakest of the projects will be replaced by the average of the other two projects. Further, UQ course grades are at the levels 7, 6, 5, etc. To obtain a grade of 6 or 7, a student must complete and pass all three projects. In addition, to obtain a grade of 7 a student must get more than 60% on the quiz. Note that from a UQ course profile perspective, Projects 1, 2, and 3 are considered as a "series".
Here is the schedule listing the lecturers, units of study, practical activity per week, and assessment.
Week
Monday (5-7pm)
Tuesday (6-7pm)
Units
Practical
Assessment Due
Perspective Seminar
1
Jul-25 (PB*)
Jul-26 (PB)
1
A
2
Aug-1 (PB)
Aug-2 (PB)
1+2
B
3
Aug-8 (PB)
Aug-9 (YN)
2
B
Chan (Tuesday)
4
Aug-15 (PB)
Aug-16 (PB)
2+3
C
5
Aug-22 (PB)
Aug-23 (AF)
3+4
D
BigHW
6
Aug-29 (AF)
Aug-30 (YN)
4
E (only 1 Zoom practical)
Quiz units 1--3 (Tuesday)
7
Sep-5 (AF+YN)
Sep-6 (PV)
4+5
F
8
Sep-12 (PV)
Sep-13 (PV)
5
F
9
Sep-19 (PV)
Sep-20 (YN)
5
-
Project 1
Mehta (Tuesday) - Cancelled - Zoom Help Session instead
Break
-
-
10
-
Oct-4 (AF)
6
G
11
Oct-10 (AF)
Oct-11 (AF)
6
G
12
Oct-17 (AF)
Oct-18 (AF)
7
H
Project 2
Foeglein (Tuesday)
13
Oct-24 (YN)
Oct-25 (YN)
8**
I
Exam period
Project 3 (Due Nov 18)
* All four lecturers, PB, PV, AF, and YN will be introduced in the first lecture.
** In addition to the 3 lecture hours, an extra video supporting this unit is provided.
There are 9 practicals (A-I) in total and this is a description of each practical (note the 2021 course had two practicals less).
Practical A - Basic tools: Using Jupyter. Markdown. Basic LaTeX formulas. Basic HTML in Jupyter. Basic Julia code running in Jupyter. Basic Julia REPL.
Practical B - Julia essentials: Variables, logical statements, conditional statements, loops, generic functions, scope, arrays, input/output, and a few more Julia essentials.
Practical E - Project submission guide - (no Jupyter notebook for this practical): How to create, manage, and submit projects with multiple files, and GitHub.
Each assessment item is to be handed in with an experience voice recording. In the voice recording, the student(s) state how they felt working on the assessment, what they found easy, difficult, dull, or interesting, and importantly state (if true) that the work is their own.
HW2, and the three projects (1-3) are to be handed in via GitHub (or GitLab). In these cases, the students are to create repositories for the submission. HW1, is in a more laid out and pre-specified format.
The quiz is in simple pen-and paper format (scans/photos are handed in).
For HW2 and the three projects (1-3), in addition to the GitHub (or GitLab), a single PDF file including printouts of all student source code should also be handed in. This is for easy annotation feedback of tutors.
A Unix style shell with Julia in your path. Note that such like shell is available by default for Linux or Mac users. For Windows users we recommended you install GitBASH.
This video describes installation of Julia and IJulia:
With the software installed, please bring your laptop to practicals (and ideally to the lectures). In exceptional circumstances, where you plan to attend a face to face practical and cannot bring your laptop, you may install Julia and the associated software on the Windows desktop machines available in the practical classroom. This is a workable solution but is not ideal. The installation may take several minutes and it is not guaranteed that it will remain on the machine over time. Hence whenever possible, bring your laptop.
Here are additional resources that may be of use for the course (or for introductory Julia programming in general). None of these are mandatory as there are plenty of examples in the lecture units and practicals. That is, it is recommended that you try running every bit of code from the lectures and practicals, investigate it, look at the Julia help to explore. Nevertheless, you may find some of these additional resources helpful as well:
The Julia Express: Provides a dense summary of language features. It may be a bit too dense for those that haven't programmed much before. Still, it is useful.
MATLAB–Python–Julia cheatsheet: This is useful for those coming with a bit of MATLAB or Python experience. It shows how things are done in Julia in comparison to the other two languages.
Think Julia: How to Think Like a Computer Scientist: This is an excellent introductory book for programming. If you haven't done any programming previously it is a good read. However for more experienced programmers it can be a bit too elementary; still useful.