Add LSA question sheets

hpc_lecture_notes/2022-lsa_1.md

@@ -0,0 +1,72 @@
+# LSA Assignment 1 - Matrix-matrix multiplication
+
+This assignment makes up 20% of the overall marks for the course. The deadline for submitting this assignment is **5pm on Friday 1 September 2023**.
+
+Coursework is to be submitted using the link on Moodle. You should submit a single pdf file containing your code, the output when you run your code, and your answers
+to any text questions included in the assessment. The easiest ways to create this file are:
+
+- Write your code and answers in a Jupyter notebook, then select File -> Download as -> PDF via LaTeX (.pdf).
+- Write your code and answers on Google Colab, then select File -> Print, and print it as a pdf.
+
+Tasks you are required to carry out and questions you are required to answer are shown in bold below.
+
+## The assignment
+
+In this assignment, we will look at computing the product $AB$ of two matrices $A,B\in\mathbb{R}^{n\times n}$. The following snippet of code defines a function that computes the
+product of two matrices. As an example, the product of two 10 by 10 matrices is printed. The final line prints `matrix1 @ matrix2` - the `@` symbol denotes matrix multiplication, and
+Python will get Numpy to compute the product of two matrices. By looking at the output, it's possible to check that the two results are the same.
+
+```python
+import numpy as np
+
+
+def slow_matrix_product(mat1, mat2):
+ """Multiply two matrices."""
+ result = []
+ for c in range(mat2.shape[1]):
+ column = []
+ for r in range(mat1.shape[0]):
+ value = 0
+ for i in range(mat1.shape[1]):
+ value += mat1[r, i] * mat2[i, c]
+ column.append(value)
+ result.append(column)
+ return np.array(result).transpose()
+
+
+matrix1 = np.random.rand(10, 10)
+matrix2 = np.random.rand(10, 10)
+
+print(slow_matrix_product(matrix1, matrix2))
+print(matrix1 @ matrix2)
+```
+
+The function in this snippet isn't very good.
+
+### Part 1: a better function
+**Write your own function called `faster_matrix_product` that computes the product of two matrices more efficiently than `slow_matrix_product`.**
+Your function may use functions from Numpy (eg `np.dot`) to complete part of its calculation, but your function should not use `np.dot` or `@` to compute
+the full matrix-matrix product.
+
+Before you look at the performance of your function, you should check that it is computing the correct results. **Write a Python script using an `assert`
+statement that checks that your function gives the same result as using `@` for a pair of random 5 by 5 matrices.**
+
+In a text box, **give two brief reasons (1-2 sentences for each) why your function is better than `slow_matrix_product`.** At least one of your
+reasons should be related to the time you expect the two functions to take.
+
+Next, we want to compare the speed of `slow_matrix_product` and `faster_matrix_product`. **Write a Python script that runs the two functions for matrices of a range of sizes,
+and use `matplotlib` to create a plot showing the time taken for different sized matrices for both functions.** You should be able to run the functions for matrices
+of size up to around 1000 by 1000 (but if you're using an older/slower computer, you may decide to decrease the maximums slightly). You do not need to run your functions for
+every size between your minimum and maximum, but should choose a set of 10-15 values that will give you an informative plot.
+
+### Part 2: speeding it up with Numba
+In the second part of this assignment, you're going to use Numba to speed up your function.
+
+**Create a copy of your function `faster_matrix_product` that is just-in-time (JIT) compiled using Numba.** To demonstrate the speed improvement acheived by using Numba,
+**make a plot (similar to that you made in the first part) that shows the times taken to multiply matrices using `faster_matrix_product`, `faster_matrix_product` with
+Numba JIT compilation, and Numpy (`@`).** Numpy's matrix-matrix multiplication is highly optimised, so you should not expect to be as fast is it.
+
+You may be able to achieve further speed up of your function by adjusting the memory layout used. Focusing on the fact
+that it is more efficient to access memory that is close to previous accesses, **comment (in 1-2 sentences) on which ordering for each matrix you would expect to lead to the fastest
+matric-vector multiplication**.
+

hpc_lecture_notes/2022-lsa_3.md

@@ -0,0 +1,69 @@
+# LSA Assignment 3 - Sparse matrices
+
+This assignment makes up 30% of the overall marks for the course. The deadline for submitting this assignment is **5pm on Friday 1 September 2023**.
+
+Coursework is to be submitted using the link on Moodle. You should submit a single pdf file containing your code, the output when you run your code, and your answers
+to any text questions included in the assessment. The easiest ways to create this file are:
+
+- Write your code and answers in a Jupyter notebook, then select File -> Download as -> PDF via LaTeX (.pdf).
+- Write your code and answers on Google Colab, then select File -> Print, and print it as a pdf.
+
+Tasks you are required to carry out and questions you are required to answer are shown in bold below.
+
+## The assignment
+
+### Part 1: Comparing sparse and dense matrices
+For a collection of sparse matrices of your choice and a random vector, **measure the time taken to perform a `matvec` product**.
+You should use Scipy to store the sparse matrix is the most suitable format.
+Convert the same matrices to Numpy dense matrices and **measure the time taken to compute a dense matrix-vector product using Numpy**.
+**Create a plot showing the times of the sparse and dense for a range of matrix sizes** and
+**briefly (1-2 sentence) comment on what your plot shows**.
+
+For a matrix of your choice and a random vector, **use Scipy's `gmres` and `cg` sparse solvers to solve a matrix problem using your CSR matrix**.
+Check if the two solutions obtained are the same. 
+**Briefly comment (1-2 sentences) on why the solutions are or are not the same (or are nearly but not exactly the same).**
+
+### Part 2: Implementing a custom matrix
+The following code snippet shows how you can define your own matrix-like operator.
+
+```
+from scipy.sparse.linalg import LinearOperator
+
+
+class CSRMatrix(LinearOperator):
+ def __init__(self, coo_matrix):
+ self.shape = coo_matrix.shape
+ self.dtype = coo_matrix.dtype
+ # You'll need to put more code here
+
+ def _matvec(self, vector):
+ """Compute a matrix-vector product."""
+ pass
+```
+
+Let $\mathrm{A}$ by a $n+1$ by $n+1$ matrix with the following structure:
+
+- The top left $n$ by $n$ block of $\mathrm{A}$ is a diagonal matrix
+- The bottom right entry is 0
+
+In other words, $\mathrm{A}$ looks like this, where $*$ represents a non-zero value
+
+$$
+\mathrm{A}=\begin{pmatrix}
+*&0&0&\cdots&0&\hspace{3mm}*\\
+0&*&0&\cdots&0&\hspace{3mm}*\\
+0&0&*&\cdots&0&\hspace{3mm}*\\
+\vdots&\vdots&\vdots&\ddots&0&\hspace{3mm}\vdots\\
+0&0&0&\cdots&*&\hspace{3mm}*\\[3mm]
+*&*&*&\cdots&*&\hspace{3mm}0\\
+\end{pmatrix}
+$$
+
+**Implement a Scipy `LinearOperator` for matrices of this form**. Your implementation must include a matrix-vector product (`matvec`) and the shape of the matrix (`self.shape`).
+In your implementation of `matvec`, you should be careful to ensure that the product does not have more computational complexity then necessary.
+
+For a range of values of $n$, **create matrices in your format where each entry is a random number**.
+For each of these matrices, **compute matrix-vector products using your implementation and measure the time taken to compute these**.
+Create an alternative version of each matrix, stored using a Scipy or Numpy format of your choice,
+and **measure the time taken to compute matrix-vector products using this format**. **Make a plot showing time taken against $n$**.
+**Comment (2-4 sentences) on what your plot shows, and why you think one of these methods is faster than the other** (or why they take the same amount of time if this is the case).

hpc_lecture_notes/2022-lsa_4.md

@@ -0,0 +1,92 @@
+# LSA Assignment 4 - Solving a finite element system
+
+This assignment makes up 30% of the overall marks for the course. The deadline for submitting this assignment is **5pm on Friday 1 September 2023**.
+
+Coursework is to be submitted using the link on Moodle. You should submit a single pdf file containing your code, the output when you run your code, and your answers
+to any text questions included in the assessment. The easiest ways to create this file are:
+
+- Write your code and answers in a Jupyter notebook, then select File -> Download as -> PDF via LaTeX (.pdf).
+- Write your code and answers on Google Colab, then select File -> Print, and print it as a pdf.
+
+Tasks you are required to carry out and questions you are required to answer are shown in bold below.
+
+## The assignment
+
+In this assignment, we will look at solving the matrix-vector problem
+
+$$Ax=b,$$
+
+where $A$ is an $n$ by $n$ matrix and $b$ is a vector with $n$ entries. The entries $a_{ij}$ (with $0\leqslant i,j\leqslant n-1$) of A and $b_j$ of $b$ are given by:
+
+$$\begin{align*}
+a_{ij} &=
+\begin{cases}
+1&i=j\\
+1&i=n-1\text{ and }j=0\\
+1&i=n-1\text{ and }j=1\\
+1&i=n-2\text{ and }j=0\\
+1&i=0\text{ and }j=n-1\\
+1&i=1\text{ and }j=n-1\\
+1&i=0\text{ and }j=n-2\\
+-1/i&i=j+1\\
+-1/j&i+1=j\\
+0&\text{otherwise}
+\end{cases}
+b_j &= 1.
+\end{align*}$$
+
+For example, if $n=6$, then the matrix is
+
+$$\begin{pmatrix}
+2&-1&0&0&1&1\\
+-1&2&-0.5&0&0&1\\
+0&-0.5&2&-0.33333333&0&0\\
+0&0&-0.33333333&2&-0.25&0\\
+1&0&0&-0.25&2&-0.2\\
+1&1&0&0&-0.2&2
+\end{pmatrix}$$
+
+### Part 1: creating a matrix and vector
+**Write a function that takes $N$ as an input and returns the matrix $\mathrm{A}$ and the vector $\mathbf{b}$**. The matrix should be stored using an appropriate sparse format - you may use Scipy for this, and do not need to implement your own format.
+
+### Part 2: comparing solvers and preconditioners
+In this section, your task is to evaluate the performance of various matrix-vector solvers.
+To do this, **solve the matrix-vector problem with small to medium sized value of $N$ using a range of different solvers of your choice,
+measuring factors you deem to be important for your evaluation.** These factors should include
+the time taken by the solver, and may additionally include many other things such as the number of
+iterations taken by an iterative solver, or the size of the residual after each iteration.
+**Make a set of plots that show the measurements you have made and allow you to compare the solvers**.
+
+You should compare at least five matrix-vector solvers: at least two of these should be iterative
+solvers, and at least one should be a direct solver. You can use solvers from the Scipy
+library. (You may optionally use additional solvers from other linear algebra
+libraries such as PETSc, but you do not need to do this to achieve high marks.
+You should use solvers from these libraries and do not need to implement your own solvers.)
+For two of the iterative solvers you have chosen to use,
+**repeat the comparisons with three different choices of preconditioner**.
+
+Based on your experiments, **pick a solver** (and a preconditioner if it improves the solver)
+that you think is most appropriate to solve this matrix-vector problem. **Explain, making use
+of the data from your experiments, why this is the best solver for this problem**.
+
+### Part 3: increasing $N$
+In this section, you are going to use the solver you picked in part 3 to compute the solution
+for larger values of $N$.
+
+For a range of values of $N$ from small to large, **compute the solution to the matrix-vector
+problem**. **Measure the time taken to compute this solution**.
+**Make a plot showing the time taken and error as $N$ is increased**.
+
+Using your plots, **estimate the complexity of the solver you are using** (ie is it $\mathcal{O}(N)$?
+Is it $\mathcal{O}(N^2)$?). Briefly (1-2 sentences)
+**comment on how you have made these estimates of the complexity and order.**
+
+### Part 4: parallelisation
+In this section, we will consider how your solution method could be parallelised; you do not need,
+however, to implement a parallel version of your solution method.
+
+**Comment on how your solution method could be parallelised.** Which parts (if any) would be trivial
+to parallelise? Which parts (if any) would be difficult to parallelise? By how much would you expect
+parallelisation to speed up your solution method?
+
+If in part 3 you used a solver that we have not studied in lectures, you can discuss different solvers in parts 3 and 4.

hpc_lecture_notes/_toc.yml

@@ -41,6 +41,11 @@ parts:
  - file: 2022-assignment_2
  - file: 2022-assignment_3
  - file: 2022-assignment_4
+- caption: LSA
+ chapters:
+ - file: 2022-lsa_1
+ - file: 2022-lsa_3
+ - file: 2022-lsa_4
 - caption: Tasks for Monday Practical Classes
  chapters:
  - file: 2022-class_1