This repository contains exercises and projects on computational science and AI for the CompSci program
Machine learning with linear and non-linear regression, logistic regression and support vector machines as well as Bayesian linear regression. This involves linear algebra (matrix inversion, determinants, eigenvalues, SVD and more from FYS4150), convex optimization problem (gradient descent, steepest descent, stochastic gradient descent, iterative solvers) and several central (deterministic) ML methods. Calculation-oriented statistics with Bayes' theorem and MCMC sampling can also be included. Bayesian linear regression can be omitted.
Datasets you study can be adapted to your research field, whether it is astro, physics, chemistry, bioscience, geoscience or mathematics. Planned finished end January 2023
Deep learning: standard neural networks, convolution and neural networks (CNN), recursive neural networks, Boltzmann machines, various autoencoders and possibly general adversial networks. Reduction of dimensionality in scientific problems. Possible topic to work with: solution of ordinary and partial differential equations. Here we can take this from a deep learning perspective and a traditional final difference form taught in FYS4150. But we can also focus on classification problems. Datasets can again be adapted to the field.
See project description here: https://raw.githubusercontent.com/CompPhysics/CompSciProgram/main/doc/Projects/2022/Project3/pdf/Project3.pdf
In total 20 ECTS.
- Intro to machine learning and linear regression, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week42/html/week42-reveal.html
- Video of Lecture at https://youtu.be/C8dL1pLUJ3A
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesOct262022.pdf
- See also notes on derivatives of matrices and vectors at https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2022/NotesExercise5Week452022.pdf
- From ordinary least squares to Ridge and Lasso regression, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week43/html/week43-reveal.html
- Video of Lecture at https://youtu.be/EiO7WOm_DLs
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov22022.pdf
- Video of Lecture at https://youtu.be/XIdtY2_9rgk
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov92022.pdf
- Video of Lecture at https://youtu.be/HvBSIGAemvE
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesNov232022.pdf
- Logistic regression and intro to optimization problems, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week45/html/week45-reveal.html
- Video of Lecture at https://youtu.be/me_tglaPvI0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec122022.pdf
- Gradient methods, from simple gradient descent to stochastic gradient descent, slides at https://compphysics.github.io/CompSciProgram/doc/pub/week46/html/week46-reveal.html
- Video of Lecture at https://youtu.be/8WoA-MQt_8U
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec132022.pdf
- Stochastic gradient descent, from momentum to adaptive methods and discussions of project 1. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week46/html/week46-reveal.html
- Video of Lecture at https://youtu.be/EFNlvtjhpMo
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesDec142022.pdf
- Start Deep Learning, basics of neural networks, mathematics and architecture. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week3/html/week3-reveal.html
- Video of Lecture at https://youtu.be/JY-7F6TXK7U
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan172023.pdf
- Deep learning, back propagation algoritm and automatic differentiation, part 1. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week4/html/week4-reveal.html
- Video of Lecture at https://youtu.be/KaL0W1jaRmQ
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan242023.pdf
- Deep learning, back propagation algoritm and automatic differentiation and code for neural networks, part 2. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week5/html/week5-reveal.html
- Video of Lecture at https://youtu.be/cWCebuNKrA8
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesJan312023.pdf
- Deep learning and codes for neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week4/html/week4-reveal.html
- Video of Lecture at https://youtu.be/_pOf4__oN28
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb72023.pdf
- Discussion of project 2 and start discussion of Convolutional neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week7/html/week7-reveal.html
- Video of Lecture at https://youtu.be/leazXTMbRCM
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb212023.pdf
- Convolutional neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week9/html/week9-reveal.html
- Video of Lecture at https://youtu.be/C8Pj7Wq_7fw
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesFeb282023.pdf
- Summary of CNNs and start discussion of Recurrent neural networks. Slides at https://compphysics.github.io/CompSciProgram/doc/pub/week10/html/week10-reveal.html
- Video of lecture https://youtu.be/VvgiK3TUdxg
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMarch72023.pdf
- Summary of RNNs and work on project 2
- Video of lecture at https://youtu.be/TD63ggQbkTk. No whiteboard notes.
- Work on project 2
- Intro to part 3 of the course (Bayesian methods)
- Video of lecture https://www.dropbox.com/s/u5nmoodg3v7vc2b/2023_March_28.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMar282023.pdf
- Aspects of Bayesian statistics
- Video of lecture https://www.dropbox.com/s/5upi56c54e92kxy/2023_April_25.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesApr252023.pdf
- Wrap up intro to Bayesian statistics
- Nested sampling
- Video of lecture https://www.dropbox.com/s/jajawr636pdksx4/2023_May_2.mp4?dl=0
- Handwritten notes at https://github.com/CompPhysics/CompSciProgram/blob/main/doc/HandwrittenNotes/2022/NotesMay022023.pdf
- Wrap up discussion of nested sampling
- Gaussian processes
- Gaussian processes, cont.
- Video (from 2022): https://www.dropbox.com/s/yl7uwpx6fc1pv72/2023_May_16__more_GP_regression_from_2022.mp4?dl=0
- Handwritten notes: same as previous lecture