Lecture Big Data: Scalable Machine Learning

General Information

Introduction

In this lecture series, we will discuss how large scale learning is performed. Individual algorithms will be studied, and shown how learning algorithms are modified in order to be able to deal with large data sets. These approaches don't necessarily lead to scalable computations in the sense of distributed systems, but more often rely on skillfull approximations and simplifcations which nevertheless ensure that the resulting algorithm leads to good predictions.

Topics include:

• Fast approximation algorithms for classification and regression including stochastic gradient descent and bundle methods.
• Optimization theory.
• Sampling and approximations.
• Graphical models and Variational Bayes.
• Markov Chain Monte Carlo for learning.
• Hashing and sketches.
• Sequential Analysis for Testing and Cross Validation.
• Distributed Infrastructures for learning (e.g. parameter servers)

Prerequisites

This is an advanced course which assumes working knowledge of machine learning algorithms as provided by the lectures Machine Learning 1 and/or Machine Learning 2. Apart from that, working knowledge of linear algebra, multivariate analysis, probability theory, as well as computing architectures.