Bayesian optimization. Part 1

Loading...

Del curso dictado por National Research University Higher School of Economics

Addressing Large Hadron Collider Challenges by Machine Learning

4 calificaciones

National Research University Higher School of Economics

Addressing Large Hadron Collider Challenges by Machine Learning

4 calificaciones

Curso 7 de 7 en SpecializationAdvanced Machine Learning

The Large Hadron Collider (LHC) is the largest data generation machine for the time being. It doesn’t produce the big data, the data is gigantic. Just one of the four experiments generates thousands gigabytes per second. The intensity of data flow is only going to be increased over the time. So the data processing techniques have to be quite sophisticated and unique. In this course we’ll introduce students into the main concepts of the Physics behind those data flow so the main puzzles of the Universe Physicists are seeking answers for will be much more transparent. Of course we will scrutinize the major stages of the data processing pipelines, and focus on the role of the Machine Learning techniques for such tasks as track pattern recognition, particle identification, online real-time processing (triggers) and search for very rare decays. The assignments of this course will give you opportunity to apply your skills in the search for the New Physics using advanced data analysis techniques. Upon the completion of the course you will understand both the principles of the Experimental Physics and Machine Learning much better.

De la lección

Detector optimization

This module covers several cases of detector design optimization in high energy physics experiments using Bayesian optimization with Gaussian processes.

Detector optimization7:05

Normal distribution3:37

Gaussian processes for regression4:26

Bayesian optimization. Part 14:36

Bayesian optimization. Part 25:09

Optimization examples3:29

Conoce a los instructores

Andrei Ustyuzhanin
Head of Laboratory for Methods of Big Data Analysis
HSE Faculty of Computer Science
Mikhail Hushchyn
Researcher at Laboratory for Methods of Big Data Analysis
HSE Faculty of Computer Science

In this video, we start to talk about the Bayesian optimization.

So, what is Bayesian optimization?

Bayesian optimization is a method of finding the optimum of expensive cost function.

This cost function is also called objective function and denoted as f. It's

supposed that calculation of the objective function at one point is expensive.

The derivatives of objective functions are unknown.

The goal of Bayesian optimization is to find the optimum of

the objective function using as small number of the function calculations as possible.

Consider an example with the following objective function f.

The goal is to find minimum of this function using the Bayesian optimization.

Optimization algorithm is following.

The first, find the object function approximation using previously calculated values,

solving a regression problem.

Then, using the approximation,

find the optimum point of an acquisition function.

Then, sample the objective function in the next point and repeat these steps.

Acquisition function is used during Bayesian optimization

to estimate the next point of the objective function calculation.

There are variety of acquisition functions.

One of them is a Lower Confidence Bound shown on the slide.

In this objective function,

mu is the mean value of approximation of the given objective function.

Sigma is standard deviation of the approximation

and k is adjustable parameter of this function.

There are also Upper Confidence Bound for the objective function maximization.

Consider how

the optimization works for the following objective function shown on the slide.

Let's start the optimization from three observations.

At the first step of the Bayesian optimization,

find approximation of the objective function f,

using Gaussian processes and known observations.

This approximation is shown as

a green line with 3 sigmas confidence region of the approximation.

In this example, the Lower Confidence Bound is used as the acquisition function.

This acquisition function is shown as the blue line in the figure.

At the second step of the Bayesian optimization,

find minimum point x_4 of the acquisition function.

This minimum is shown as a blue dot on the figure.

At the third step of the Bayesian optimization,

sample the objective function at the found point x_4.

These steps are repeated several times.

Let's consider how the function approximation changes with iterations.

This is approximation after the second iteration.

This is after the third iteration,

after the 4th iteration.

This after the 10th iteration, after the 15th iteration,

and finally, after 20 iterations the minimum of the objective function is found.

The Gaussian processes approximate the function very well,

especially in the minimum region.

Also pay your attention that there are larger number of

observations in the minimum regions than in other regions.

In the end of this video,

I would like to remind you the Bayesian optimization algorithm.

As a first step, find the objective function approximation

using previously calculated values and solving the regression problem.

As a second step, using the approximation,

find optimum point of an acquisition function.

After that, sample the objective function at this new point and repeat all these steps.

In the next video,

we'll talk about the exploration-exploitation

trade of the Bayesian optimization.

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2018 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play