Volver a Prediction and Control with Function Approximation

estrellas

753 calificaciones

In this course, you will learn how to solve problems with large, high-dimensional, and potentially infinite state spaces. You will see that estimating value functions can be cast as a supervised learning problem---function approximation---allowing you to build agents that carefully balance generalization and discrimination in order to maximize reward. We will begin this journey by investigating how our policy evaluation or prediction methods like Monte Carlo and TD can be extended to the function approximation setting. You will learn about feature construction techniques for RL, and representation learning via neural networks and backprop. We conclude this course with a deep-dive into policy gradient methods; a way to learn policies directly without learning a value function. In this course you will solve two continuous-state control tasks and investigate the benefits of policy gradient methods in a continuous-action environment.
Prerequisites: This course strongly builds on the fundamentals of Courses 1 and 2, and learners should have completed these before starting this course. Learners should also be comfortable with probabilities & expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), and implementing algorithms from pseudocode.
By the end of this course, you will be able to:
-Understand how to use supervised learning approaches to approximate value functions
-Understand objectives for prediction (value estimation) under function approximation
-Implement TD with function approximation (state aggregation), on an environment with an infinite state space (continuous state space)
-Understand fixed basis and neural network approaches to feature construction
-Implement TD with neural network function approximation in a continuous state environment
-Understand new difficulties in exploration when moving to function approximation
-Contrast discounted problem formulations for control versus an average reward problem formulation
-Implement expected Sarsa and Q-learning with function approximation on a continuous state control task
-Understand objectives for directly estimating policies (policy gradient objectives)
-Implement a policy gradient method (called Actor-Critic) on a discrete state environment...

AC

1 de dic. de 2019

Well peaced and thoughtfully explained course. Highly recommended for anyone willing to set solid grounding in Reinforcement Learning. Thank you Coursera and Univ. of Alberta for the masterclass.

SJ

24 de jun. de 2020

Surely a level-up from the previous courses. This course adds to and extends what has been learned in courses 1 & 2 to a greater sphere of real-world problems. Great job Prof. Adam and Martha!

Filtrar por:

por George G

â€¢28 de feb. de 2020

Fantastic course! Despite the challenging content, this course actually is taught at least at the same level as the ones by Andrew Ng, Daphne Koller, and Geoffrey Hinton. Congratulations Martha and Adam! You are awesome and are my heroes! Thanks a lot! George

por Neil H

â€¢3 de nov. de 2021

Excellent.

BEFORE this course: Iâ€™ve done a number of Coursera courses before. Whilst they are good, the level of learning tends to be superficial.

THIS is the first time I've dona a series of course (a module). These are the best courses Iâ€™ve taken and (after 3) I now feel I have learnt a very significant amount. Below applies to all three courses.

I have seen someone criticize the course by saying â€˜it is just them talking through the Sutton & Barto bookâ€™. In defense: (i) the book *does* seem to be *the* seminal introductory text, (ii) importantly, they have selected which bits to cut out, (iii) I have now read through the recommended chapters as part of the course and have far greater insight than if just reading the book myself.

In some cases, the slides show things clearer than in the book. In some cases, the sentences are far too complicated to digest oinne one go. You need to rewind again and again to understand things.

I have found the time taken to do the Python assignments to be much longer than they suggest but this is largely down to my lack of Python abilities. I lot of time was spent improving my Python â€“ which was a good by-product. The intermediate checking of code (within Jupyter) could be better but the forums help.

por Mukund C

â€¢27 de mar. de 2020

Excellent Course and Lectures. Loved it!! So important to read the chapters in the book ahead of time. Book is also excellent!! I liked the way the instructors explained the equations and broke them down. Nicely done!! I wish some more of the questions in the quiz reflected the data structures we use in the programming exercise, which will be super-helpful to reinforce the concepts when we do the programming exercises. In other words, an intermediate step of a worked example between the Pseudo-Code Algorithm in the Texbook/Lectures and the Programming Exercise. For example, more of the Feature_Vector -> Action_Value Calculation - even if we have to do some matrix manipulation by hand, that'd be wonderful. One of the quizzes has something like that (but more simplified) - which was perfect.

por Navid H

â€¢16 de oct. de 2019

The material is very good. But this course needs better instructors/ method of teaching. The book is also written in an unnecessarily technical way filled with jargon. explanations are not clear, simple stuff is presented in a very complicated manner for no obvious way.

por Maxim V

â€¢23 de ene. de 2020

Good content, but there was a highly unpleasant surprise in the programming assignments, namely this: "Retakes: You can attempt this assignment 5 times every 4 months." First of all, this is a highly unusual and therefore unexpected requirement on Coursera. Also, considering how buggy graders are and that some assignments require submitting results separately from the notebook, this is a really high risk of having to wait 4 months for another chance.

por Arthur O

â€¢9 de oct. de 2020

Excellent instruction that guides through the core material of part 2 of Sutton & Barto's Reinforcement Learning: an Introduction. The instructors additionally teach complementary material not found in the book. The notebooks got me "making something real" with the material in a way that deepened my understanding beyond a theoretical/pen-and-paper treatment. I appreciate the care that went into setting up the RL learning environment, creating test cases, and visualizations of performance -- it's awesome when the agents come together and you can see how well they perform!

A couple very minor notes on the lectures. The pacing of speech by Dr. Adam White often felt stiff and clipped. In future video courses he might benefit from practicing changing his tone, speed, and pauses to sound more natural. Similarly, Dr. Martha White's microphone was positioned in such a way that her breathing between sentences is captured, and sounds pretty loud. Improving these aspects of presentation in the future can make the lectures flow more naturally and reduce some friction from the distractions.

Those are nits on an otherwise excellent course. Thank you very much for putting the materials together! See you in the next one!

por Maximiliano B

â€¢31 de mar. de 2020

The third course of the specialization is excellent and it provides a solid foundation on problems with arbitrarily large state spaces that rely on approximate solution methods. The lectures are very well explained. Itâ€™s strongly recommended to read each book chapter in advance before watching the lectures to be able to better understand the concepts and be able to answer the quizzes. The content in this course is quite abstract and it is heavily dependent on statistics and calculus. It was very nice to integrate reinforcement learning with neural networks as part of one of the assignments as well as to implement the swing-up pendulum. I am looking forward to begin the capstone project.

por D. R

â€¢31 de dic. de 2019

Excellent course. The videos, quizzes, and especially the exercises add a lot of extra value to the text book (which is available for free - Sutton and Burto, 2nd edition). Of course it is not perfect - the videos are sometimes a bit dry, the NN part was brushed over too quickly for a beginner (luckily I had taken some courses about deep learning, so I was ok - but if you don't know the basics of NN, week 2 might be quite challenging for you). Other than that the biggest disadvantage is that the course forums are still quite empty - and so if you get stuck you can be on your own... But you shouldn't get stuck, and I guess this will improve over time.

por Mark J

â€¢22 de oct. de 2019

This, the third in an exceptionally well-paced series of four courses on Reinforcement Learning, extends the scope of the subject to include parameterized functions (i.e., neural networks). The section on tiling methods is especially interesting. The course is taught under the auspices of professors who, quite literally, wrote the book on reinforcement learning, and includes several video lectures by leading practitioners and theorists in the field. The final programming assignment, in particular, made me feel like I did when I wrote my first computer program that actually did what it was supposed to way back when -- delight and amazement.

por Ian W

â€¢17 de jun. de 2021

Learned a lot through the course. This specialization is to teach you through the whole reinforcement learning textbook. Very informative, but the programming assignments are very difficult.

There are many tiny details to notice while programming, and the discussion forum is not very active. I suggest find some clues from the old posts when having difficulties, and ask / answer questions as much as possible to help yourself and the others.

por Julien T

â€¢12 de nov. de 2019

Great course and specialization. The teachers are great, the material well presented and balanced. I strongly recommend this course to anyone interested in the field of Reinforcement Learning. For maximum chance of success I suggest following all 3 courses in succession and investing the necessary amount of time to read the textbook chapters as specified at the beginning of each week.

Looking forward to completing the capstone project now!

por Gordon L W C

â€¢23 de mar. de 2020

The course is very comprehensive on the content. But I think the difficulty of this course is in some sense too high for most people who don't have a background in engineering degree due to the extensive use of advanced mathematics. I think it might be a better idea if you are focusing on a few critical algorithms that trying to cover too much algorithms which is quite overwhelming

por Walter O A

â€¢9 de dic. de 2019

An almost overwhelming amount of material, however we managed to navigate through the thicket. The labs were well maintained and provided robust tests so that one could have a high degree of confidence in the solution before submitting to the grader. I really appreciate this. I would recommend this course to anybody wanting a serious introduction to reinforcement learning.

por Stefano P

â€¢19 de may. de 2020

This course is very rich of both mathematical and practical concepts, and it actually provides you with powerful tools to understand and use Reinforcement Learning. So far, it is the most interesting course in this specialization. Lectures are very clear and they often explain more deeply some concepts you find in the text book. Quizzes are challenging and well constructed.

por Sebastian P B

â€¢2 de dic. de 2019

This was a very good and though course. The content in this course is perfect to get yourself the necessary bases in order to start getting into deep RL. It doesn't really explain that far, but at the end you will have a basic idea of how deep learning can be used with RL. Enough to start reading papers about it or to watch other lectures focused on that topic.

por Zhang d

â€¢6 de may. de 2020

This course is amazing and wonderful, teaching us the knowledge to use function approximation to solve the vaule and policy estimation. Compared to the preceding tabular presentation, function approximation is appropriate for the reality problem and makes RL more powerful and interesting. I am looking foward to learning more skills in the RL area!

por Jesse W

â€¢29 de jul. de 2020

This is a rigorous but clear course in using function approximation to calculate (rather than tabulate) state values and policies in reinforcement learning applications. The videos and assignments are well-constructed and instructive, and the free online textbook is great as well.

por Guilherme V

â€¢20 de nov. de 2020

It was a great course, but I think it could be better if at least one practical example of function approximation with Neural Networks were in it, maybe integrate it with Tensorflow to make it simpler, but illustrate how it would work. Nevertheless, it was Really a nice course

por Tobias L

â€¢14 de oct. de 2020

Once again a nice course in the series of lectures. Giving a good overview of function approximation and its use in RL. Moreover, a lot of the piecies given in the previous lectures start to fit together. And I finally can connect DeepLearning with RL.

Thanks, Martha and Adam

por Alvaro M A

â€¢3 de abr. de 2020

Excellent course. You'll learn how to apply techniques like Neural Networks to Reinforcement Learning Agents. The assignments are very well designed, and they have the right difficulty so you can learn the most fundamental concepts of the course.

por Surya K

â€¢30 de abr. de 2020

Very well made course. This helped me build Agents for our mobile board game! I'd like some more practical tips as well though, like how to make algorithms converge and things to try out. Other than that, I recommend this to every RL enthusiast.

por CÃ©sar S

â€¢8 de ago. de 2021

Awesome course that complements both first courses on RL. Excellent chapter selection from the book that shows just the necessary and sufficient information to get a good idea of many of the concepts of function approximation. Very recommended.

por Lim G

â€¢10 de may. de 2020

I enjoyed the course because the content delivery was clear and concise. The hands-on assignment helped me better understand the concepts that were taught. I was able to draw connections and link between textbook and hands-on experience.

por Thomas G

â€¢21 de abr. de 2020

A very ambitious course where you have to invest a lot in reading the book but therefore you also learn a lot. I prefer more of those advanced courses here on Coursera.

The course is a very good complement to the book from Sutton.

por Mateusz K

â€¢29 de oct. de 2019

Its got a great variety of very applicable examples, use cases, and assignments. May be tough if people don't quite understand how neural networks work, so I suggest having a basic understanding of NN for parts of this course.

- Analista de datos de Google
- Certificado profesional de comercio electrÃ³nico y marketing digital de Google
- Certificado profesional de AutomatizaciÃ³n de TI de Google con Python
- Soporte de TI de Google
- GestiÃ³n de proyectos de Google
- DiseÃ±o de experiencia del usuario (UX) de Google
- PrepÃ¡rate para una certificaciÃ³n en Google Cloud: arquitecto de la nube
- Analista de ciberseguridad de IBM
- Analista en datos de IBM
- IngenierÃa de Datos de IBM
- Ciencia de datos de IBM
- Desarrollador de la nube de pila completa de IBM
- Aprendizaje automÃ¡tico de IBM
- Contabilidad en Intuit
- Desarrollador de front-end de Meta
- Certificado profesional de DeepLearning.AI DeepLearning.AI desarrollador de TensorFlow
- Certificado Profesional de Programador de SAS
- Lanza tu carrera profesional
- PrepÃ¡rate para una certificaciÃ³n
- Avanza en tu carrera
- CÃ³mo detectar errores de sintaxis en Python
- CÃ³mo descubrir excepciones en Python
- Ver todos los tutoriales sobre programaciÃ³n

- cursos gratuitos
- Cursos en Inteligencia artificial
- Cursos de Blockchain
- Cursos de ciencias de la computaciÃ³n
- Cursos gratis
- Cursos de seguridad cibernÃ©tica
- Cursos de anÃ¡lisis de datos
- Cursos de anÃ¡lisis de datos
- Cursos de inglÃ©s oral
- Cursos de desarrollo web de pila completa
- Cursos de Google
- Cursos de recursos humanos
- Cursos de TI
- Cursos de aprendizaje de inglÃ©s
- Cursos de Microsoft Excel
- Cursos de gestiÃ³n de productos
- Cursos de gestiÃ³n de proyectos
- Cursos de Python
- Cursos de SQL
- Certificaciones Agile
- Certificados en GestiÃ³n de proyectos (CAPM)
- CertificaciÃ³n CompTIA A+
- Certificado de anÃ¡lisis de datos
- Certificaciones de Scrum Master
- Ver todos los cursos

- Cursos en lÃnea gratis que puedes terminar en un dÃa
- Cursos gratuitos populares
- Empleos en negocios
- Empleos de seguridad cibernÃ©tica
- Trabajos de nivel principiante en TI
- Preguntas de entrevista de anÃ¡lisis de datos
- Proyectos de anÃ¡lisis de datos
- CÃ³mo convertirse en analista de datos
- CÃ³mo convertirse en gerente de proyectos
- Habilidades de TI
- Preguntas de entrevista de gerente de proyectos
- Habilidades en programaciÃ³n Python
- Puntos fuertes y dÃ©biles en la entrevista
- QuÃ© hace un analista de datos
- QuÃ© hace un ingeniero de software
- QuÃ© es un ingeniero de datos
- QuÃ© es un cientÃfico de datos
- QuÃ© es un diseÃ±ador de productos
- QuÃ© es un Scrum Master
- QuÃ© es un investigador de UX
- CÃ³mo obtener una certificaciÃ³n PMP
- Certificaciones PMI
- Certificaciones populares en Seguridad CibernÃ©tica
- Certificaciones populares en SQL
- Leer todos los artÃculos de Coursera

- Certificados profesionales de Google
- Certificados profesionales
- Ver todos los certificados
- Licenciaturas
- MaestrÃas
- TÃtulos en ciencias informÃ¡ticas
- TÃtulos de grado en ciencias de los datos
- MBA y tÃtulos de grado en negocios
- TÃtulos de grado en AnÃ¡lisis de datos
- TÃtulos de grado en salud pÃºblica
- TÃtulos de grado en Ciencias Sociales
- TÃtulos de grado en administraciÃ³n
- Programas con tÃtulo de grado de licenciatura en Arte versus licenciatura en Ciencias
- Â¿QuÃ© es una licenciatura?
- Los 11Â buenos hÃ¡bitos de estudio para desarrollar
- CÃ³mo escribir una carta de recomendaciÃ³n
- Los 10Â empleos mÃ¡s demandados con un programa de grado en Negocios
- Â¿Vale la pena hacer una MaestrÃa en Ciencias informÃ¡ticas?
- Ver todos los programas de grado
- Coursera India
- Coursera Reino Unido
- Coursera MÃ©xico