Explore
For Enterprise
Join for Free
Log In

artes y humanidades
Negocios
Ciencias de la Computación
Ciencia de Datos
Tecnología de información
salud
Matemáticas y Lógica
Desarrollo Personal
Ciencias Físicas e Ingeniería
Ciencias sociales
Aprendizaje de un idioma
Grados
Certificados

Explorar todo lo que Coursera tiene para ofrecer

Explorar
Buscar
For Enterprise
Inicia Sesión
Únete de forma gratuita

Speech

Loading...

Building Interactive 3D Characters and Social VR

Universidad de Londres

4.2 (34 calificaciones) | 2.7K estudiantes inscritos

Curso 4 de 5 en Realidad virtual Programa Especializado

Meeting another person is one of the most amazing experiences you can have in Virtual Reality. It is quite unlike communicating through any other medium except a real life face-to-face conversation. Because the other person is life size and shares a virtual space with you, body language works in a way that cannot be done on a flat screen. This course will enable you to create realistic social interactions in VR. You will learn about both the psychology of social interaction and the practical skills to implement it in Unity3D. We will take you through the basics of 3D character animation and how to create body language. You will learn about how to make characters that can respond to players’ speech and body language. You will also learn about avatars: the virtual representation of other players, and agents: computer controlled NPC characters and how to implement both of them. As many people have said before us, social is the future of VR. This course will help you become part of the future of Virtual Reality social experiences.

Revisiones

4.2 (34 calificaciones)

5 stars
20 ratings
4 stars
7 ratings
3 stars
3 ratings
2 stars
2 ratings
1 star
2 ratings

De la lección

Body Animation in VR

In this module you will learn about the role of the body in social interaction and also about how to animate and control virtual characters' body movements.

Introduction to Body Language1:52

Introduction to Skeletal Animation5:18

Introduction to Motion Capture3:38

Posture and Gesture4:15

Backchannel4:25

Sensing the User’s Body Language2:11

Body Language for Avatars5:52

Impartido por:

Dr Sylvia Xueni Pan
Lecturer, Department of Computing
Dr Marco Gillies
Senior Lecturer

Transcripción

[MUSIC]

Speech is the fundamental underpinning of human social interaction.

When we interact with other people, we spend most of the time talking.

We talk to communicate things to other people.

But talking is also just a general form of interaction that we use to pass the time

and to build relationships with people.

What we often call small talk.

So any conversation has two components.

A component of meaning, which is mostly carried by the words we say.

And a social and emotional component

which is carried mostly by the body language as well as the tone of voice.

In most of the following videos,

I'm going to talk about the social component of body language.

But I'd like to start by talking about the meaning component and language itself.

If I want a virtual character to have a conversation with me,

it has to do a lot of very complex things.

First, it needs to know when I've stopped talking and it's time for

it to start talking.

Body language can help with that, but the easiest way is just to check

the microphone volume and see if I'm making a noise or not.

It then has to take the audio data of my speech and figure out what the words are.

It has to take these word and figure out what they mean in this context.

And then it has to come up with a response.

Another set of words that appropriately answer me.

And finally, it has to take those words and

generate audio of realistic and ideally, appropriately expressive speech.

These last two components are the generation of speech and

include the component of meaning, generating words and

a component of expression, generating the audio.

We can try and do all of this from scratch using an algorithm.

Chatbot, pieces of software that generate dialogue,

have improved massively over the last couple of years, and they can create some

reasonably realistic conversations, at least in constrained domains.

But it's still an incredibly difficult problem.

And is very easy for chapter books to get things wrong.

Speech generation goes from words to audio isn't easy either, there are a lot of good

text to speech systems that can generate clear understandable audio from words.

But they lack the expression of a real human voice and since the tone of voice

an expression is so important to social aspects of conversation,

having an unrealistic voice can ruin the social impact of an interaction.

There have also been big advances in speech generation the last couple years.

And expect that what I just said will no longer be true in a few years' time.

But doing realistic, expressive conversation in the algorithmic

way is can be really hard, if not impossible.

It certainly isn't something I would recommend to a beginner.

Luckily, there's an alternative.

We don't need to generate all our speech from scratch.

We can just record someone, an actor, performing the speech, and

that will give us the audio we need.

The quality's likely to be very high, particularly if you've got a good actor

and a good microphone, and it will sound realistic and expressive.

The downside is you'll only have a small number of possible responses your

character can do.

Generating speech from scratch allows you to respond

appropriate to all sorts of different things.

But if your audio is prerecorded, you'll only have a few tens of clips to use.

But in many situations it's quite easy to make that work, since the nature of

the dialogue will be constrained by the narrative you'll experience.

There are particular things you want to accounts to communicate and

you record audio for that.

Now that leaves us the other side of the conversation,

what the real person is saying?

Turning audio into words actually works quite well now.

I often use dictation software to write the scripts for my video lectures.

Speaking makes a text more natural than if you're always typing them.

And the dictation works pretty well, it makes a few mistakes but really not

that many, it's probably good enough for interaction with a virtual character.

The hard part is figuring out what those words mean and

how the CAD should respond to them.

As I mentioned just now, you could use a chatbot software and that might work but

there are a lot of difficulties particularly if you want

to write your own chatbot.

There's also another,

more artistic problem with allowing freeform dialogue with characters.

If you're creating a narrative experience, a fiction,

you want to have great dialogue, all of which is really in character.

But the problem is that your players aren't scriptwriters, aren't actors.

They aren't going to be able to improvise dialogue that will live up to

the character that they're playing.

So asking to do so is putting a lot of pressure on them.

And when they say things that aren't in character,

it will make them feel less embodied in their character and more like themselves.

The easiest way to get around a problem is to limit what your

user can say to only a few phrases so that you can be sure what they mean and

you can be sure which responses you should give to them.

Before your experience start, you can explain to the player the words and

phrases that they can say.

The important thing is to be very clear what will and won't be recognized.

You don't want the player trying to say lots of things and

the character failing to recognize them all, that can be very frustrating.

An idea I particularly like is not to have players speak at all, but to use gestures.

There are lots of gestures that we use to communicate with other people.

In Western cultures, these include nodding our head to say yes, and

shaking it to say no, and pointing at things.

Yes, no, and indicating objects by pointing

can actually communicate quite a lot, as you all know if you've tried shopping in

a country where you don't speak the language.

I know this from my own experience, my uncle was unable to speak or

use sign language due to his disability, but

he's able to take a full part in family conversations using only simple gestures.

Sure, it limits the input from players, but it also put less pressure on them and

focuses the interaction on the character's dialogue.

You need to write dialogue where the character takes initiative and

the player is mostly responding to it.

So you save yourself the trouble of understanding comments from the player.

You also know the kind of thing the player's likely to say.

If this dialogue is good, it will drive the interaction and feel light.

Using standard gestures like nodding your head means it will feel right.

In terms they will be plausible because we're interacting with the character

in the way we interact with real people.

Even if it's a bit more limited.

So speech interaction is really challenging, but

it's still possible to create a compelling experience by using a set of prerecorded

audio clips, and limiting the personal interactions with the player.

This forms the basis of an interaction with a virtual character and

it can be massively enhanced by realistic body language as we'll see.

[MUSIC]

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2019 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play

Coursera
Acerca de
Liderazgo
Empleo
Catálogo
Certificados
Certificados MasterTrack™
Grados
For Enterprise
For Government

Comunidad
Learners
Partners
Developers
Beta Testers
Translators

Conectar
Blog
Facebook
LinkedIn
Twitter
Instagram
Youtube
Blog de Tecnología

Más
Términos
Privacidad
Ayuda
Accesibilidad
Prensa
Contacto
Directorio
Afiliados