Hello and welcome to this teach out on Augmented Reality,

where I want to actually show you a couple

of augmented reality devices that are available today.

So, I'll start, essentially the way I've structured it,

I want to start by showing two augmented reality capable smartphones,

and I'll explain what the differences

between these are because of technological differences.

And then in the second round,

I would like to bring in as a whole set of extensions for smartphones.

So, these are essentially display wrappers.

The phone goes inside them,

and it turns the phone into a heads up display.

And then, in the end,

I want to cover HoloLens on the other side.

It's essentially a fully self-contained computer that also goes on the head,

and it does not require any smartphone anymore.

So, to get started, you may have heard about Tango.

So, Tango is essentially a specification.

This is a Tango phone.

It is making sure that the phone has additional hardware.

In this case, the phone has additional cameras,

essentially a depth camera to understand geometry of a room,

and the motion camera to understand how the device is moving in space.

This used to be a very promising set of

technologies but there's only two devices that are actually supporting this.

This Tango idea has been discontinued.

Instead, over the last summer,

we have seen a new set of technologies for the different mobile platforms,

and this is an ARCore enabled device, the Pixel 2.

It does not have any additional camera.

So, the augmented reality now becomes a software problem.

So essentially, the main vendors of mobile phones have figured out ways to understand how

a device moves in space just

by looking at the small jitter that is happening with my device,

and basically creating images from different perspectives,

and then triangulating them to understand how far away any kind of pixel is,

that the camera would see.

So, in other words this is essentially software based augmented reality ARCore and ARKit.

The two kinds of platforms for the different types of mobile devices that we have,

and no additional camera required.

All right, so, this was just smartphone based AR, and now,

I want to talk about how we can actually turn a smartphone that is normally

hand-held into a heads up or head worn display.

So for that, we have a variety of options that we can choose.

So, you may have heard of Cardboard.

So, Cardboard is literally a display wrapper made of cardboard.

The way it opens this way.

I've already pre-assembled it

a little bit but it becomes essentially this display wrapper.

The phone goes inside here and you close this up,

and you have now two lenses and these map onto my eyes,

and what I now see is a stereoscopic image.

So, essentially for each MI,

I have a different rendering so that I can perceive depth.

This Cardboard is essentially known as a virtual reality display so is a Daydream.

So, Daydream is, in some sense the,

a little bit more advanced than Cardboard.

It's finer. It has the same idea.

The phone goes in here.

What comes with it, is also a controller.

So now, I have a pointing device

because what is not possible with virtual reality headsets,

is essentially pointing in front of the phone,

because the phone cannot see anything.

This won't work. So, instead you use a controller.

And this is where I want to make the main difference.

Cardboard and Daydream are actually display wrappers for virtual reality.

Cardboard is the same except that it can be modified,

which I've done here very cheaply.

Essentially cut a hole in here so that the camera of the smartphone is exposed and what I

can produce now is a see through augmented reality interface.

So, there is also another way which is essentially a modified cardboard,

done a little bit better than my version of it.

It has interesting combination of mirrors in there.

So, I'll show you. The phone goes essentially in here.

The screen of the phone will be mirrored up this way,

and so this becomes a see through display.

Phone goes in here,

I have to make sure that the camera is inserted the right way,

and then, I now have

a little bit more advanced augmented reality headset but the base is it's still my phone.

So, there are also many different kinds of Cardboard versions out there.

One is this one,

which is a little bit more mobile and small.

It's still called Cardboard.

This is actually made of plastic.

These are now essentially spectacles that open this way.

The smartphone is put in here so

it requires a little bit of assembly but it's relatively fast and easy to do.

And now you can just hold it in front of your eyes and you can produce

both a virtual reality experience or by making use of the camera of the phone,

an augmented reality experience.

So, I want to show you a couple of demos of this heads up smartphone

based augmented reality experience So,

here we see the horizon,

cardboard headset for AR.

I've already put the phone inside,

and all I need to do now is use this marker

and the phone's camera will track this marker.

And then, as I look through here,

I see an image,

in this case of a moving elephant and that image is

produced wherever the marker is recognized.

And so, it's inserting this digital elephant here,

and shows the capabilities of AR using a marker,

standard phone, and really just this cardboard headset.

Very cheap way of transforming a smartphone into a heads-up AR display.

So, in this small demo I'm going to show how measurements in AR work.

There's lots of apps out there,

where you can actually measure distance and size of objects.

So, here in this example,

I'm going to take a measurement from here and just by me using my phone,

it tells me that this is roughly 20 centimeters and to the rest of the table,

it would be much longer.

So, this is just like based on the camera input and estimation,

and how far the object is away,

so it detects the plane here an estimation of how much distance you would travel.

In this case, 20 centimeters.

So, in this example, I'm going to insert two virtual objects here,

on this table, using an ARCore enabled smartphone.

And so, this is software-based AR,

and these objects are really just like sitting

there and I can interact with them, go closer.

This one is sleeping right now, and the doughnut,

I can also move them closer to each other.

Looks, like the coffee sleeping at the moment.

Yeah, and I can walk around and they will start interacting with each

other. What I've shown you so far is

essentially just smartphone based, handheld augmented reality.

I've shown you a couple of examples of display holders to turn

a smartphone into a head worn variable display.

But these are still smartphone based.

So, the last example I want to show you is,

essentially devices like HoloLens.

So, these are fully self-contained computing devices, no smartphone required,

and it is a wearable augmented reality headset

that adjust to different head sizes and shapes,

and you place it on your head and it

has many similarities to some of the previous technologies that I showed you.

They're in fact, multiple cameras that can

track the device and head movement and those kinds of things.

There's microphone in built so you can actually interact with speech.

And we also have a first gesture vocabulary where you

can actually interact with your hand in front of the display.

So, I want to show you a couple of examples also using HoloLens.

So, without a lens what I'll show here is the first person view.

On the right, there is a set of holograms that are augmentations.

They're not real. On the left,

you see a physical screen and you see how these overlay.

So, now I interact by,

what I want to show you essentially,

is bringing in a new augmentation, a new hologram.

So, I'm going to grab this hamster here,

and I'm placing it in front of me on the table.

And now, this is a virtual hamster and it can also move and make some noise.

All right. So, what I want to show now is an interactive gaming experience

here called Robo Raid. All right.

So, in Robo Raid, the idea is that aliens are attacking you,

and they're essentially coming through the wall.

And HoloLens is now asking me to take a look around here in the studio,

to scan the environment.

The game is now ready and lets see what happens.

And now, we've just had an explosion here in the studio,

and the aliens are attacking essentially through a wall.

My name is X-ray Mark 17.

X-ray Mark 17 is a friendly guy,

teaching me what I can do in this game.

I'm here to help.

He's here to help me.

You may capture me. Your planet is doomed but if I transfer my powers to you,

we can still stop the, quickly come closer.

So, he's now asking me to go closer to the object.

This is a, I'm not sure whether I have captured him.

Future is established.

Oh, we have established a link.

Weapon systems online.

My weapon systems are going online.

Engaged. He'll be here any second.

Oh my God. They'll be here.

Brace yourself.

Any second, we should brace ourselves.

They're breaking through this wall.

And now, we can see how the first kinds of objects appear here, and shoot at me.

I need to dodge and I can shoot back with lasers.

Nice move.

Okay, this guy up here is actually behind

the screen and he's now shooting at me and I'm dodging.

It's relatively easy.

Of course, you can get better at this by a more practicing.

So, you dodge and you can shoot.

A little side effect of this game is that,

I'm essentially destroying the studio here at least virtually.

You're getting a hang of this. Keep it up.

I'm getting the hang of this.

Make it through.

We have another, these are flying so,

this is even more interesting.

So, the level becomes more difficult and

the objects that are appearing in front of me are becoming more flexible.

They're really like the [inaudible].

Look out.

So, this was essentially an overview of mobile and wearable augmented reality devices.

We started with smartphones,

we put smartphones into display wrappers,

and we ended with a fully self-contained computing device like the HoloLens and well,

there are of course lots of differences in

the user experience and also limitations of these devices.

And I would like to discuss them with you in the discussion forum.

Overview of the Technologies & Applications

Augmented Reality, Virtual Reality, and Mixed Reality: Opportunities and Issues Teach-Out

Conoce a los instructores