[MUSIC] We're going to begin a series of lectures now to talk about different components of game programming. And the way we're going to do it is we're going to use a motivating example for a particular case study. We'll describe that in a little bit but we'll walk through and we'll add different components to this game while describing the technology that's underlining it. It'll be a 2D game, we'll build on the pong metaphor, we'll show you in a second. Whenever you begin a game there are some things that you have to think about when you're starting your game. And I want to talk about some of those things in this lecture. So the high level, what we're worried about when we start a game is making sure that we have a design for it that's thought through before we just jump into coding. By design I mean we need to think about the platforms we're going to be using, we need to be thinking about where we're going to get the assets. We need to think about the game mechanics, the fun part of it. We need to think about the flow not just through the game itself but also through the interface and how that's going to operate. And how we're going to have the user participate with social media maybe, or sharing scores, or leader boards, all that stuff. So it's very important that before you begin, you really think about how your game's going to be laid out. You do as much prep work as you can before you get into the coding, because once you get into the coding it's hard to change stuff after you've already gotten a long way along in your architecture. So the first thing that I want to make sure that we have a foundation for, is on tackling screen size. Screen size is pretty varied, the iOS now has many different screen sizes. And unlike the work that we were doing in vanilla UIKit where we had the different view controllers and the different widgets, screen size matters a lot more in games. And the reason why is because it affects aesthetics. Games are among other things, an artistic medium. And so the exact pixels and the layout and the rendering and the aspect ratio of it all matters. Matters to making the game that looks good and has a certain kind of experience and expression that's aesthetically pleasing. It can also affect playability, because if you render your icons or your touch surfaces too small, it'll be too hard for a player, especially one with fat fingers to successfully play your game, so it might mess up your playability. Or if you're not paying attention to screen size and you've render important components off of the side of the screen, so that the user can access them, that's not a very fun game. And finally, it could potentially effect the leaderboard, if one of your goals is to develop a community around your game and one of the aspects of that is to have a high score list, that social network, maybe through game center. And it's important that you think about how your game is going to play on different devices, because if it turns out that your game is particularly easier on maybe an iPad Pro because it has a really large amount of screen space, then you're going to find that all the people on the leaderboard, that are at the top, will be using an iPad Pro. And the people that are stuck with the iPhone 5, thank you very much, aren't going to enjoy that aspect of the game nearly as much. These are a couple of things that we worry about with screen size. Now screen size has a lot of different components and it starts with points. Points are a unit of measurement on the screen. They refer to the coordinates of all the drawings that we've talked about so far. So we specify coordinates on the screen or we specify the UI, specify the placement of an object on our screen, we're actually talking in points, not pixels. Points are abstract, they really only make sense in the mathematical coordinate space of our phone. So for example, an iPhone 6 Plus has 414 by 736 points. Kind of a strange pair of numbers but that represents the fact that the device is rectangular and it's longer on one side than the other. Now the points lead to rendered pixels. So while you may start with points, those points and the objects that are on the screen that are laid out in points get transferred and transformed into pixels. The point-based drawings are rendered into pixels and sometimes this is called rasterization. Sam and I have a long discussion about whether just because you're going from points to pixels, you should call that rasterization. We decided that no, it shouldn't be called rasterization just because you're going from points to pixels. But occasionally in the graphics hardware, in the graphics pipeline, there'll be a pause as the transformations are being made from moving your graphic from points representation to pixels representation. They'll be put in some sort of buffer, where they've been rasterized, possibly to do additional visual effects on the graphics or if you have a particularly complicated scene. In any event, points have to be rendered into pixels at some point. They frequently are scaled in order to get to pixel coordinates, and most of the time, they're scaled up. As far as I know, all of the iOS devices, points resolution is smaller than the pixel resolution. The higher the scale factor, the more detail you have in your images. So for example, the iPhone 6 Plus gets rendered at 3 times the number of pixels as the number of points. So the number of rendered pixels of an iPhone 6 Plus is 1,242 by 2,208. If I go back a screen, you see that the points is considerably smaller, and the points should be one-third of that amount. 414 times 3 should be 1,242, and there we got 1,242. So although we talk about the space in points coordinates, when we render our screen, we render it in a bigger pixel size. Finally, the number of pixels that we render our screen may not actually be in the number of physical pixels that are on our devices. Meaning, the number of little bits of colored light that can be light up independently. The device screen may actually have a lower pixel resolution than what we've just rendered. And in which case, we actually need to, after we've upscaled it from points, we need to downscale it in order to fit it onto our device, and this is the case with the iPhone 6 Plus. After you've rendered it to that larger size, you have to downsample it by 87% to get to a 1040 by 1920 physical pixel size. You may recognize this resolution because it's a common video format now. So if that's not enough, just because you have created physical pixels, doesn't tell you much about how large the image is actually going to render on a physical device. Pixels get displayed on a physical screen but there's another measurement called PPI, which stands for pixels per inch. That's a number that tells you the screen density of your particular device that you're working with. So pixels per inch translates your physical pixels into a density that renders in the material world. So again, on the iPhone 6 Plus there's 401 pixels per inch and so if you were to go back to the rendered pixels and do the trigonometry on that, you'd see that the size would wind up being 2.4489 inches x 4.7880 inches. And if you look at the diagonal starting with the pixels and then converting them from 401 PPI, you'll see that the diagonal is just shy, just scosh shy of 5.5 inches which is the advertised diagonal size of an iPhone 6 Plus. So let's put that all together. What we do is we start and we think about, abstractly, the space that we're working with in terms of points. We think about moving things around the screen in that points coordinate. The assets that we make, however, need to be scaled up. Because when we make graphics, when we make a ball, when we make a paddle, when we make an explosion, those things get rendered at higher detail than the points. So that for example on a Retina display you get really high precision graphics independently from how far around the screen you're moving. So the points become rendered pixels, the rendered pixels possibly get down-sampled into physical pixels. And then the physical pixels actually get put onto the physical device and cause the screen to glow with the pictures that we enjoy on our screen, okay? So, this matters because when we start making graphics assets, we have to be paying attention to this a little bit. And the way it works is, you have to think about the resolution at which you want to make it. So when I say graphics assets, what I mean are pictures. I mean images that you're going to be using in order to create your game. So whatever sprites you have, whatever background. Images you have. You have to think about what resolution are you going to get that image at? And when you think about the resolution, what you should be thinking about in creating these assets is you should be thinking about the rendered pixels. The rendered pixels is the asset size that you are concerned about. Those rendered pixels will get shrunk down and displayed as the in points. Then you have to think about how they're going to be laid out on the screen. Because you might, if you're not making a image that's going to take up the whole screen, you're just making a portion of an image that's going to be a portion of your game, a character or something. You need to think about, okay, how many points do I want that character to take up on the screen, how much space and points do I want that character to take up? And then I have to think about, okay, which device am I working for? And I need to create images that are at one times the number of points, two times the number of points, and three times the number of points. And the reason why is because all the different screen sizes render points to pixels differently. So here's an example of the iPhone 5s screen size. Basically, all of the iPhone screen sizes are 16 to 9, or a 9 to 16 screen ratio, which is the letterbox movie resolution. And ideally, that would be 1 to 1.778. Now, the iPhone 5s in points is 320 by 568. And that actually comes out to just shy of 9 by 16, so it's 1 to 1.775. Again, if you're worried about the detailed pixels, you're going to be annoyed to find that it's not exactly 9 by 16. Now, when you create assets for the iPhone 5s, you create them at two times the number of points that they're going to get rendered at. So if you were to create an image that was 640 pixels by 1136 pixels, that would get rendered as over 320 points by 568 points. This disconnect between points and pixels got introduced with higher density displays. Because a higher density display can take that larger 640 by 1336 image and display it in a really dense, high resolution, beautiful screen. So that's the iPhone 5s. The 6 is different, it's 375 by 667 points. Again, it's about 1 to 1.778, this is a more pure 9x16 ratio. And when you render things on the iPhone 6, you render them at two times size. So you take a 750 pixel by 1334 pixel image and that would get rendered at 375 points and 667 points, shrinking it down by half. Now the iPhone 6 Plus has all three of these different transformations. So it starts at 414 by 736 points, again a 9 by 16 ratio. But because of the really intense pixel density, the assets that you create on the iPhone 6 Plus are three times the density of the points. So you would create a 1,242 pixel by 2,208 pixel image in order to fill up 414 points by 736 points. Now when it actually gets rendered to the screen, your screen doesn't have that full three times pixel density. And so it gets downsampled by 87%, so the actual resolution that everything is being shown on your screen is 1080 by 1920. But, as a developer, what you're thinking about is the three times resolution. Now the iPads are a little bit different. The iPads, unfortunately, don't have that 16 by 9 aspect ratio, they have a 1 to 1.33 ratio. Or a 3 by 4 ratio, so a more traditional, or a little bit older US television screen ratio. The iPad Mini family works at 768 points by 1024 points, or a 1 to 1.33, so these are, I think all of them are exactly 3 to 4 ratio. But the Mini also, when you create assets for them, you double the number of points you want to cover in order to get the number of pixels. So if you're working in Photoshop, and you make an image that's 1,536 pixels by 2,048 pixels, and you display that on the iPad Mini screen, it's going to fill up exactly that number of points. The iPad Air family, including the iPad, the iPad 2, those all render at 768 points by 1,024 points and are also at two times the pixel resolution. And finally, the newest iPad, the iPad Pro, it has a larger point density. So it's 1,024 points by 1,366 points. It's still a 3 to 4 ratio and it's only a two-times blow up, so when you create an image that 2,048 pixels by 2,732 pixels, that will fill up 1,024 points by 1,366 points. So now, holy cow, you've got all these devices, these are just the most recent devices, some people still have the 4 and the 4s out there. There will be more devices in the future. How do you handle all this? If you're going to create an app, what screen size are you working with? Well, there are a couple strategies that you could possibly adopt. You can't do all of these strategies in your game and some of them may be appropriate for your kind of game, or maybe they'd be not appropriate for your kind of game. But for starters, the first thing that you can do, and this is supported in the assets within the game, is that you can supply all of your graphics assets at three different levels of resolution, 1x, 2x, and 3x. And in that way, regardless of which device you're working on, the iOS application can pick the best resolution image in order to display in high density images. So you can have high density, very nicely rendered images for the iPhone 6 Plus. And you can use a 1x resolution for an older device. And you'll see this in the access code asset manager, you'll see there's this place to drop these different density images in, and we'll do that in the demo shortly. Now another strategy you can adopt is you can decide to only design for the iPhone or the iPad only or primarily, perhaps. In that way, what you can do is you can start with one of the two ratios that dominates in those devices. So you can either start with the 9 by 16 or you can start with the 3 by 4. You can do your layout, you can do your game play based on one of those aspect ratios. And then you can just use Aspect Fit and that will enable, which if you're doing the iPad class that's going to fill up the screens on all the iPads with a nice scaling. So you're not going to have trouble because the X and the Y are going to get scaled in the same way. If you decide to design for the iPhone and you want your game to run on the iPad, well one strategy is to just accept Distortion. If you accept distortion, then you can scale to fill the screen. But you're going to find that if you design for the iPad, your graphics are going to get stretched out in order to fit the 16 by 9, or squished in order to fit the 3 by 4 if you go the other direction. If you don't like that stretching, what you can do, and you've probably seen this on some of the apps that you've used. If you have an iPad and you download iPhone apps to run on it, is you can accept that sometimes there will be unused screen space. And this will happen because you will download, for example, an app that is designed for an iPhone aspect ratio. You use it on your iPad and if you set programmatically the aspect fit parameter of those graphics. Then it will make it the correct aspect ratio so you don't lose the ratio of the length to the height. But because you can't fill up the screen, you'll have black bars on the side or on the top, depending on which way you're going and how your design worked. So these are different strategies that you can use. Of course, you can always make an app special for every single different device, that just takes a lot of resources. And then finally, another strategy is to only allow your iPhone to present in one orientation. Because of course the same problems that you have in trying to adapt to the content for an iPad to an iPhone, you also have in trying to adapt your content from a landscape mode game. And trying to play the same game in a portrait mode is really tricky. So, it's pretty common to just lock your orientation into of those two, either landscape or portrait, to reduce the amount of crazy design stuff you have to do. As far as some of the tools within Xcode, in the Palm demo you saw us using the Scene Builder, and we've really had some limited success with the Scene Builder. We've had a fair number of bugs show up, and very quickly we find that the Scene Builder's power just isn't high enough for most games. And so it seems okay for prototyping, if you want to try putting some sprites onto the sandbox Scene Builder and simulating them, you can kind of get your numbers right. Maybe it's a little bit faster than prototyping on the real device. But it's almost easier to lay out your game and to prototype everything on graph paper, either digitally or actual graph paper, and then you'll know the coordinates that you need. And then when you go to write your code, you can just do everything programmatically. You can create your objects in code, you can place them in code. And that gets you around some of the bugs we've encountered in Scene Builder. Now, Scene Builder will probably get better and stronger over time. A lot of Apple's tools have, and so we may find in the future it does more for us. One thing that Scene Builder is nice for though is it really exposes the range of properties that are available. Because when you put some kind of a sprite onto the Scene Builder canvas, it shows up under property pane usually on the right. You can see all the different things, all the different properties that that object has. And sometimes that's an easier way to explore them and to see the effects of them than it is to go look in the documentation. Although you can always look in the documentation. So without further ado what we're going to do going forward is we're going to take a particular case study and we're going to look at the different technologies that are associated with building a game. But We're going to do it in the context of a particular game that we're going to make together. So the game that I want to make is Break Out. It's kind of an old school game made famous on the Ataris, back, I don't know, probably in the late 70s, or early 80s. We're going to try it this particular way. What this game has is it has four corners, it's a static screen, it has a paddle at the bottom like pong, and it has a ball that bounces around the screen. And the goal of the game is to clear all of the bricks that are along the top. As if you're trying to break out of jail by bouncing a ball against the bricks. If you miss and the ball falls down below the paddle then you lose a life or you lose points. The other three walls are all bouncy walls, and it's fine to bounce the balls off them. So just for kicks I thought it might be fun to actually have two balls and have them connected by a simulated spring and just kind of see how that makes the game play. Right, so we're going to design for, actually, we're not going to design for 16 by 9, we're going to design for 3 by 4 because we're going to run it on an iPad. That's an error in the slide. We're going to use AspectFit to scale. So if you did run it on an iOS phone device, then you would see black bars on the side. We're going to lock the device to landscape, so we're going to play it in landscape mode. We're going to make all of our sprites image based. So we're going to have a lot of graphic assets that we're going to drop in to our folder, and we're going to construct those sprites in code rather than using the Scene Builder. This is going to be a one player game. The player's going to move the paddle with their finger in order to try and catch the ball. We're going to have an opening screen that launches and welcomes the player to the game. We'll have a primary play screen and then we'll have a closing screen that's like a game over screen. Or perhaps we'll have a leader board or something like that. Like I said, the ball is going to actually be two balls connected with a spring. We're going to put some sounds in there. I'd like to have a background sound that's providing some ambient noise. We're going to have a bounce sound for when the ball bounces off the paddle or the walls, and then we'll have a collision sound when the ball hits one of the bricks. Finally when there's a collision, we're going to show a Particle System Explosion. We're going to show you how to do Shader Explosions on top of that. And like I said, the side and top walls are solid, and the floor, if the balls collide with the floor, that would be a loss of a game, a loss of a turn, or a loss of a life. So just in summary, you need to make sure you design your game before you start coding it. You need to think about who's going to be playing it, who your audience is, what devices you're going to be working on, what the orientation is. You of course need to think about the game mechanics and how you're going to play the game. Think about how it flows through the different screens in the game. What assets are you going to need and how big are you going to make them? What resolution are you going to focus on? All these things need to be done before you get ready to actually start coding your game. All right, so we're going to move forward in this case study. Thank you for your attention. [MUSIC]
