What are you doing? What are you doing? I'm practicing. I got an away game to go to. Oh, away game for the warriors. Well, I got something that might help you maximize your experience at the airport. Tell me more about that. So, today we're talking about Google Cloud IoT Core pushing airline's data into pops up, dataflow, and BigQuery. Interesting. So, what are you collecting and what are telling me about that? So, we're talking about customer events data like whether they checked bags, and how much time you have in between trips so that you can possibly get a drink or upgrade your seats. Nice, I would love that. I spend a lot of time at the airport. Yeah. Well, hopefully your Dataflow game is better than your basketball game. Come on now, all right let's see it. Let's see it. Five minute demo starting now. So, before I jump into the actual time, I just wanted to cover the architecture that we'll be representing through this demo today. We are generating random data through Compute Engine and feeding it into a pub sub topic which we will then use Cloud Dataflow to stream it into BigQuery in real time. Let's go ahead and just jump into the actual demo. So, the first thing I want to show about how I got the setup is I' m going to head over to Google Cloud Storage which I mentioned is how we did the device simulations and also where recruited some folders for our Dataflow pipeline. So, we created a bucket titled Iot-airlines-v1-data. We created two folders within this bucket and so that's going to include some information about our device simulators here. We've also created another bucket for our Dataflow pipeline. So, for our Dataflow pipeline, we've also included some files here and we created two folders for staging and tamp, it includes spinning up and shutting down Google Compute Engine instances and they access these GCS buckets for both IO and temporary file staging. Now, we've also set up a pub-sub subscription. So first thing we do here is head over to pub sub and you see here we've created a topic and this topic is titled airline messages. The type of subscription we created under this topic is a pool subscription. We've named it Dataflow here. So, you can see here a publisher application, in this case it would be our IoT simulators. They create a topic in pub sub, which we've titled airline messages and that's a service that sends messages to the topic. Now, going back a little bit, I'm going to show you IoT Core because this is also the component in which we register our IoT simulators. What we've done here is we've registered this right here called iot-airlines-registry. We've essentially cloned the Git repository where we have Java files and we compiled the application here, upload a jar file to the storage bucket that we showed earlier, this is our registry ID. It's going to accept in the form of MQTT and HTTP protocols in the region us-central1, we have pub sub set up for this IoT device registration which is the one we created earlier, airline messages. Anytime you register device to connect to IoT Core, you just need to register it first with Cloud IoT. It consists of adding a device to a collection which was this registry and defining some of the essential properties like we've done here. The last thing I'm going to show you is our Dataflow pipeline that we've created. What we've done to build this Dataflow pipeline is we've included SDK client libraries for Java and Python plus Rest APIs, and we've essentially cloned the Git repository, we've updated some files there and then we deployed it directly to Dataflow here. I'll show you here what the pipeline we've created looks like. So essentially, it's reading messages, applying window and techniques for streaming information from our IoT devices, and it's creating a table row and entity convergence so that we can actually push it into other sources like BigQuery and Datastore. Okay, now that we've had this whole pipeline set up, let's go ahead to the last step and show you what it looks like in terms of the output and BigQuery. We have our data, event_data and this is where all the data coming out of Dataflow gets pushed into this table. So, we've created a dataset called iot-airlines-v2 with our table events_data. So, you can see here this is what the schema looks like, it includes customer ID, the event type, things like customer name, we also have how many bags are checked, how many seats are assigned under their name, how long the trip was, what the origin is at the destination. Let's just do an example really quickly of how we can compose a query on this data set. Certain routes might be more likely than others for certain upsells like for example if somebody's flying out at six in the morning on a Monday, maybe it's a business trip, you can offer them to buy Wi-Fi, you can also upsell them if customer's carrying bags or has a connecting flight, for example, maybe they want to upgrade from coach to first-class. So, let's go ahead and just do an example query just pulling customer emails, IDs, and see if there's anybody who has actually did a back check for example so we can potentially upgrade them. So, let's go ahead and open this. I have my example query here and you just input it directly into this query box here and run the query, we are selecting customer email, ID from this data set and table event_data and we're looking for customers who have more than one bag and the tripleg has more than one. You can see here this is just fake generated data but in a real case scenario, might have a different list of customers and their ID so that we can immediately email them and offer them another service. Well, that was amazing. Thanks for sharing all that information. Now I know when's the best time to get upgraded because I'm tired of sitting in coach. I need to sit first class. All right. Well, here it's [inaudible] in style. Thanks again everyone for watching TAKE5 with us. Please comment, like, and subscribe below.
