Django Supporting Virtual Reality Game Development by Rudy Mutter

Speaker 1: Come on, no, yeah, yeah, yeah.

Speaker 2: Sweet, so this talk uh is about how we took Django, uh, our favorite tool, or our backend, our web frameworks, and Unity. uh to create a game. So real quick, I was just introduced. Uh co-founder, head of tech at Yeti, product development firm Yeti. I also organized the San Francisco Django Meetup group. Been doing that for a couple years now. Generally full stack developer kind of start a lot with Python and Django, but I've done Swift, iPhone apps, JavaScript, Angular, React, Java, Android, and then recently added C sharp when I started to dive into some Unity. As a kid, I loved games. I'm sure many people here also felt the same growing up. I made games in RPG maker. I don't know if anyone is familiar with that from a long time ago.

Speaker 2: Uh I made this like mutant cow, you were like a farmhand game. I don't know. I still gotta save somewhere on a PC. Uh so it was really exciting that uh we t undertook uh making a game at Yeti So quickly to get some some context, uh we want to make a game, we wanted to try out virtual reality, we're an agency, um, and it was like we oh We want to use Django. How does Django work with Unity? We want to create an API, add some more dynamic functionality, be able to have users in the game. How do we hook that up with Django? So generally, we're going to go through today. We got 45 minutes, uh, so a longer slot. So we got an introduction to virtual reality.

Speaker 2: Uh talking about our game a little bit just to understand the game mechanics and why we're using an API, uh how virtual reality in Unity works, uh some quickly going over the Django APIs. And then how really the meat of this talk and the takeaways is how to consume those APIs in Unity. So really quick, smaller, smaller aspect ratio, my slides are gonna be all messed up. Uh what's not in this talk? This isn't a Unity 101 talk. We're not gonna be talking about how do I build my first game And Unity. There are plenty of tutorials and other things you can go. That would take a really long time in itself for the APIs. We use Django Rest framework. Plenty other talks and tutorials. I'm not going to be going in in depth on how to use DRF.

Speaker 2: So this is not like a step-by-step walkthrough, let's cobble together our first game and talk to Django and DRF, that's way too much Uh I will provide a bunch of resources at the end of this talk though, where you can go to find those things. Uh what is in this talk, uh the idea is to be familiar enough And to know right now what you don't know about maybe Unity or virtual reality development so that you could go, you know where to look to get started, uh and really that you don't make the same mistakes that cost me like many weeks of trying to figure out how to get Unity to talk to Django and just getting familiar with it myself coming from a predominantly Python and Django background. Also, there's a quiz uh halfway through the talk. So you are motivated to pay attention.

Speaker 2: Uh the quiz will be happening either via your laptop. tablet or phone, so make sure you're ready with that halfway through. All the questions are gonna be and the answers are gonna be things I went over. The top two people will get a Google cardboard Uh I came up with this idea rather recently, so I don't physically have them. I will ship them. I know, I know. They're back we have a bunch back at the office. I was like, I wish I had this idea earlier, but uh we will ship them to you wherever you live. Uh inter international. We'll have to see how much that shipping costs. It's only a piece of cardboard. But so the top two people uh will get some prizes, so you should pay attention. Um So quickly I want to talk about basically virtual reality. Um

Speaker 2: show of hands. Who would say they're familiar with the different virtual reality headsets and platforms and what's going on? Okay, maybe maybe half. Half the audience says they're familiar. So um this is gonna be painful. So the uh brief state of virtual reality. Um there's a picture missing off to the right. Uh so right now basically there's the there's virtual reality headsets that deal with using your smartphone And there's ones that are likely connecting to your uh uh computer. And so going from left to right there, we've got the Vive, the Rift. PlayStation VR, Google cardboard, and the one off to the screen is the Gear VR. Um so generally

Speaker 2: the PlayStation 1 obviously hooks up to PlayStation. The two on the left are currently you likely have a gaming PC and you're hooking those headsets up into your PC. The two on the right, the cardboard, is you put your smartphone into it, either iOS or Android. Uh and then the gear VR which is off to the right there uh is for your Android phone only currently. So a lot of people have dubbed this year as the year of VR. So mostly all of these headsets shipped. The Rift and Vive uh started to ship out to all the people that pre-ordered it Um the Samsung Gear VR started to get packaged with actual Android phones that were being sold. The New York Times sent out to their subscribers.

Speaker 2: a bunch of Google cardboards with the with the with their subscription and they also came out with an app that had a bunch of kind of like 360 experiences for you to use in the cardboard. And PlayStation VR, kind of the last one, is due out at the end of the year. So there's kind of this desktop versus um mobile. So some people consider smartphone VR, i. e. the cardboard or the gear basically as like virtual reality light. Uh and it's pretty true. I mean it's hard for you to use your smartphone and repurpose it by putting it in maybe just a piece of cardboard versus buying a dedicated headset and controllers that you plug into your computer that are taking

Speaker 2: the full effects of your graphics card that's on the computer, all of the things built into the headset itself. Google Cardboard actually rebranded as Google VR kind of recently, so I might interchangeably Call it Google Cardboard or Google VR. Essentially Google's trying to come out with maybe another controller and their own thing called Google Daydream. And probably cardboard maybe isn't the greatest name if they want to make that a solid platform that they have marketing behind. So they're trying to rebrand, but when we did this several months ago, it was not Google VR. So you'll see me referring it to cardboard, Google VR Um a big concern with this VR light, hence the pumpkin that's throwing up on the side there, is the is feeling nauseous and this screen

Speaker 2: door effect. So if you took your smartphone and you put it up right in front of your face, uh the resolution isn't good enough, you will see the pixels. Uh that are actually on your phone. And so essentially that's called the screen door effect. Some of the dedicated headsets, right, that you plug into your computer uh have improved like the field of vision and the resolution and They've come they've tried to come ways to get rid of that screen door effect and make you feel more immersed so you less of a chance of getting sick while you have it on your head. Um Probably the other important fact also is that on mobile you have limited inputs. So uh the Vive and the Rift and PlayStation VR are all gonna have actual other handsets or controllers. that you are using while playing with it.

Speaker 2: The cardboard, for example, has just one interaction point, a button, and the gear has some some buttons in touchpad on the side, so a little bit more than cardboard But because you're you for cardboard you actually have to hold it up to your head, it's not even strapped on, your hands are are not free to be using other controllers. So those are all concerns when trying to make a game and thinking about your game mechanics. Um So there's some other applications right now for VR too. Obviously games was the first one. Um but people have started to do a lot more around entertainment and media. Uh there's 360 photos and videos. There's a whole bunch of cameras that have come out. There's some interesting things in like education and healthcare and also travel and real estate. You imagine you could go and

Speaker 2: explore some house via virtual reality sets or you could go see where you might be staying somewhere. From one of like the early demos, um this screenshot is this uh Titans of space Uh which is a cool demo someone put together for one of the original Oculus uh development kits where you basically fly around a bunch of the planets and it's uh narrating you through them, talk telling you about the planet, the size, how far away it is from the sun, etc. Um, so there's a lot of possibilities with education in the VR space. as well. So we're going to talk about uh the game really quick, just so that everyone has an understanding. So disclaimer, this is the first uh this is the first game I've ever really built and launched on uh an actual platform. Um really it was an experiment to try out virtual reality.

Speaker 2: We came up with a proof of concept that seemed like, hey, uh this is pretty fun. Maybe we should build this out further. And then we actually created uh what we like to call at Yeti an MLP, minimum lovable product versus minimum viable product, which doesn't mean necessarily that you're using users are actually going to enjoy using your MVP. So we tried to come up with something that might be fun for someone to download, try out on uh on their phone. And for us as an agency, this was experience with new tech, uh a bit of a portfolio building thing, and for fun. Uh we have basically like 20% time at Yeti and this probably spilled over 20% started in our 20% time as a project and then turned into a real one we wanted to see through uh onto the app store. So why do we choose Google Cardboard?

Speaker 2: Um accessibility. It's only $15 for the headset, the piece of cardboard. At the cheapest, there are some more luxurious Google cardboards you could buy compared to spending hundreds and hundreds of dollars right for the dedicated ones that you would plug into your computer. Uh there's a greater potential audience, essentially anyone that owns a smartphone can just get a $15 piece of cardboard and become a virtual reality user. The game we came up with, which I'll talk about, has pretty simple game mechanics and doesn't necessarily need to take the full uh control of having a bunch of controllers um or any kind of room scale VR where you're actually moving around. The idea is you're you're standstill Uh and it generally it seemed more approachable as a first-time virtual reality game that we were building.

Speaker 2: It seemed much easier than maybe tackling all of the different inputs and other things going on with the rift or the vibe. So the name of the game is Tiny Eye. It's essentially if you've ever as a kid uh played the game I spy with my little eye and you said something and one of your friends uh picked out something else nearby. Um it's You're in this 3D world. Um you're looking around. It prompts you with four objects you need to find. You need to look around all around you. Essentially you're in these miniature worlds. The levels were actually made with uh toys themselves. Because of the camera and their perspective, you look like the size of like a NASCAR, like a a little car toy, or you look smaller than a doll, for example.

Speaker 2: There's a bunch of hidden objects. There is a mode that you can play without uh a Google cardboard in it. So you can just basically like look around in space, or you can have it up and actually be using uh the cardboard Um we uh you basically play through the levels, you unlock them, there's different uh difficulties, some objects are harder to find than others. So really quick, just so everyone excellent. Oh there's audio and it's loud. So basically this is Ellie from our office. This is what this is one of our proof of concepts that we made first. This is not the final version.

Speaker 2: But the idea is she's like looking at two levels right now that we we made in our proof of concept. The TV is actually hooked up in streaming what the phone is in the cardboard right now that she's using. So essentially you pick a level, you go in, it prompts you with the objects that are there So a lot of our proof of concept stuff was just like, how does this work? How do we get um a 360-degree photo mapped to a Google cardboard app? Um we had a lot of a lot of things that the designers weren't happy about that I had to polish and work on, animations, interactions. But really so she's just looking around right now in the world, right, and trying to find some of the objects that were uh specified there.

Speaker 2: Uh the actual levels themselves, I didn't partake too much in it, but I know uh the people who did, the office had a lot of fun. They literally took toys and made uh little worlds. Um we use this camera on the right here. It's called a Rico Theta S. Uh 360 cameras surprisingly are not uh as expensive as you would think. There's people that put together like a GoPro rig where you can You put like 10 GoPros in a circle, create these large 360 photos, videos. They're starting to come out with smaller and smaller cameras that are cheaper and cheaper. So this camera just basically has two lenses on each side that stitches together the photo into 360. So really we put together these little levels uh with toys.

Speaker 2: and then essentially just put the camera in the middle of the level and took a lot of different trying to it took a lot of figuring out which objects in the photos are gonna be the ones that the user was gonna have to find, uh making sure that things were the right distance away from the camera. meant our design team got to play around with toys for a week or more. They had a lot of fun. So we had a lot of different themes. Some of them got pretty elaborate. This one on the left is like a different landmarks of the world type of one. So that was uh that was a lot of fun putting the levels together. We tried a few different types of concepts until we landed on this. uh one called Dino War, which was Dinosaurs versus Army Men. Uh and that was when it clicked

Speaker 2: that having themed toys and it being in this miniature world was was a lot of fun Uh and so once we got that first one down, then we just started getting kind of sets of toys and setting up these setting up these levels. Um So tiny eye, kind of a little overview of the architecture. Uh the back end, the API is Python 3. 4, Django 1. 9. We use Django REST Framework for the APIs. Uh for Unity, we use Unity 5, which is the latest version. We use C sharp, not JavaScript. You're able to use both in Unity, and at first I was like, well, out of the two, I guess right now I'm going to pick JavaScript. And I quickly stopped and switched back to C C sharp. There is some concerns about you writing code in JavaScript and it integrating with other plugins or other code that you're using

Speaker 2: that's using C sharp. uh especially like an order of uh execution and if you want one that needs to import another one. There seem to be a lot of complexities and a lot of uh example code and stuff you're gonna find on GitHub and whatnot is all written in C sharp. So I took Took the plunge. Um we use Postgres for the database on the back end. Uh we did deploy it cross-platform through Unity, which means uh we built it for both iOS and Android and deployed it on both stores. And we use Google Cardboard's Unity SDK. That's essentially the thing that helps make it run on the phone and show you those two different uh eyes, which I'll talk about more later. So some other services used, and we uh got to use a lot of other common stuff that we use on most of our other apps.

Speaker 2: So Fabric, we have some like Twitter integration, but specifically we use their statistics Reporting and Crash Lytics. So they Fabric has an integration with Unity. We use it also on our iPhone apps, for example, our Android apps for error monitoring, uh pushwhush for push notifications. We use a lot of just Unity. Unity has a whole Unity services section, so we use one for in-app purchases that they have. And then we use Sentry for any error reporting on our API on the back end. All right, so quickly talking about just VR uh in Unity. Unity basically is is making a big push to include virtual reality just as part of the platform, trying to have a very much a batteries-included type approach.

Speaker 2: Unity is all about crass cross-platform development You try to create your game and it's gonna work on uh many platforms. Um so right now it has built-in virtual reality support for Rift, Vive. Gear, um PlayStation. Cardboard's basically the only one it doesn't have built-in support for. It's coming soon. Instead, you just have to download uh SDK from Google that you just put into your Unity project. But it's essentially the same as building your 3D game. The key differences are thinking about the controls and the inputs are quite different than maybe someone using a mouse and a keyboard, for example Uh and it's really all about thinking about the person with their headset on and that the camera in your game uh

Speaker 2: is really the person's head. And so there's a lot, uh it's really the key to just thinking about virtual reality game development for me and Unity. So uh really what we saw is you've got the two different uh perspectives for each eye. And really the it's all about this camera object that you create in Unity and it has a different camera for both your left and right eye. Virtual reality works by just offsetting what each of those eyes are seeing to give a perception of depth uh in the virtual reality world. Um so Google Cardboard comes with this camera object that you just use in Unity. These pre-built objects are called prefabs. So there's this camera prefab that you can just use. Uh and it's also key for if there's things in the UI that need to always appear in front of the user's

Speaker 2: gaze and where they're looking. So for example, if you want to have some kind of cursor or target or reticle in front of the user, you can apply that to the camera and just make sure it's always um X whatever units of measure you want to say uh in front of where the user is looking. So that's kind of key about how to think about Um virtual reality development, how it differs from regular game development Unity. The other issues we came across, um, it seems if we were want to port this game to uh gear or vive or rift. Right now it doesn't seem super straightforward to build one game and then have it build for PlayStation Rift. vibe, gear, and cardboard. I think maybe Unity's ultimate goal is to get there.

Speaker 2: But right now we definitely we definitely looked at how Unity was working and we picked cardboard. And so If you were to try to make a game, I think you're going to want to pick one to start with and worry about making it cross-VR platform, if that's a phrase, uh later. Uh because the inputs and controls differ greatly. Um For cardboard, you can, like I said, deploy to both Android and iPhone. The trick is for Unity, uh a lot of it uh you have to pay for all the different add-ons and plugins. So worth noting, if your end goal is to end up deploying this onto one of the stores you're gonna have to be paying for Unity Pro and you're gonna have to be paying for both the iPhone and the Android add-on if you want us if you want to deploy to both. So it's a caveat, you can actually wait to purchase all those things right until you're about to deploy on the App Store.

Speaker 2: Uh tip, if that's what you're thinking about. Uh all right, so quiz time. So on your device of choice, uh go to Kahoots. Dot it is gonna be an interesting exercise. I can't control the volume, so this might be really loud. My apologies. Alright. So ask you for the game pin. So we're gonna do is top two people. Um I will ship some Google cardboards too. So not only does it count,

Speaker 2: there's only five questions, you got twenty seconds to answer each question. But points are not only about getting the answer correctly, uh, it's about getting it the quickest. So we're not going to end up with like 20 people all getting all five questions right. All right. How are we doing? Anyone need more time? Huh? Yeah, yeah. Alright, alright. I think we're gonna start. We're gonna start. It's we're still in names are still coming. I don't

Speaker 2: All right, we're starting. No, uh, uh All right, which headsets has the least amount of inputs? So you gotta click the right corresponding answer on your device. We didn't we didn't even need twenty seconds before on top of it. Cardboard. It's only got the one button. Everything else at least has controllers. Even the Samsung gear has multiple buttons and a touchpad on the side. The one the one button on cardboard

Speaker 2: uh makes it kind of really difficult to make any complex VR uh interaction. Alright, there's the top five. Be slick and dane Alright, next question. Which headset is not yet released? Two people didn't answer. Uh PlayStation doesn't come out until October. It's the last one to release.

Speaker 2: this year out of the major ones. People are mostly paying attention. B Slick's still up there. But TechTor came in second. What does MLP stand for? I quickly Googled MLP when I was putting answers together for this one. Guess which one came up when I Googled MLP. Yeah, trying to trick people on this one. Read carefully. Yeah. Minimum minimum lovable product. Oh Beast

Speaker 2: Leg still just edging everyone out. All right, two questions left, which is not a reason why we went with Google cardboard Which is not a reason we went with Google cardboard. There's supposed to be sound right now. I'm really disappointed there's not any. Oh so it was revenue. We weren't trying to make money. Uh it was it was a little bit it was a little bit of a trick 'cause maybe you thought, well, there's a bigger audience and maybe that means more revenue But the app's actually free. There is like an in-app purchase for a couple extra levels, but no one's really bought that.

Speaker 2: Uh so and we didn't and we we didn't plan on making any money. Um We did go to Google Cardboard because of the game mechanics. There's only one button. Our app was pretty simple. It was actually just mostly around looking at stuff. So um That wasn't oh no, be slick, what happened? Last question. It really might be anyone's anyone's game at this point. A Google cardboard costs about how much Oh man.

Speaker 2: Fifteen dollars. Cheapest cardboard, fifteen dollars. There might be some that are thirty dollars or more. A little bit of a trick question there. People got it. All right, Charlie and Roller are winners. Char Charlie and Roller? Alright, just come see me after the talk and I'll get your Yeah, does Beast like want us? Everyone's got sleepy from lunch, you know? Uh

Speaker 2: okay, so that's a little intro to VR. Hopefully got some understanding of the different headsets, Unity, integration to Unity, what you're getting into. So for the next I want to talk about and get into some code samples of how we integrated um Unity with an API in Django. Apologies, I have a feeling some of the code may be a little hard to read because of the aspect ratio. Um, but we'll see. So who has used DRF before? Imagine Okay, so maybe 75% of the room. I'm not gonna go too much into DRF. I'm gonna just show some of our code. Our DRF implementation is uh pretty uh uh

Speaker 2: straightforward. We're not doing anything super unique. Um So why do we even need an API for our game, right? You might be asking yourself that question. There's level packs, there's levels in each level pack, and then each level has a bunch of hidden objects that you need to find. Um so we wanted to create an API. One, because we want to be able to add new levels and level packs without having to ship a new update to the app store. uh because that's a painful process sometimes if anyone's been through that they've been trying to get make it better. Um we found through testing that some of our objects were too easy So we wanted the ability to actually change the object that people were looking for in the level without having to push an update after we did

Speaker 2: Did some testing. We wanted to store asset bundles. I'm going to talk about that in a little bit. But we wanted to essentially store a bunch of the pictures and images. And sound clips on S3 so that they just get downloaded to the phone. And when you download the app, you're not downloading this hundreds of megabytes app when you first download the app. You download a small app and it downloads the assets as you play the levels, not all of them at once. Uh for in-app purchases, we want an API so that we can set up identifiers. for the level packs that we're setting up in the app in the App Store. So essentially in Google Play, in the Apple App Store, we can create a new app in app purchase for a new level pack, create a whole new level pack with levels and hidden objects all without pushing an update. Uh

Speaker 2: and a little bit of future proofing. There could be more complications around actual user accounts or other things in the future, um, normal reasons why you might need to have a backend to your game. So our models, pretty straightforward. I have left out um a bunch of the fields, but I kept in the ones that I felt were most important for just talking through this. So, like I was saying, we have level packs. Each level pack has multiple levels. Each level has multiple hidden objects that you're finding. in the game. So uh and the key here also is that we uh store these asset bundle names which again just Keep them in your minds. We're also on our models storing essentially these asset bundles and the links to S3.

Speaker 2: Um we also play music, so we also store an audio clip um for each level. For the hidden objects, we actually store where the objects are in space in the game. So if you notice, uh we have fields in there for scale and position. Position. That's so that when we actually load up the level, we put the object in the level based on the data coming from the API. Uh really, this is our main endpoint from the back end, which is just giving the list of level packs. Um I don't think I left anything out here. This is really it. Um Yeah, it returns a list of our level packs. The serializers, again, we just have a bunch of nested serializers. So essentially what this means is when we send down all the level packs.

Speaker 2: We are sending down all of the levels and all the hidden objects at the same time. It's not a lot of data. We could have done it so you get a list of the level packs, and when you click on a level pack, then you see all the levels then we make another API request. Didn't really seem necessary. It's really not that much information to send down at once. Uh we have we do have an API key permission on the endpoints. Really, the r there's nothing secure or private though about this endpoint whatsoever. We did it maybe in the future, there'd be something there. Uh so really this is pretty Pretty transparent, maybe not the most secure, but more or less our endpoint. You need to pass it the correct API key. Store that API key uh in our settings. Maybe that should even be um

Speaker 2: And then that API key obviously is also stored in Unity on our iPhone and Android builds that are compiled. And then we just configure that one URL with the router. So the Django REST framework, again, try not to give a tutorial, but at least quick Our models, the key thing is to think about we got level packs, level packs have levels, levels have hidden objects, they all contain the data that we need for the game to kind of dynamically create some of those scenes. So let's talk about how it works with Unity. Um so when you first open up the app Uh unlike the proof of concept, we decided when you first open the app up, you're actually uh greeted with a 2D list of all the levels.

Speaker 2: Uh originally we thought, well, what if you're just in VR the whole time and you pick the level? With Google Cardboard, there's actually even more fatigue. Uh with V Basically with smartphone VR there's even more fatigue by users than if they have the full headset on because of the screen door effect, people getting more nauseous, you have to actually hold the cardboard up to your head, it's not strapped So you look at all the level packs, you look at the levels, you select a level to play, then you go into virtual reality to play the game. From there you can play a couple levels in a row until we kick you back out and you come back to these screens. So essentially this first screen makes an API request, gets the list of all the level packs, we show them in a list in Unity, and then there's a little bit of logic on the phone to figure out if you've downloaded the assets for those levels yet.

Speaker 2: uh if you need to purchase it, uh or if you can just play the level pack. And so really all that happens here is that download button changes to one of those three words, uh depending. So uh I spent a lot of time trying to figure out from Unity how to actually make an API request. and get the JSON data and be able to use it easily in our app. Surprisingly, it's not that easy. And what I'm going to show you is going to look really easy. Uh but don't let that fool you. It took weeks of figuring out the right package uh and how it's supposed to work. So an issue with uh Unity development is specifically compiling to iPhone and Android

Speaker 2: is that there's certain uh there's certain sh. net libr libraries in the core library that won't work on Android and iPhone. Phone. So there are some built-in libraries that help make networking calls that unfortunately won't actually work on your mobile phone. So that's a huge thing. That wasn't very obvious when starting out with Unity. Once I figured this out, did some reading, I started to look at what alternatives there are, other libraries. I'm like, well This was Python and Django, I'd be looking at what other people do and looking at their blogs and looking at GitHub and getting some examples. Uh I felt that there was there's a little bit of a lack of that community feel in in Unity development. Um and I ended up finding this one library after trying a few others that also didn't work.

Speaker 2: Uh some of them also depended on Xamarin, which is another kind of cross-platform uh library. Uh I landed on best HTTP. The name also threw me off a little bit. I was like is this really gonna be the best library that uh that calls itself best HDP. But it was. So maybe they knew. Maybe Maybe the the person who first made that library was like I'm gonna make the best one. So you actually have to purchase this through the Unity Asset Store. So At first my initial reaction was like, there's a store to buy stuff in Unity if other people made. This seems like this different than what I'm used to. Uh eventually though, I mean I think this was maybe there's also a best HTTP Lite and Pro version.

Speaker 2: I'm not honestly sure what the differences were. But I think one was $30 and the other was $60, and then you can just use it forever and you're game. Comparing that to the already weeks of time I had spent like trying to figure out how to use other libraries or just C sharp and the. NET libraries itself, definitely worth the the time. Um so I purchased it through the asset store. It kind of like downloads into your app and then you can use it. Uh and it was great. The only uh problems I have with it is that the documentation is just a Google Doc. It's it's not great. Let's just say the documentation is not great. Um stuff stuff to be improved there. But so really quick, I mean you probably already glanced at the code once I found best

Speaker 2: HTTP To actually finally make my a uh my API request was super straightforward, uh I give it uh uh the URI, essentially the URL to the API endpoint that I'm calling. Uh HTTP request is an object from best HTTP. It's a class. So I and you instantiate HTTP request. With the URL that your API is going to, and a callback function so that when the API returns, that function is called with your data Uh Best HTTP seem to also have a bunch of advanced um functionality for WebSockets, obviously making rest calls for all the different verbs, post -get, um delete. It seemed really fully featured uh once I started to dig into it, which is great. Uh so this is pretty straightforward.

Speaker 2: This is all it ended up taking once finding the right uh tools to put together. Um so this other thing full serializer, I'm not gonna go through what this is per se, but the idea is uh from other experiences on iPhone and Android and in JavaScript. When we're calling our endpoint, I want to take that JSON data on my client and I want to turn that into actual objects that have that data. And so on a lot of other platforms, there's libraries and ways to kind of automate deserializing your JSON data into actual objects, whatever programming language you're writing in. Um so I was looking for something similar in Unity so I could just have some C sharp objects I that actually had a level object and I had a level pack object that had a bunch of levels.

Speaker 2: So this is just up here for reference later. This is a class that you put together. I think it's partly in the full serializer documentation. This is just a GitHub repo with some open source software that you can use. He is actually using it though. So this is that callback again on request finished that sends back the data once best HTTP has returned. Uh so if I call that other class I just showed you, the deserialize method, I say, hey, here's the data that best http got back. Here's my level pack object, which I show you below here. Um just ch deserialize all that JSON into this object for me and then give me back a list of level packs. And that's great. And that's more used to what I'm doing in some of our other like

Speaker 2: thin client platforms that are calling the API And so like this level pack class that I have below, we also have one for a level and one for hidden objects. Right. Um so kind of the last part really is how do we make that final uh scene that's in 3D or virtual reality depending on how you're playing? Um so that this screenshot is actually what the scene looks like in Unity. So the idea is that we have this very bare bones scene that has some of our UI setup. And from the data that's coming from the API about that specific level, all those green cubes are going to turn into actual objects somewhere in the scene. Um the like the level name is gonna turn into the actual level name coming from the API.

Speaker 2: And all those white squares in that menu actually turn into little icons of the objects you're supposed to Supposed to be finding as well as their name. So all this data is coming from the API. So originally we had a we had a we had like six different scenes with all of this hard-coded in there. And as a programmer, I was like, I feel Like I'm repeating myself over and over again right now. And so it made a lot of sense to kind of abstract it out, put it into an API, and have these levels dynamically load. And this is like this took me a while. I talked to another like experienced Unity developer uh about the problems and uh so this is what we ended up with um specifically also to note the the photos are just these three hundred and sixty degree photos that are mapped to The sphere, if you will, around the camera.

Speaker 2: So in this picture right here, the kind of the white-tiled background, all of that gets replaced with a 360-degree photo that was taken inside of the miniature world. The green cubes are actually just invisible. They're placed in the scene over where that object essentially exists on the photosphere. So it's not like we're actually taking a 3D model of an army man and putting it in there. This cube, we actually just change the shape and the size so that it overlaps on the photosphere with where that object is in the image. So the key here are these asset bundles, which again was this whole concept that seemed very overwhelming at first, because I had never dealt with something like this. Um

Speaker 2: so the game assets are really large. The photospheres in the game are really large, and if someone had download all of these at one time and all the sound clips and everything, it would uh they'd have to be on Wi-Fi to download it. They'd be downloading everything. If they only played one level, they have a ton of assets and taking up a ton of space on their phone that they're not even using So the idea is for them to be able to download assets on demand when they need to. It also gives us the ability to update assets. Maybe we found there was a mistake with one of the uh photospheres in the level. We could re-upload our asset bundles to S3 Uh the asset bundle manager essentially also has another schema file that has some meta information about what version of this asset bundle it is So the game can actually detect, hey, there's new asset bundles to be downloaded, and we'll prompt the user to download the new one and throw out the old one.

Speaker 2: And these asset bundles can actually contain all different things. They can contain essentially just an image. They can contain a whole scene. So you could actually create a whole new Scene, be able to change it kind of on the fly, upload it to your server in this bundled up package, and the game would be able to download it. So a lot of the games, if you've ever played on your phone, are doing things like this. Uh interestingly, Pokemon Go has made in Unity. Uh fun fact. So uh a little bit about the asset bundles um before my time is up. So there's a whole asset bundle tutorial. I've got some links on in a couple slides. So Unity has all these example projects and how to use asset bundles and whatnot, but I am just going to show you a little bit of code that might make some of it easier.

Speaker 2: Because they show a lot going on there. And it's helpful to just see it actually used. So asset bundle utility, this class, is actually a class that's provided by Unity. They give you that you can download through the asset store but for free from Unity is this whole asset bundle package. Utility is a class in there that try to helps it helps manage the versioning and all this other stuff that you You don't have to write yourself, which makes a lot of sense. This function here is specifically about us loading the photosphere. So in in Unity terms, that photosphere is a material. That material is applied to the This uh sphere that is around the camera. So what we're trying to do is we're trying to get from the asset bundle asynchronously the uh material that we're then gonna put onto

Speaker 2: the sphere in our scene. So at first we use asset bundle manager to load an asset asynchronously. You give it the asset bundle name and the asset name So we create one asset bundle for every level pack, and each level knows the names of its assets in that bundle. So it was a better strategy than having a different asset bundle for every level because if you notice on our first level select screen, we prompt you to download a whole level pack. So it made more sense for the whole pack. To be an asset bundle, and it contains the assets for each level. Those levels from our API know what their name, what the names are of those assets in the bundle So we load this asynchronously, once it comes back, this I

Speaker 2: enumerator um in front of the name of the method essentially denotes that this function can be run asynchronously as a coroutine in uh coroutine in Unity. So when it returns, we get a material um from the request to the asset bundle manager, and then we can just put that material onto our mesh renderer, which is You get into Unity, mesh renderer is just the thing that's attached to the sphere that actually renders the picture. So in our actual code, once all of once I created this function inside of Unity 's asset bundle utility, um I would then just call off, I would start start a new coroutine, which means basically call this method asynchronously.

Speaker 2: Because in games often you don't want to you don't want to block the game because you'll the user will just get this black screen. Especially if you're in virtual reality, uh you would cause your user to get Immediately. If all of a sudden they're you're they're in some virtual world, you pause the game and it shows black because you're loading assets in, you should need to think a lot about probably preloading uh assets so that they have a continuous smooth. smooth experience and you're not jarring them with like a loading screen in virtual reality or flashing from black back to like a new world. Things like that you need to be careful of. Um so here we just give it the asset bundle name of the level pack. We give it the name of the photosphere asset itself that might be like Dino War Photosphere.

Speaker 2: Uh and then the actual renderer that we're using in our current scene that we're loading. So a bunch of resources that you can use. So specifically, I learned a bunch of my Unity VR stuff by watching this Udemy Unity VR tutorial. It is really great. It seems even since I've watched it, they've added a ton of content. It looks like they've added content uh also for Rift and Gear VR as well on top of the cardboard stuff that was in there. There is a specific GitHub repo I looked at about how to actually map photospheres to a 360-degree world in Google Cardboard. That was super helpful. A link to Best HTTP, full serializer.

Speaker 2: I would recommend if you're getting involved that you just open up Google Cardboard's example unity project and just click around and explore. That's how I figured out a lot of what was going on. The Rico Theta S is the camera that we used. If you probably wait even longer, there'll be cheaper ones that are coming out. And then the UniCardboard. The tutorial, Unity's got a bunch of tutorials themselves and videos as well, video walkthroughs if you'd like to learn that way. And then the asset bundle manager. And that's it. So these slides are up at that URL. If you want to go back here, all of those are actual links that you can click on to go to there. That's it. I got some I got five minutes for questions.

Speaker 3: Pretty awesome what you did there. Um will this scale well when you have like say a million users?

Speaker 2: Um Yeah, right now, yeah. Because really the API interaction is just when someone opens up the app. There's no because there's no ongoing while you're playing the game. It would be concerning if, for example, we were uh like saving scores or achievements or whatnot. But still it's not super complicated. You would just need to make sure that you were scaling your servers well enough to handle the traffic.

Speaker 4: Hey, why did you choose to query for all the level data from a server rather than just bundle that data into an object in the asset bundle?

Speaker 2: Um so if we went that route, that means when the app opens, before anything happens, it needs to. The app needs to have a hard-coded URL of where to go to get the asset bundle, if that was to happen. And then it would need to check the version of that asset bundle and download it and then figure out the data in there. Um our idea was the API would also grow into more than just having the information for the scene. But it's a pa it's a possibility. It's a way you could do it. Thanks.