Lessons Learned Teaching Undergraduate Astronomy with a Video Game with Andrew Mshar

Hi, I'm Andrew Michar, and I want to talk to you about the lessons I learned using Django to teach astronomy to thousands of students with a video game. I want to start by giving you the background of what the game is. It's called the University of Mars and the company we founded in order to sell the game. Astro Venture. And then I want to talk about the lessons I learned. I want to talk about this from a perspective of what I think worked really well for this project. And then what I might do differently if I were starting over today, hopefully to inform some folks who might be starting a project now or in the near future. I'd like to note that this has been a side project, so some of these lessons learned may be a little bit different for

someone that was working full-time on this. So I want to talk about managing infrastructure again from someone who the point of view of someone who this is a side project. Also, I don't have much professional experience doing DevOps, and so these were choices I made to minimize that burden. Software testing, how I thought about testing for this project, how to improve, how I improved testing as I went along, and what that has helped us do I want to talk about Django Rest Framework and how we've used it in this project and how I might start a little differently next time using Django Rest Framework. And then a few quick points about error tracking, logging, and documentation.

So the beginning of this project, the history of it, goes back to before 2007, but we'll start there. Professor Jane Charlton at Penn State University. Has always wanted to create a game to teach astronomy. And around that time got a group together to try to do that. But the game development tools weren't quite mature enough at the time. And so they instead chose to create a course That was based around a story. And so they had static web pages that told this story with uh artwork to kind of immerse the students in this story. And then multiple choice questions to make sure the students were gathering the knowledge they needed as they went through this story. A few years later, I returned to Penn State.

and started working on a full game version of this. Luckily the tools had matured by that time, uh including Unity 3D, which is what we built the game in. And so we were able to build the game in a few years and started teaching it around 2014. I ended up leaving to pursue other professional endeavors in 2015, but a few years later. The university reached out to us and asked if we wanted to license the game and so and to sell to other universities. We decided to do that. We found Astro Venture, our company, and that includes myself Dr. Jane Charlton, I mentioned, uh, Knox Turrell, who is the artist who developed uh the vast majority of the artwork in the game, and Dr. Chris Palma, also from Penn State. Now that I've given that history of the game and where the company is at now, I want to give you, I want to show you the game and talk a little bit about how it functions before I dive into the lessons learned and the Django side of things.

So this is the game. It takes place in the future, in a future where we have colonized Mars and the students take control of a character. Who is one of the first students on Mars? So it's not a vast uh Mars colony. We haven't taken over the entire planet, just the first colony on Mars. And the students uh play as one of these characters. and their digital classmates engage in discussions around the topics. And that's how we have a nice pattern for how we teach each lesson. So they start with a discussion among those um the playable character and a couple of the non-playable characters and then some kind of interactive element or mini game um that reinforces those concepts And then a quiz.

So for this one, you can see this interactive here for Kepler's laws. They're able to mimic the graph there with their data. And then they uh roam through the solar system and they follow uh those interactive elements or mini games with a quiz. And that quiz ensures that they got the information that they need out of each lesson. The quiz data along with the data for their entire save file is saved in our database And that's so that if they need to switch computers or if they have some kind of data loss, they're able to pick up where they left off and able to play that way. The quiz scores are also accessible by their instructors, most of whom use it for a

small portion of the grade. Um and so those are the main backend tools for the the um the game itself I think that's everything I want to say about the game. And now since this is a Django conference, I can talk a little bit more about the specifics of how we use Django for this game. So uh the original version of this game uh when we developed it at Penn State, uh I developed some back-end code. Uh we had a small managed server, I don't remember where we got it from. Um and I just used some PHP scripts and then Apache server to handle that. Um since then, when we founded the company, I decided I had some professional experience with Django.

I decided to move into using that and moving the code over to AWS. You can find our website at theastroventure. com to see what I'm talking about here. But I want to tell you a bit about uh why we made the moves we did and what was successful for us and maybe what I would do differently. So I'll start with infrastructure. So you may have been asking, okay, why AWS? Why did you choose to go there? The primary reason was we founded the company using Stripe's Atlas program. For a small fee, they incorporate your company and give you all the documentation and set you up with a bank account. I can recommend that. But the big a big portion of that was they gave us free AWS credits for two years.

Judging by their website, I think that's still going on. I couldn't get the specifics, but it looks like they are still offering AWS credits. I also had a small amount of professional experience with AWS, so I felt comfortable enough starting up with that. I started off by creating an EC2 server and setting it up to serve the Django application. If I were doing this again today, I would probably start with Elastic Beanstalk on AWS. If you're using something like Azure or GCP, I'm sure they have other similar services to Elastic Beanstalk, but they allow you to deploy your code a little more easily than having to just set up the server yourself. If you're not so comfortable with DevOps, if you haven't spun up a server at all yourself,

these are some services I've seen recommended by other folks that use Django and have recommended for folks just getting started with Django. So maybe look into those. In terms of um how I set up these environments and uh our servers. The first thing I want to talk about is different environments. The first of which is local. That's just my PC here. Then production, that is our live server. If you go to theastroaventure. com, you'll be interacting with our production server. And then finally a staging server. For us, that is a server that is set up almost identically to production, but lives at a different URL so that when I introduce new code or any changes to the infrastructure, I do them in staging first.

And that allows us to catch any bugs we might that might be different in terms of uh setup from our local setup to our live setup. Um Highly recommend having an environment like that, a secondary environment where you can catch bugs like that. I also uh make sure to turn that environment off when I'm not using it. And therefore the cost is almost nothing. It's maybe a few dollars a month as opposed to the live environment, which you pay for the server being live all the time and the database being live all the time. But with staging and turning that off, you're able to save a few dollars there. In terms of my local environment, I use Windows and so I use Windows subsystem for Linux.

which is a Linux virtual environment that allows me to closely mimic the servers that we use, both for the servers and for WSL. I use uh Ubuntu as the Linux distribution. Um now because it's a VM on my local machine it I'm sure there are some slight differences between that and the live server. If I were starting this over again today, I would probably start by using Docker. When this project started, I didn't have any professional experience with Docker and it was a little harder to get it set up with uh Windows uh WSL. And so I I did not start it at that point. But if I were starting again today, I'd probably do that.

As I mentioned previously, the um I started the server and started installing things manually. So that was our first server that I used. And as I was doing that, I wrote down every step into a Google Doc. That was our first staging server Then to create the production server, I followed those steps and then fixed anything or clarified any notes that didn't make sense to me at the time. And so that Google Doc served as insurance for me, uh, making sure that if any time if there was ever an issue with the server, uh and I got it into a state that I couldn't recover from that at least over a couple hours I could recreate it going through these steps without having to

figure anything out again or remember what I had done. I was able to just step through these. Fortunately, we never ran into a catastrophic event like that. But having that backup just gave me a lot more confidence that if something were to go wrong, I could at least recreate the servers. Since then, I hired a contractor briefly to help us move from that kind of more manual setup to what we refer to as infrastructure as code. And if you start with something, some other type of infrastructure, I'd highly recommend moving in this direction when you can. Previously, in order to deploy our code, I would have to log into the server, manually pull it down, restart

the um the server. But with infrastructure as code, with something like Terraform and Packer, I'm able to simply redeploy the server, recreate it, and then replace the old server with the new one with the new deployed code. This has been incredibly valuable. It makes me much more confident in deploying the code that I'm not typing something wrong because if I'm typing something wrong, it's locally and it just won't send it won't deploy the new server. It's much more difficult to break something using these. I will say that Terraform and Packer have worked very well for us. There are other tools that people use professionally for this kind of thing. Ansible and Chef are two that I'm familiar with.

So you're not limited to Terraform and Packer, but they have worked very well for us. One last story I want to tell about this. About a year ago. Um I had a serious issue with one of the servers. For some reason, um something was happening to cause um the CPU to spike. So we were having a CPU spike and then the CPU uh percentage utilization was pretty high and stay high for a while, uh or pretty much indefinitely. And so We were able to fix this temporarily for a few hours or a day or so by just resetting up the server. And so being able to do that with a few keystrokes Made it much easier having these scripts set up and then we were able to debug and figure out what was going on and

get that but to a better state But if I would have had to do that manually every couple days, it would have been a whole lot more stressful and difficult. So highly recommend moving in this direction if you can. It just gives me a lot more confidence in deploying our code and is worth the investment it has been for us. Next, I want to talk about testing and how I've tested this project. and uh how I got there. Um so there are many different ways to think about testing a project. You can use test-driven development. Um some folks like to strive for a certain cover number or percentage of uh test coverage. Um But for me, the thing that drove me to improve our testing was getting to a state where I felt confident

changing and deploying the code. I realized early on in the project, uh, once I had everything set up, I didn't have a lot of test coverage. Um, that's something I would do differently this time. I would start getting tests prepared earlier. Um but After everything was set up, I had the features I needed, people could purchase the game. I found that when I would make changes or add new features, I would ask myself. It's the middle of the semester. Do I really want to deploy this? Or can I wait and bundle a few more things and do all of the tests and run all the tests and make sure everything's okay at the end? Because my testing was largely manual And so I realized that's not how I wanted this project to be.

I wanted to be able to be confident that if I made changes, that when they went live, I wasn't breaking something. And so I want to talk about how I went from that lack of testing to a much better testing setup now. First, I'd like to define the types of tests that I'm talking about just for so we're on the same page about those. First, unit tests. These are tests that test the smallest unit of code that you can. A good example of this would be a test of just a single endpoint with not much processing. Just What data comes in, make sure that the data that comes out is what you expect. Integration tests are bundling a few of those units together. think of your process or your purchasing flow on the back end.

Usually you might have to reach out to a third party, verify that, maybe add the person to your database, check against your database, do some verification on the data. There's usually a few units that fit together for something like that. That's a good one for an integration test. End-to-end tests are testing something including the front end. So checking What data comes from the fr or inputting data into the front end, making sure it comes to the back end, and you get back what you expect from the back end to the front end. For those tests, uh we use Selenium now. Um if I were starting over today, I would be looking into playwright. Uh I'm actually planning to do this soon. Um I haven't really

dove into playwright yet, but I've seen a lot of people recommend that as a a good system for running end-to-end tests. So that's where I would start. But Selenium has worked fine for us. I will say that I really like having end-to-end tests, specifically around critical pieces of the code. For us, that's the purchasing workflow. So making sure that after I've deployed purchasing still works the way I expect. That's been very valuable. I wouldn't encourage you to do end-to-end tests for everything because end-to-end tests tend to be a little bit brittle brittle and finicky. and require a good bit of maintenance because they touch so so many components that anytime you change anything, um, you're you could break them.

So despite that downside, I have found having a few end-to-end tests for critical pieces very important. And that's something I would have set up much earlier if I were doing this again today. In this project, I don't have many integration tests. That's mostly just because I don't have many pieces that aren't covered by the end-to-end tests that require an integration test. A lot of the rest of it is smaller endpoints that are pretty direct that are covered by unit tests. In other professional contexts, libraries I've worked on, et cetera. I've used integration tests more extensively. I definitely think they're valuable. I just don't have many in this project. Unit tests I find valuable for two reasons. First, when I develop code and develop a feature, I find that almost no matter how much planning I do, when I start writing tests for it

I think about it differently. I run across something that I missed, some assumption or some side effect pops out, and I find that helps me redevelop that code. Whether that's a refactor or just catching an exception that I hadn't thought of, usually writing unit tests gets me to rewrite my code at least a little bit. The other piece is like I said that confidence in changing my code. After having unit tests for significant portions of the code, I feel much more confident when I make those changes in the future that I haven't broken that bit of code. And so the lesson here for me is just I would have written these tests a lot sooner in this project. Again, being a side project, I put it off a little longer than I should have.

Next, I want to talk about Django REST Framework. First, when creating this project, I decided not to use Django for the front end. Our frontend consists mostly of uh jQuery, Bootstrap, and HTML. It's a pretty small, simple front end. And so I I didn't want to do anything complicated. Um, and I also but I also wanted it to be separate so that if someone else was working on it. They didn't have to necessarily learn Django. They could use HTML, JavaScript, and CSS. As a result of that, in thinking about what this application would be, I decided to use Django REST framework for the API Now, at this time, I had professional experience with Django. I didn't have professional experience or much experience at all with Django

Rest Framework. So I started using their tutorial, but then I just kind of dived in and started developing some endpoints and uh making some models and ran into a few issues. I think the primary issue I had was not remembering to sync my models with my serializers. This wasn't immediately obvious to me, but in hindsight, I think I was moving a little too fast. And so the advice I would give is if you're diving in with Django Rest framework, start with their tutorial, actually go through the whole thing. And then maybe do another tutorial, some some project that looks interesting to you. There are a lot of tutorials on the on the internet uh with Django Rest framework. And once you finish that, extend it.

Change some data in the models, change some data types, add a field, something like that. And then make sure to fix the serializers based on that. Maybe give different data back or filter the data that you're getting back. Make some changes to that so that you have a better understanding of what's going on with Django Rust framework That was the problem I ran into when I first started using it. I just didn't realize what was going on behind the scenes. So I would have taken, looking back, I should have taken a little bit more time to spin up with Django Rest Framework. Having said that, where Dango Rest framework is very valuable to us now is endpoints that do exact it match exactly the data model that we have. Grabbing the semesters, the data for each semester, what courses are available, what courses are available to a specific instructor, etc.

When we're pulling down that basic data, using Django Rest framework allows me to write less code And makes it very clear what's going on there as opposed to a little bit more boilerplate in vanilla Django. There are a few cases where uh I've decided to stick with vanilla Django, and that's because There are cases where the data coming in specifically from the game part of or the game, um it's it's data that is uh largely strings. And should be integers, dates, other things like this. And that's what we have in our models. But I need to do some processing. It actually touches several different tables. When things tend to get more complicated like that, In in that instance, I've used Django.

I don't always use it when it's more complicated like that, but I just want to encourage you to think about using both. Um it's not necessary to pick one or the other. In fact, the the versus there is a bit of a misnomer. It should be or and or. Um so that's what we've taught we've done in this project is embraced both Django Rest framework and Django depending on the situation. Finally, just a few tips and things that I found helpful for us as well, starting with error tracking For us, I've been using Sentry. That's been valuable for us. We fit within the free tier. I know there are other options for this, but Sentry has worked for us.

But feel free to use something else. But the main idea here is we once I installed Sentry and got it up and running for us, I ran uh I, you know. Found a few errors that weren't really anything important or uh giving me any signal for what's wrong with the application. Um got rid of those And now when I see an error, it's usually something that has gone wrong with uh a user or a purchase flow or something like that. Um Having this kind of error reporting means many times I'm able to reach out to one of our users before they reach out to us. We have some protection on the front end for users not being able to make a duplicate purchase, but every semester there's one or two students that somehow it it ends up coming through as a duplicate.

And it charges their credit card twice. And I feel terrible terrible about that. I've tried to make other fixes, but uh when they come in, I get an error. And um I reach out, I refund them immediately and then I reach out to them and they have been very receptive to hearing, oh, I didn't even know I had a second charge. Thank you for the refund. As opposed to, hey, I really need you to refund this, you know. I have this on my credit card, this is affecting my bill, something like that. So that has been very valuable for us. I also highly encourage you to set up logging. And that Basic logging is important, can give you a lot of information, but there may be other events that aren't necessarily errors, but you want more data from that are happening on your server.

Uh that um you could log. And this is something I'm actively working on for this project. I'm trying to get a little bit better information about the the flow that people have when they're hitting edge cases that aren't necessarily errors, but um I want to log out so that they can that I can follow what they're doing. um in order to debug in the future. So this is another thing I'd highly recommend that I'm actually actively working on for this project. And lastly, I talked a little bit about this earlier, but in terms of documentation, I have found that more is better. I was talking earlier about documenting our process for the API or for the infrastructure I also have some great documentation for how we use the admin ,

how we deploy code, things like that. And I found that incredibly valuable for Just being able to either point someone else to it or remind myself, oh, how did I do that six months ago? Um and I find that when I don't have documentation for those kinds of things and I have to re-remember or remember or re figure out what I was doing. I usually write up documentation so I don't have to do that again and save myself time in the future. So with that, I want to say thank you very much for listening. I hope that if you've started a project recently or you're about to um that you've gained some valuable information here. If you have any questions or want to talk more about this, I am on Twitter at programmylife. Or you can email info at the astroventure.

com, and I look forward to hearing from you. Thank you.