Simpl framework, big impact! by Joseph Lee & Jane Eisenstein

Speaker 1: Great, thank you. Um so yep, my name is Joe Lee. I'm the director of the Alfred West Junior Learning Lab at the Wharton School of the University of Pennsylvania. Uh this is my colleague Jane Eisenstein, who's the lead developer of Simple, and we're really excited to Share simple with you today, which we recently open sourced, so we're really excited about that. I do want to preface this talk a little bit beforehand. You don't need to be a higher ed developer. You don't need to be a simulation developer or game developer to get something out of this talk. I want to give a big shout-out to Kendall and Henry's talk. I don't know if they're here. There they are. uh from yesterday about uh Django channels. If that was interesting to you, uh I think a lot of that content will resonate with uh with you as well during this presentation. We're doing a lot of uh similar things and uh but we're building games with it so I think it's really interesting and I hope you find it interesting as well.

Speaker 1: So before we get into the actual platform Uh about the Learning Lab. You don't work at a higher ed university unless you have an overwrought mission statement. So this is ours. The Learning Lab's been around since 2001. We have 33 active simulations that we uh write in support, about two-thirds of which are ones that we've written. Uh they really span uh the tons of different languages. We have simulations written in Perl. He has simulations written in Python, Java, uh Cold Fusion, and uh simulations are written on uh off-the-shelf like commercially available simulation platforms as well. Um And these simulations really run the gamut in terms of subject matter. We have simulations in the accounting space, the finance space, marketing, entrepreneurship.

Speaker 1: We try to touch at least every department at the school with the simulation. Uh and the simulations themselves uh can range from 10-minute simulations. That you use your mobile device for to simulations that are the an entire class. We have a simulation that we run in late August for all incoming MBA students, which is 864 people and we run them through a simulation in the course of four days, which is really an intense experience. It's really awesome. And it really speaks to kind of the reach of the learning lab at the school. So we have um Over twenty our simulations are played over 20,000 times a year. Um and they're also played over 10,000 times a year around the country and the world.

Speaker 1: We have three simulations that are for sale. on Harvard's B Harvard's uh Harvard Business Publishing, which has a simulation platform. So simulations are available to any university that wants to use it. And it really speaks to kind of the reach of the lab and the impact that the lab has with a fairly small team. So Jane's really the only full-time developer that I have. We work with vendors, which I'll talk about later. But we don't have a lot of development resources. So simple, really one of the reasons we wanted to develop it was that it would help us build these games faster. We work with a lot of enthusiastic faculty that you know come to us and are like, oh, can you build this game? You know, I want to use it in a month. And no, simple doesn't help you do it quite that fast, but it does help you do it faster. Uh so that's where we're uh one of the things that we're really excited about

Speaker 1: uh sharing. So what do we mean by a simulation? Well for us a simulation uh is a web-based multi-user application based on a mathematical model. So you could think of it like this. We collaborate with faculty and we come up with uh they either have a piece of research or a piece of pedagogy in their class. And um they'll build a mathematical model around it. So that model can take the form of a 60-page PDF with all these different formula uh that um you know we translate into Python or they could be an Excel spreadsheet that they've been using for years that they want to build a game out of. So Jane will take that information, build a mathematical model, and then we'll build a game in Python and we'll build a game around that. So that's what we talk about there when it comes to Simulations.

Speaker 1: And um business education is really a great fit for simulations when you think about it. We have students that will be managing millions of dollars worth of assets in the real world and it's really much better for them to fail in a simulation than it is to fail in the real world and then kind of bringing that pedagogy home uh is one of the things that simulations are really good at and that's why one of the reasons why uh we use it a lot at the school and I think it could be used on a lot of other places that have don't previously uh use simulations. So, what are the benefits of a framework? Well, uh kind of the basic stuff. We don't need to start from scratch every time we build a game. When we build a lot of our games have been built from scratch before, before we had a play a platform to build on. And those games can take anywhere from six months to a year to make

Speaker 1: because games are still really complicated things to make. And just keeping data in sync and all the plumbing required to make an interactive experience It's not easy. And even with technology having, you know, grown dramatically in the last few years, that's still not an easy thing to do. Uh just can the talk yesterday and channels, it still takes a lot of plumbing to make all that happen seamlessly. And our platform gives you that uh plumbing for you so you can concentrate on making the game. So making um you know keeping data in sync between users and things like that are already done for you. So you just concentrate on building the game. So there are some existing platforms in the space that you can build games upon. There's a few open source ones that aren't really super robust, but they're out there.

Speaker 1: And then there's also commercial offerings. Which a lot of our simulations are built. Again, there isn't a lot of players in this space, but there definitely are platforms out there that help you build games. But we're talking about licensing fees northwards of you know five figures per year. Um they require you know a specific set of skills that uh just really aren't easily transferable. You need to be If you're good at programming, it'll help you learn these platforms, but those aren't skills that you're gonna be able to take to your next job. And um It makes it really challenging for us to recruit and retain developers when they have to work on these platforms where they're really uncertain about what they can do next. Uh and then it creates a lot of stratification in uh higher ed. You know, you have universities like Penn uh that can afford uh the resources to build these simulations, but a lot of other schools

Speaker 1: Don't have those resources available. And we've done this, a version of this talk about four or five times now. And we've done it at other universities in the Northeast. And uh you know people come up to us afterwards and like, oh wow, I've been using this pen and paper exercise in my class for two decades. It would be really cool if I we could build like an interactive game around this. And there just isn't a lot of resources out there for them to do that. I told them it's not, it's not simple doesn't make it uh easy to do that, you know, pun not intended, uh, but it definitely helps get you much further down the road than you could previously. And Python is really prevalent in higher ed from the research angle and reachers perspective. So those Python people with Python still are out there, and there's no reason why they can't take our platform, clone our Reaper's.

Speaker 1: repos, do a couple tutorials and build the game in in fairly short period of time. So that's really exciting. So why did we want to build it? Uh well kind of building on the uh points I just made, we want to control over the platform. Uh We run into a lot of issues uh just working within the confines of our existing tools before we built simple. Um there are things we just couldn't do and as our needs evolved the platform we actually we ran out of uh uh functionality in those platforms to make them viable for us to build games in the future. So if we wanted to build simple, draw upon all the experience that we had. And build a platform that was flexible enough to accommodate not only games that we built today, but also hopefully games that we build in the future as well And we wanted to build a community around this.

Speaker 1: Like I said, there isn't a lot of players in this space, but I think there's a lot of untapped demand. There's a lot of awesome opportunities out there for people to work on simulations, build simulations, and they just don't have the tools for it. And I hope Simple gives them those tools and we'll help build that community around the tool, which will be really cool. Really awesome. And we didn't want to be locked into different the vendor. We have a lot of technical debt in the learning lab of those 33 simulations, maintaining them and they're all you know different programming languages and professors use them and you know forever. like decades and decades and they expect it to work forever. So like building it, you know, on this uh really I we think r really rock solid uh platform will help us, you know, in the future as well. So, what did we start?

Speaker 1: So, we started in the spring of 2016. Um, shout out to the RevSys folks. Uh they were our vendor partner, Building Simple. Uh, Flavio, Frank, Steven. uh Jeff, they were all integral in us building this platform from scratch. Um shout out to Sarah Toms who was my predecessor at the Learning Lab. Uh it was her idea to build a platform to begin with and her idea to open source it so I won't be standing here giving this talk without her. So thanks Sarah. So how did we start? Well we did a lot of whiteboarding. So we took our games and we uh which have a lot of different use cases. And we kind of just mapped out what a model would look like that could accommodate all these games and still be flexible enough for us to build upon, but not so uh opinionated that it would, you know uh harm our flexibility going forward.

Speaker 1: So it was a lot of work on a lot of whiteboarding. The rest of these folks really brought in you know their perspective and they really helped us build this model up from scratch. So their diverse viewpoints really helped us as well So we launched our first simulation, which is the one that uh Jane will be sharing with you today, uh Rules of Engagement, which is a simulation that we do in a marketing class for about 300 people. Uh we run it in the fall. Uh we ran it uh in fall twenty seventeen. Um it may or may not have exploded at during the due date. for said simulation. Uh I always I've worked in high red for over ten years now and I'm still astonished by just how close to the deadline students will wait to do an assignment.

Speaker 1: And I just we underestimated just the amount of load and it just kind of blew up in our face. We were able to, you know, limp over the finish line and delivered for the professor. And despite all that, I still thought it was a success because you know we did it, we got there, we generated tons of very interesting data about the platform itself and how we want to use it and how we wanted it to evolve So that was really cool. And then we basically had to sit down and look at the areas where it fell flat, and we had to rip out a lot of the plumbing and do a lot of refactoring. So it was really our goal to open source it in the first half of this year or late last year, and we just open sourced it now because of all the changes that we made, uh, which we think really makes it uh now a very robust platform. You know, one of the things that we had to do was move the entire platform

Speaker 1: to use asynchronous requests. Uh we also built a profiling framework that actually helped us simulate of the WebSocket, 300 folks going through the, you know, the WebSockets at a, you know, at simulating a deadline scenario. So we were actually able to To profile against our production server that had the old code and then the development server that had the new code and actually see that the gains we were we were getting and we were on the right path in terms of refactoring all of that code. So we went open source, which is awesome. So they we we opened up the repos a few weeks ago. I may or may not be checking the GitHub. uh traffic stats on the games API to see who's cloned it. There's been at least two people, so that's really exciting. And maybe there'll be more than two people after this talk.

Speaker 1: Uh we opened it up on GitHub under GPLv2 and really I couldn't be prouder of the team uh at Revsys, you know, Jane, uh and and the entire Learning Lab team that really helped make this possible. It's over two years worth of work and it was really awesome to see it now live and and out in the wild. Uh one of the great milestones we had um last week is we had a developer who had a React background, which we'll get into super valuable. looking at uh simple. But they were able to clone the platform, clone all the repos, go through the tutorials. In the span of about a week, we're able to make a rudimentary version of a simulation that took us over a year to make from scratch. So that was really a a great milestone for us. They're like, hey, this thing actually works. Which you're in your little bubble of development and you're like, oh yeah, I think this thing works.

Speaker 1: But then when actually someone else says it works too, it's like, oh cool. Work. That's really awesome. So we're really happy about that. So now I'm gonna pass it over to Jane, who's gonna go kind of more into the weeds of simple and gonna show you uh rules of engagement. Jane.

Speaker 2: All right. Thanks, Joe. Yeah, I'm Jane Eisenstein. I'm a senior application developer at the Wharton Learning Lab. I uh have been working for for the Learning Lab for four and a half years and I joined the team because the it sounded like fun to create games that help people learn. And I also am really proud that we're able to take something that we developed for our internal use and offer it to the wider community. So I'm going to spend a couple of slides just kind of going over the concepts of what we mean by a game and because We've got some cool architecture, but Simple

Speaker 2: is really focused on helping us create these games. So we built it to be flexible and support the needs of these games. So as Joe had already covered, games are built around a mathematical model. User decisions go into the model, it cranks. And a step through and then it produces results. And this can happen iteratively, you know, new decisions, results over several periods. These periods with their associated input decisions and output results are grouped together into scenarios. Users, and by that I mean Django users, participate in a game by being assigned to a run of the game as either a leader or a player.

Speaker 2: So a run is an instance of a game for a particular group of users and the way we use it is if a professor has a class or it has a section of a class and they want them to all be playing the game at the same time like two o'clock on Friday the thirteenth, then they're gonna put them in the same run and so that they will they will have that same timing that the run gives them Players in a run may be grouped into worlds, they don't need to be. But often uh if they are grouped into worlds, that's just a team of players. who typically either are collaborating together or competing against each other, but the decisions they put in um of all the players goes

Speaker 2: into the model and the results come out and they're they're shared by everyone in the world. So depending on your UI you could see every other player in the world. You could see all the decisions they had made, uh all the players had made. You could see all the results that came out. And it really is a UI decision about how you want to design your game. Uh the thing about both runs and worlds is they're independent of each other. One run of a game is totally independent of another run of a game. It might have a different set of parameters. It will definitely have a different set of users , least players. And worlds also are independent of each other. What goes on within a world is hidden from what goes on in other worlds within a run.

Speaker 2: All right. Um players are often assigned roles. When they come into a game and that defines what they're able to see and it also can be used to may define the decisions they may make. And we'll see that with rules of engagement when we do the demo. Run interactions occur in phases. What is a typical uh pattern for us is it'll be a setup phase. Oh, wait a second. I need to I didn't talk about leaders and players. I talked about leaders briefly, like teachers are primary leaders, so are admin people. A leader can log into a game and they can see multiple runs and they can see all the data about a run.

Speaker 2: When a player logs into a game, they just see information about their private scenarios or they s and or they the worlds that they the one world that they belong to. And that is quite um a focus. So again, going back to phases, setup phase, leaders would log in, they would be entering parameters, they might be adding players to the game. Uh when a leader moves the run to play phase, that's when players can get in and they can start entering those decisions and seeing the results. And then when the play phase is over, a leader again will move the game to a debrief phase. And that's what I think really to me the secret sauce of a lot of what we do with the simulations at Wharton

Speaker 2: Players run these models, they get experience using them, but then the professor has a debrief session and he looks at results from all the different players or all the different worlds within a run. And he really can use that as a pedagogical tool. Ah, I got the word pedagogical in there. So that leads us to the simple model in the Django sense. And I believe all the concepts that I had mentioned in the previous slides appear as a model class. Except the run user one down over sort of midway down on the right. The run user is really a model that represents a many-to-many relationship between the Django users and runs.

Speaker 2: within the simple database. So a user may be assigned to multiple runs either within a game or across games. And a run may have multiple users. But for any particular run, one user One user is associated with one run in a one-to-one relationship and they either are a leader or a player Um and the other thing here is that I don't know if you can see it, but there's a lot of one-to-in relationships going down. So when the uh model is actually instantiated, at runtime you have a tree structure. Even though it looks from here like, you know, oh gee, a scenario could be associated with the world or a scenario could be associated with a run

Speaker 2: user, that's not really what the platform supports. If a a user, either a leader or a player, wants to run the model in kind of a private environment, then they have their own private scenarios. But if uh but the scenarios associated with a world are are uh shared across all the members of the world. So this guarantees that every Model instance has a single parent which allows you to kind of traverse down from any place up to the top of the game and it allows you to pull out subtrees out of the model instances, which is really an advantage. And I'll just go into the next slide that talks about this.

Speaker 2: When we are running games, we are pulling all the data, the game data that's relevant to a particular user into the browser and storing it in a Redux. State and we really want to hone in and bring in only information that's relevant to the user and not have them getting like updates from things that have nothing to do with them. So it's really cool that we can do this, but like how how does this work? And since this is a Django conference When I'm going to focus a little bit more on the front-end service than um we might do otherwise. Is composed of a front-end service and a model service that are unique to that game.

Speaker 2: And it uses the database service that is shared across all the games. And in the front end service, there is actually a very small Django application. And its responsibilities are authenticating users. And it is once the user is authenticated, then the view uh for that Django app. Well it it's not shown here, but the the front end will actually query the simple database to find out what runs the user is associated with and whether they're associated as a leader or a player And based on that, it figures out what subtree of all the data instances for that game it should pull back and put into the

Speaker 2: into the React store. So Once it's done that query, it hands it off to a single page application and the rest of the UI logic is handled In React in Redux. And from then on, anytime when the React wants to know something about the database or make changes in the database, it asks that It c it communicates with the model service over WAMP, which is Web Application Medic Messaging Protocol, using WebSockets. And so all that communication Goes through the model service to the database service.

Speaker 2: Any updates that happen in the database will trigger webhooks which fire and go back to the model service and which then they broadcast back to the the subscribers for the particular portions of the of the uh data model. Uh let me see what else I want. Yeah, so the model service is um again written by The developer, it's specific to the game, it implements the simulation model in Python, and it also creates custom classes with methods that can be accessed via WAMP And it does that by subclassing off these scope objects that are defined in the simple model the model

Speaker 2: service package. So when the model service starts up, It queries the database for all about all the active runs. It pulls that in that information. It creates these scope objects that um Basically, it creates a copy of the active runs of of the um game in memory. And that means when the browser just wants to refresh a page, it that nobody has to hit the database. It just says, hey, tell me about that, you know, tell me about this portion of of the data objects again, and then it can display it, which Which keeps the load off the shared database service. Um and

Speaker 2: So I think I've covered the to me the the primary things that the you as a developer you need to know about the simple model service is it provides these scope classes that can be subclassed to create the business uh features that you need for your particular game, things like to submit a decision, you do that by creating a subclass of a scope object and say when I submitted a decision, you need to create a decision in the database. And also the web hook callbacks. And now I will go on and take a quick look at the technology stack We went out the door with the 0.

Speaker 2: 7 release using Python 3. 6, Django 111. I'm not sure of the Django REST framework version. The webhooks are implemented using Thorn. The WAMP is implementing using Crossbar I. O. on the Java side. Java side, on the Python side, and Audubon on the JavaScript side. And again, we're using React and Redux to implement our single page application. It is now time for the live demo. Please send good thoughts. Um so All right. This

Speaker 2: First part of the demo is um running on my local machine. It is just to give you an idea of what goes on when you've logged in and how what information is needed to pull the information into the Redux state that's specific to this user. So um this stuff they got so but okay slow down rules of engagement is a typical it's a multiplayer game it has worlds Each world comprises three players with three different roles. And then it has a leader, and the leader, as you can see here, can be looking at multiple runs It's called classes, but there's a default run and a DjangoCon

Speaker 2: run, and they're in different phases. The debrief the default run is still in setup And here is where someone can go in and change the start date and open date. There's a whole bunch of parameters that I don't think we're ever going to change, but it's nice to know that they could be changed. Uh and that allows me to mention an important point. We have this flexible kind of generic model, but each game is going to have data that's specific to that game And the way it's stored on the model is that the model classes all have a data property which is stored as JSON.

Speaker 2: So when I create a decision The role of the decision, the name of the decision, that's all part of those are all model properties. But when I want to say like what the heck the decision, you know, with the values of the decision, that's all stored as JSON. So just when you were looking at the those game settings, that's all JSON data that gets stored on the run. So when you log in, some of the queries uh are that for a leader it looks for what runs the leaders associated with and it creates topics They're at the run level. So that's model. run. run. model. run dot two

Speaker 2: But for a player, again, because of the players associated with the world, we want to know stuff about that particular run user, such as uh what are their private scenarios. And we also want to know about what worlds they belong to, which is one. So the topics for a player are going to be, if I can get that queued up. Their topics are going to be okay, I want to know anything that happens about the run user that represents this particular logged in user, and I want to know anything that happens about what goes on with the world that this run user's been assigned to. Now, the thing that got me so excited about simple early on, and I really didn't understand it for ages.

Speaker 2: is the auto magic property. My first blog about Simple was talking about its auto magic properties. So it's now time to open play, in fact It is a bit late to open play. It was supposed to open at 11. 30, but we had the silly talk we had to get through first. So now I'm going to move this run to play. And boom, suddenly this player gets to play. There's no polling involved. There's no pushing involved. It's fantastic. I do get to create a new test. If I was imaginative, I'd call it something besides tests, but my adrenaline is going, so I'm not going to. I'll put just put 2001 eight to show that I can do something that isn't pre-um already in my presets

Speaker 2: So to back off a little bit and say something about this game, when a player comes in here, they're really simulating the Canadian children's uh Serial company case study. Each player, I'll go back to the dashboard. I got too excited about this. So my role is I'm playing the product manager for Kellogg. And there are two other product managers. The competition is Post and Kellogg's. So my goal is to create a budget. That's gonna like knock the socks off the competition and their goal is to create budgets, they're gonna knock the socks off the competition. And they all have a different case history, they all have um basically more or or less uh

Speaker 2: resources. And the re the reason this game is called Rules of Engagement rather than Canadian cereal company, children's cereal company, is because you define the budget for the current period But you also create rules that are gonna set future budget. So this game is a little weird for us because it runs over many periods, but there's only one set of decisions, and those decisions Affect the budget that is for the future periods, but you're not creating new decisions with each period. So I'm not going to mess with anything, but there's all sorts of options.

Speaker 2: You can keep it fixed, which is the default. You can react to the highest, the lowest, uh by percentages above or below whatever they did the last period. Um and I'm going to just save that and run but it doesn't want to save because I don't have a name. Um so So I'm gonna save this and run it. And I'm not gonna show you how cool it is and on the model service it's saying, Oh, I ran this, but Anyway, I can step it through multiple periods and because everything was fixed, it's just everything's flatlined. But um I can also say, okay, that

Speaker 2: so I can create as a Rules of engagement player, I can create multiple test scenarios, I can do what-ifs. This is something the professors have been asking about for years and we weren't able to give it to them with the previous uh simulation platform that we had. And then when I'm done and I've got my best, I figured out what my best deal is given my role I can then say, okay, I'm going to submit this as my final decision. And basically, I've done everything I have to do in the game as a player. So at some point The time will be up and it's going to be time for the professor to debrief the pay uh the students on on the results of their decisions

Speaker 2: So we just moved it to debrief. If you see now the user can no longer make corrections on their final decisions because the play period is over. And so if we go in near here now, then we see the it's only two worlds in this little test run, but you we can say I want to increment it. Like five periods and step it forward just like we did with the test scenario. And there's a lot of other detail which I'm not going to dig into But what I'm going to do now is switch over and trusting that the talk gods are with us. Um I just want to do a quick refresh.

Speaker 2: Well no, it's good. This actually is our production server. And what we're seeing here on the top four rows are the runs from last year. And what we're seeing on the bottom four rows are the empty runs that have been set up for the coming For this semester, which the will start running in um ne the end of next week. So oh never mind Never mind. Um so I'm gonna go in here and say that when we first start telling the professor that the old version, because this is a re-implementation and it's an improvement on a previous version of the simulation. When we told him that basically the sands of time were running out on the old version and we were going to give him a new version, he said, well

Speaker 2: I gotta know that I can have 300 students log on at the same time and the system isn't gonna choke. And I gotta know that if I take one of these runs, which has 78 students, which means 26 worlds in it And step at five periods, it'll finish in less than ten seconds. So we were nowhere near there a year ago. And um I don't know if we're going to be here this minute, but we are here more often than not this year. So here we go. This is like 26 worlds, all being step five periods. And I'm going to show the JavaScript because that is uh let me see how long it took

Speaker 2: Okay, we ran it in 6. 76 seconds and it took a a couple of seconds to refresh. And one of the things you gotta know is that we do a lot of stuff automagically, but In order to get this response time, not necessarily on the server for running the model, but to get the refresh, we're not waiting for This the cycle do you go to the database and the webhook fires and then it comes back and it filters back to the Redux store and then the page would you that was like that's why there are all these thrashing um notices because there was a lot of thrashing and and the refreshes would fail. So what we do is we run it, we scoop up a JSON, we serialize the

Speaker 2: data and we send it back to the front end and that's how we're doing the refresh. So now I'm going to switch back to Joe who's going to take us home. Where are you? There you are.

Speaker 1: That moment when you uh Hit refresh on a live demo and pray that it works that time. So that was awesome. Awesome, Jane. Thank you. Uh so where are we now? So we're beginning work on our next simple simulation. It's already in development. We've written a a model for it. It's another marketing simulation that we're going to be running with the idea of delivering it into the classroom for over 200 executive MBA students. next fall. We're really excited about that. The repos are available. They're open source, github. com slash simpleworld. We have documentation, we have tutorials, uh we have a beautiful website, uh HTTP right there, simple. world Uh shout out to Chad Whitman. He's not here today. He works at Warren Computing. He did an amazing job on the website.

Speaker 1: It looks really like totally legit. And uh it's always really, really cool. And we have t-shirts, so that's really awesome too. Next up for us, well, we want to uh do this stuff. This is the largest talk that we've done so far. Uh we want to take this thing on the road, go to other places. Maybe to PyCon, we'll see. Uh, but we also want to do workshops locally in the Philadelphia area at Penn. Uh we w for there's folks in the university uh at other schools that might want to use this. Uh so we're gonna have some workshops there. free workshops in the in the region and maybe we'll have SimpleCon someday. We'll see. But uh we're really excited about the prospects. Uh if you want to learn more about this, I think we're gonna have a really few minutes for questions but you can reach out to us

Speaker 1: learninglab at Wharton. upen. edu is the web is the um email address for our team. The homepage is there, the GitHub link is there. If you go on the webpage, at the very bottom, there's a place to sign up for our mailing list. So we're gonna be as we uh build content out, as we make changes, as we have workshops, we'll be broadcasting that information via the mailing list. So I hope you got something out of this. It's a really exciting platform for us and a really big milestone for the Learning Lab. We really think this could be uh you know a pretty uh impactful project uh at higher ed but also we're really excited to see folks all the diverse folks here at DjangoCon and beyond who can take it and maybe think about using it in ways that that we haven't

Speaker 1: uh even thought of yet. So that's a really exciting thing about uh having an open source project. So uh thank you for your time. Uh I think we have a few questions. We have time for a few questions, but yeah thank you. Um

Speaker 3: in regards to your live demo today, so I noticed you had you played for one player. Um And the other players had results as well. So how did that work? And how do you deal with students who might not do their homework? Is that part of that?

Speaker 1: That's uh that's a great question. Um So when the students submit decisions, when they do some test scenarios, they're entering decisions for all three roles, and they're kind of simulating what their opponents or what their other competitors might do. So when they s when we uh provision the game and load it up, we assign folks to every uh role in a world. We create the worlds. And um when we when the professor moves the game to debrief and is cranking forward those worlds, all the decisions for that world are being run at that point at that time. Uh if someone doesn't do it, the assignment, there is just default behavior that's in there to prevent the model service from blowing up. Yeah.

Speaker 4: I was curious where you guys plan to take this next, if you're gonna keep it in the finance world or if there's other areas that you're hoping to break into.

Speaker 1: Yeah, I think You know, so right now, uh we've given this talk at a few places at Penn. We've given this talk at a couple universities around um the Philadelphia area. You know, business is a natural fit because what I mentioned about simulations, but I think that there could be simulations built around a lot of other disciplines, non-business related, non-finance related. I could see this being really effective tool in the engineering world where simulations could be effective. I could see this you know even liberal arts. I could see this at a lot of other places. I could also see it as well. We've talked to a few folks after our talks in the research world. So maybe using this in a research capacity in some way, I could see that happening as well So really we'll see what kind of traction it gets outside of out of outside of business.

Speaker 1: But I'm I'm pretty optimistic that we'll have some folks that are using this in other disciplines that uh we haven't seen before. All right, so thank you. We'll be around uh the rest of today. And again, if you have any other questions, feel free to email us. We'll share the slides out. Well this talk will be online. So hopefully uh you enjoyed it. Thank you