3D Files with Wagtail

Speaker 1: Hello again. We're going to begin in in uh just a minute, but before we begin, I'd like to remind everyone in the room and online that it's not too late to join the Lightning Talks. And you can find that and sign up for that on the Wagtail Slack site. And just in about a minute, I'm going to introduce our next speaker. Our next speaker is going to be, our next speakers are going to be talking about 3D files with Wagtail. Dawn Wages is the Python community advocate for Microsoft. She's also a founder of Command Lines, LTD

Speaker 1: Ltd. And Myra Gibson is the founder of Art Tech. And again, they're going to be talking about 3D files with Wright Down. Welcome.

Speaker 2: Thrilled to be here. Uh we're just chatting with Tim earlier. Um, my name's Dawn. I am the Don on that screen. Um and and I gave my first talk ever at on this campus. over in uh uh in the business school. And now I'm giving my first talk ever with like my wife. I'm so excited. Um we were recently married. We are just under a year um and it's been really exciting. Um we didn't have a wedding, so this is our wedding. Thank you, Lab Tail, for covering it. So what we're gonna be talking about today are 3D file

Speaker 2: types. This is something that I've not actually even had an interest in getting into, but really got nerd-sniped by the awesome way that Django handles file types and uh image files um in Django. So I so there's kind of like a chunk where we start out with 3D images and get everyone kind of acclimated Then we'll go into the Django and Wagtail of it all. I'll show you a proof of concept that we got moving and working, so that was fun. And then I'm gonna ask you to rip apart my code at the sprints And maybe we can make something really cool because I think this idea has legs.

Speaker 3: Oh, yeah. Hi, I'm the Mira in this. Nice to be here with you all. This is gonna be my first uh Wagtail talk and second talk overall, so I'm happy to be here with you guys and happy to be here with my mom Oh, that does something. She put this together.

Speaker 2: It surprised you some thanks. Uh just hit just hit next. It'll go next.

Speaker 3: So um well we're going to talk about what 3D images are, um the file types, the immersive web, uh and we're gonna get into WebXR versus WebGL as well as uh 3JSNA frame. And then Dawn is going to talk about the Wagtail in Django N, how it handles files, and show you guys a proof of concept, which I think is really cool. Donna Mira. I'm an XR Debian designer. I work with Unity and C sharp and uh 3JS as well as Unreal Engine and things like that. um founder of ARD Tech, which is a ARD in the Philly area

Speaker 3: when you say okay sometimes like as in alright uh sometimes people just say ARD and that's where the name comes from And

Speaker 2: did you see the acronym of it?

Speaker 3: Oh, um the acronym is uh augmented reality design and development.

Speaker 2: Very clever. I'm just proud of you for that one. My name's Don. We talked about what I do before already. Um I do web and data, and that's my happy place. Um and I've been doing Python, Wagtail, and Django since about 2016 I have to do the obligatory shout out to Django Girls as I always do. Sh they uh show up everywhere and you don't always know it. But Django Girls is what started my career in Django in 2016 when uh when DjangoCon US was in Philadelphia. So conferences matter, showing up matters, these events matter, and I really appreciate you all being part of my community. I'm the co-founder of Command Lines and a few people here may know some of my co-founders, Mariada and Chook, who are also in the Python community as well.

Speaker 3: So XR, what is it? Um it is an umbrella term that covers uh few different types of experience. So there's AR, VR, MR, and uh actually it covers those three. AR, VR and MR, which are extended real which are augmented reality, virtual reality, and uh mixed reality. So augmented reality is anything that's kind of just an overlay on the natural world You can interact with it, but it is not as immersive. Virtual reality is a completely immersive computer generated world in which you can interact with digital items, and mixed reality is a combination of the two. It can be an overlay of an entire

Speaker 3: of your entire physical world, and you can interact with items both in the digital space as well as the physical space in new ways. So, Web XR and WebGL, what are they? What are the differences and How does it relate to XR? So I'm actually going to start with I know uh WebXR is up at the top. I'm going to start with WebGL. WebGL is a graphics library API that Helps you put um pixels on the screen. It can be used for 2D, but is used a lot for 3D images and rendering. WebXR is an API that allows you access to devices um access to specific features on devices in which you would use within extended reality.

Speaker 3: So like when you have uh like let's say Pokemon Go The Web XR API allows you to use the camera to basically throw Pokeballs and make sure that everything kind of lines up in the digital world and the physical world together. And I know it says to produce web uh immersive web experiences, WebXR is the best choice. However, WebGL and WebXR work in conjunction with each other to create a truly immersive web and a truly immersive experience. Um you definitely need both, although uh WebGL is a lot to uh to uh code by itself. So I tend to use things like 3GS or Babylon or uh

Speaker 3: Pixie, JS, these are all 3D frameworks in JavaScript that allow you to create 3D images as well as rendering them But there are other things such as Unity and Unreal Engine that allow you to do these things without a need for much code. And the different file types that we actually use within the 3D and XR space are FBX, STL, which is more used for things like uh uh what is it digital like printing digital printing 3d printing um gltf and glb files Which are very similar to each other, except one hold uh one holds binary information that includes the uh animation and the material and textures of the 3D image that you're rendering.

Speaker 3: and object files which just contain uh the object itself. So it can be a 3D object, but there is no color, no animation, nothing. It's just how it looks and it's probably one of the easiest ways to just get a 3D file to render. Currently GLTF and GLB are the standards because they're a lot smaller and easy to put on the web. So I know we talked a lot about uh JavaScript frameworks, but you guys all work in Python. So one of the uh easiest ways to use that same uh the same functionality that you get with 3ds which automatically uh uses the WebGL

Speaker 3: API is to use A-Frame which is a HTML-based uh framework that allow that is an extraction of 3. js that allows you to use certain features and components as well as it allows you to access the WebXR API at the same time. so that you can use uh both 3D images and access those functionality without having to use two separate technologies. And uh A-Frame is actually uh pretty easy uh to use and get started to write from the box. Um You can use a CDN link and start going right away adding in a scene to any vanilla website with just straight HTML and CSS, no job script needed.

Speaker 3: So, Y3D assets, why are we talking to you about XR and that? Interactivity. It creates a more interactive web. allows you to see and uh think about things in a different way. Uh accessibility um now with the web being stronger uh It's more easily accessible to get 3D images on the web and to view them. It's flexible. GLTF and GLB files are Like I said, the easiest way to get 3D on the web and it's really useful for doing just about anything. You can also make them into 2D images, uh render them as 2D images Um and then Wonder and Whimsy, um which leads into the next point, which is engagement

Speaker 3: and It gives you a sense of wonder and it makes things uh a lot more easier to uh understand if you don't understand it um how Like say uh kids in school um studying science, you see a mitochondria cell and you're more engaged in it and it makes you actually uh more curious about the world around you. And I think that's a really important as we uh grow and get into a very uh digit based uh society And then off to my wife.

Speaker 2: I'd also like to underscore the intentional misspelling in Whimsy, which is very whimsical of us. Okay, so there's a quote there. Okay. So now we're going to get into how Django and Wagtail handle some of these image files. You're going to see code on the next several screens. The intention of this is to really give a big hand clap to the framework that many of us have chosen with our feet and our time here. Um Django is so opinionated in what I feel like is one of the in of the right ways And so I'm going to be going into a lot of the inheritance that happens with image and file fields and like giving a kind of a praise clap to all of the work that I don't have to do by trying to

Speaker 2: initiate this project instead of rolling my own. So we start out with the image file field. We are inheriting this is the Wagtail image field we are inheriting from the models. image field from Django. allows us to have um access to Willow , which is a the default um uh image library that is maintained by Torchbox and Friends And it combines pillow, wand, and open CV. I don't have access or the the approaching this. Um I didn't have understanding of what Wand and OpenC V does That's the point of having some of these abstractions because it makes it easier to use. But Pillow is a fork of the PIL library.

Speaker 2: That one kind of rang a little bit of a bell to me Um so this is giving the framework of uh uh a level at which I'm coming into these problems too. Um so it kind of is intermediate problems, but this could be kind of like a nerd snipe from all um all levels because it'll be really fun to kind of approach um from all skill sets. So the PLL PIL fork from the Python imaging library. And then Wand is an API bindings for Python. And OpenCB is the is a computer vision library. So then from that we're digging into what is that Django image field. We're starting out with some code where we're defining some the attribute class. a descriptor class um and then the description with that uh lazy uh underscore uh you may recognize that for translations

Speaker 2: And then we're initializing this class. We you'll see the self-width field and self-height field Um I one of the questions that I'd like to solve over or have answered over over sprints is why is the width in the image, uh width in the height so tedious in these problems, right? But you're adding additional attributes, characteristics around uh the file, and then there's going to be lots of different levels at which you're updating and rendering and then sharing this file, um, that it ends up being a headache and a lot of these problems have been additionally solved in ways that I would have assumed would not have been so difficult. So here we also have some checks. We are also using the parent class

Speaker 2: check that is at super dot check with the keyword arguments as well from the file field. And then we're also checking if that image library is installed, which is fun because we get really great error handling in Django. So it'll respond with saying Hey, you don't have, for example, you don't have Pillow installed. Reminder, we are looking at the Django code, not the Wagtail code. In the Wagtail code, it'll say Willow , and it'll show that there's like an import error for us. We also have the deconstruct method now in this in the image field. That'll help us with the width field and the height field. and assign it with uh with our arguments and then we're also um accessing the file field from that parent class as well

Speaker 2: We have the contribute to class and it's this we're getting more and more in the weeds. You see the number of question marks that I had on the why, so confused. why is are are some of these needing to be elevated and having these these constructor classes going down and I'm gonna be I think a a really cool iteration two of this is to describe some of the decisions that I made when extending into the GLTF files. and then having some comparisons on how I wish it would have rather been done or some maybe adding some comments in Django code and that makes it easier. But Django is really good at being documented. So the contribute class helped with addressing a bug that has been had that had. And then we also get the bug number.

Speaker 2: So I can just Go and look it up and for interest of time I did not include it as well. Um but we also have so the the TLDR on this one is that when you have the model in it, it uh for post uh it will only run up on post-initialization dimensions to update non-abstract models. So just that's a hairy thing. I'm so glad I didn't have to deal with, thank God. Really is the summary of this slide. We also have a similar experience trying to dig deeper into why so confused on the update dimension fields. But there's documentation in the source code that highlights that as well. Thank you, Django. So in this one, if the TLDR is it's hooked up to the model's post-submit

Speaker 2: signal and it will update dimensions after instantiating the model instance. And again, something that seems so simple as the height and the width of an image file um really gets tedious when you're starting to work in complex systems and I don't have to handle that and I just I'm so glad that I picked Wagtail and Django. We have some more documentation that we'll talk more about with the update dimensions fields. I won't go into this too much. Don't want to zoom through it, but for accessibility reasons, just r review it at your at your leisure, but it's really kind of more of like Yay Django Uh both the dimensions fields have values that are most likely loading from data um from the database. And so these will uh solve some of these things with the image descriptor dot

Speaker 2: set. Also there are comments about when you're going to have none for the image and otherwise it should be an image field file. And then we also now near like the end of the class we're getting to actually assigning the file And then you're actually assigning the width now, and you're actually assigning the height. And if not, they're going to clear those clear those values. We also have the form field method that's going to be accessed on this same class. And there is a whole chain of

Speaker 2: information that you need to for Django that handles forms and handles it appropriately. It'll have some cleaning and some validators um for your extension types um and it'll pull from the PIL library to get some of those as well. And then go to another step deeper because it's just inheritance, inheritance, inheritance. It's really great. But here are a lot of all of the classes that are accessed in the file field. um additional uh what was inherited from the image image field. Um and I'm i again it is highlighting and underscoring the point that I didn't have to do all of these things myself and as I am deviating from norm I

Speaker 2: go through and to find the definition in my VS Code instance to figure out what I am missing when I am overriding some of these methods And this is more of the file field of what the file field does. So here the the call out is really uh that the checks method and the storage checks and the deconstruct and the pre-save are really cool. I wouldn't have known to uh do some of these things. For example, we have self. storage equals storage or default storage. um then we're making it a if it is callable, right? So we're gonna have different types that are sometimes callable, sometimes not callable. You are assigning it appropriately

Speaker 2: and then doing a kind of what feels maybe redundant, but probably solve some additional bugs that would have otherwise been raised, where you're assigning self. storage equals self dot storage with that deconstructor. Would that deconstruct? And then we also have checks. So more checks where there is a check the primary key and check where it's uploaded to. In those deconstruct and the pre-saves, I mentioned I want to call attention to that storage equals get attribute self storage callable. self-storage, you're redefining you're defining what that storage uh variable means and then you're adding it to your keyword arguments and you're adding it to that storage object. So just different paradigms

Speaker 2: that uh patterns that they're using in order to access um uh your image in your storage um that just You know, it's an opinion that Django has and I don't have to have one because it does. Again, it's also with this super pre-save. We have some storage references there as well. So now we're in the instance where I'm trying to support these 3D files. I've gone through all of this code. I'm I'm feeling jazz. I'm I'm really excited because the the code is well documented. You're seeing this like daisy chain of inheritance

Speaker 2: that's very Pythonic and you know when you're looking at good Python code it feels good. Um so I'm doing some of these things where I'm abstracting and the three steps that I really went to to be able to support these 3D files Starts with extending the functionality from Django and Wagtail and then incorporating the attribute changes from the XR file fields. If I could hearken back to the slide that my wife did Uh she 's not just my wife, Nira Gibson, person in her own right. Apologies. Um I wanna um highlight on the right side of the screen we have the scene, the camera, the mesh. um the group uh several levels of geometry material and texture in future case

Speaker 2: I want all of that to be similar to the functionality that my width and my height did. So we're going to be comparing some of those patterns in the class for our image file. Let's go back to where we okay. And then next, we already had a uh package that does the work for us. You could go to that uh repo that I have in the top right-hand corner. Um, it's tough code, friend. It's not that, but it's tough code. It's really just a proof of concept. So we'll be here at the sprints to try to vet some of this a little bit more. Um and see where where this can go. But once you have it into the binary um into the binary uh file uh file type it's saved to the database

Speaker 2: no problem it just it just worked and that was really exciting And then last, you're going to want to render it uh to the front end with A-frame.

Speaker 3: So A -Frame is, like I said before, is a is a framework that is HTML-based. You use tags in the same way. It's an abstraction of through JS, which gives you all the same functionality. And like Don said, it is a rough proof of concept because it is a Uh this was a task we got together on a few uh occasions and paired program.

Speaker 2: But the apron part was seemed very um uh very straightforward once we figured out the the version number of the documentation. And so I feel like that part is not going to be um uh changed very much. What's really gonna be interesting is because Django leverages just very understandable HTML templates for the front end and I love to do that unless I have to. I'm an HT HTMX girly when I when I need to be Um I think that'll be really interesting to like to leverage that and it'll it'll programmatically just be able to make some decisions for us. No, I think that's a good slide because I was leaning into yeah totally. So our end goals on this would be to extend the functionality into a package

Speaker 2: But what is that package going to look like? We could either go the right uh the route of um extending the Pi GLTF library. Make it more like pillow and willow. Maybe upstream it into Willow. I have not talked to the Willow team before getting on here and saying that So, you know, that was a little risky move. But I'll I'll I'll chat with them to see if that seems like a good option. But one of the cool things that Willow uh Willow and Pillow does is it creates um options for you to like kind of um inst uh instantiate the the object. and then access it and then see your file type and then play around with it. You can do that in the console. But what uh is PyGLTF is really like a lot of methods to change from one file format to another. And so it would be great to have a more robust

Speaker 2: object-oriented type paradigm to like play around with it that already exists. And if it could be supported by the main community, if that's something that seems interesting, then we can then we can upstream it in a friendly way. Um, or we could go a different route and do something similar to like a puppet, uh puppet, the bird, and it is um uh well named after the bird is the library that allows you on a Wagtail site to create a blog engine and I really like that when I'm using it for my personal blog But one of the cool things is it just kind of handles the end-to-end process for you. If you want manage. py, it's going to create new databases for you. It's going to have default references to templates you should be using, things like that. So for example, if we go with what I think is the most viable and maybe commercially attractive use case for these things

Speaker 2: which be like maybe an e-commerce site. You could have a list det uh a list det and a detail page um and certain things like that that handle those normal patterns Very similar to Wagtail's content management system, we can package that up all into like a putpit-like library instead. This could also end up being two different libraries if it's you know we'd have to like multiply uh the amount of time that we have to be able to maintain it. But this is a really fun one that got us both really excited about the end use cases And if you're seeing on more and more in e-commerce sites, people are browsing through uh with 3D images. And so hypothetically, if you just get a list page where your uh your image just renders excuse me with your image you just have to upload it with

Speaker 2: um your image just renders and you have um controls on your dash and then you also have controls in your wagtail um that allow you to edit or uh do it on the fly. I was talking to a friend just now about uh Blender's Python API. So there are the implications are really, really cool. And so that's a lot on the goal. The goals are big, but the three things that I categorize on how we can improve I think are leveraging the extra attributions. So that slide that I have where it has the seed and the textures and the models and things like that, I think are really great to incorporate We also have leveraging Wagtail strength, so those would be like having it all accessible through the admin and maybe making some cool conversations with uh cool calls to Blender

Speaker 2: maybe um that would be a pipe dream. Not a pipe dream. That'd be a big dream of mine. And then last would be um having end-to-end testing support. So I love having playwright. But then if there was also kind of some use cases that we were chatting about, um like putting in some gobbledygook just so we can run up some memory, it doesn't really matter what the images generating, but then test it on um different um uh different devices and different platforms um just from while you're while you're uh hacking on your code. And we will be at Sprints. I don't know what our time looks like, but thank you. Thank you.

Speaker 2: Well, I'm sure we have time for questions because I think we zoomed. Yeah, we zoomed through that. Any questions from the audience?

Speaker 3: Yep.

Speaker 4: So when I was much younger in high school, I made flash games that podcast the program and constantly flashes there and my experience of being a game designer with that. I'm fascinated with this topic. I've always loved this stuff as a front end developer, first and foremost, getting H2O and CSS. But my impression is that it's pretty hard to learn. Um, what is your take on it? Is it something you know photo 3D stuff self-teached these days or are there courses that you can take or is it like you gotta go to school?

Speaker 3: So I definitely don't think you have to go to school board. Um It is uh partially self-teaching, uh self-taught things, and a mixture of doing like uh online courses. Um there are tons of free online courses that take you through the basics of not just a 3D design but also development. Um particularly there are ones in A-Frame as well as like 3DJS. Um I currently use uh because I use more view I use an abstraction of 3JS called Thrace , which is a lot easier to learn because 3JS uh similar to WebGL it gets a little more complicated and you have to program out every single uh aspect of your object, whereas in uh Thrace

Speaker 3: you kind of have pre-made components that make it a little bit easier to do. But just where to start your game dreams did not die with Flash, trust me. It did not. Um and particularly because you deal uh in front end I'd say take a look at Play Canvas and their API. They do a lot with like 3D games and XR. And I think that's a good place to start besides A-Frame, taking a look at the A-Frame docs. I know someone built a clone of beat saber in A-frame, which is really cool, and it's it's pretty easy to get the hang of.

Speaker 2: And it Kind of an extension on that. That that's more of like focusing on creating the 3D assets themselves.

Speaker 3: Yeah.

Speaker 2: Okay. And then integrating it into like platforms that you'd actually use. Are you using the same tools? That's where the Unity right comes in, right?

Speaker 3: Yeah, um so different set of tools. It can be the same. Um like play canvas, you can kind of use it to create and uh program uh as well. Um things like uh bezel uh is a way to program and prototype, but if you're gonna actually if you're actually gonna be like in the weeds and creating games and things or an experience I would suggest Unity or uh Godot as well as Unreal Engine. And because there are developers in this room or mostly developers, um, I'm actually moving away from Unity and going into Unreal Engine. Um A lot of people say there is a really large learning curve and I actually don't agree with that. If you know a bit of programming, if you uh have worked with like Blender or uh node-based programs

Speaker 3: It should be relatively easy, especially if you have a programmatic mindset. So I definitely think those are the ways to go.

Speaker 2: Cool. I asked the second question, but we can do the third.

Speaker 5: I was just thinking what made you choose so I do continue someone can say about it if we want to solving with Django and white other image field and that but obviously a 3D image is a very different kind of very different kind of fish from what I understand Um so I'm just kind of interested in how you made that decision rather than to just go, oh some class file field and bring over the bits I need from image to go, ah, I'm gonna make this a type of image. What what how do you kind of make that

Speaker 2: Yeah, so I don't know if I made the right choice. It was a choice that I made so far and I haven't regretted it yet. So we'll start there. Um and ultimately the the file types can be rendered in 2D and there are ways to kind of maneuver around it. And where I saw so I have a slide that's kind of appendix now and I can probably show it. It's not done. Actually can't do that. Um on but it is all in my code on the decisions that I've made. So one or two fields I am abstracting, pulling from file uh file field form, and then the other I am doing image a little but if I'm looking at the code and I'm account accounting for gaps which I did because it got through um

Speaker 2: because it it rendered um it was able there were not too many differences in what I would need to be able to push it forward. So it really was just kind of this like hardheaded approach of like let's pick the easiest solution, the most done solution, and see where the errors happen. And I was actually talking with a friend um earlier today um that Django is unique in the fact that um we get really good error responses in our in our console so like when something went left I would I it would point me exactly to where I needed to fix or address the change and what broke Um so that was my first attempt in the proof of concept. And they live to regret that

Speaker 5: though. Yeah, I was curious to know especially without seeing the demo project like what if the manager using Wagtail is like being able to kind of collect these different 3D objects like thinking them on the back end as well um or if there's like particular use cases that y'all had in mind for that like the thing I thought of is like you know the like a museum or a library that's like digitized or archived, it probably is a management problem on the back end, but also you can just be like a cool way to be able to like play around with stuff.

Speaker 2: A hundred percent. So I I think we're one step away from being able to have so I'm not really stabby with the Wagtail admin um because I know it's it's like it had some React and then it compiles and does some things and I don't know exactly all of the things that that's available, but I would like to have a a list page similar to what the image preview page looks like in Wagtail, just at the top level, and then have them easily so for example you just click into it and you can adjust the focal point, for example. I would like something very similar. You can adjust um attributes that really just correlate to updating um uh HTML elements on the A-frame, right? So you just put in the admin and the A-frame HTML updates from a white background to a black background

Speaker 2: or what the API allows like a snowy scene. I think there's one and there's a forest. So you just type in forest And you're now automatically harnessing the power of of A-frame on the front end, um, which is declarative. Um Did I answer your question? And no, we haven't gotten to that point yet. I think was maybe your question.

Speaker 5: Well just yeah, there 's other use cases that like kind of inspired like, oh it should be cool who made this and then we could do the other stuff there.

Speaker 3: Okay. Um there are there's a use case uh that had to do more with um uh mobile delivery uh or mobile uh viewing for uh certain products and uh that was one that we're Like it's an idea that I'm rummaging around in my head and uh this would be a good way to actually do that because it's a little hard to um Depending on how you use it, it's a little hard to uh do that with A-Frame by itself.

Speaker 2: Yeah.

Speaker 3: And um

Speaker 2: you need a CMS part behind it to be able to kind of make it work So like white labeling a 3D uh 3D CMS would be really cool.

Speaker 3: Yeah.

Speaker 2: Especially for our specific use case, which we probably won't chat about. But um yeah, totally Um there was a uh hand over on this side. Hi friend.

Speaker 5: So it's gonna be very cool I I asked I I I guess that because uh when Here is correct takes on and we having some introspection and extract Fewer parts of the scene, then I can command them uh

Speaker 5: We can construct a complex scene with parts of the root

Speaker 2: Yeah. So uh to repeat repeat the question because I did I haven't been doing that for the previous questions, apologies, but um why choose uh or Why not use a JSON field instead of an image field for GLTF because there's additional information and it is under the hood a JSON format file field Um that makes sense. I think I'm new to the file format and I think that's something worth exploring. One of the things that I do like though about uh Wagtail in particular is that they do uh lean into um very thoughtful opinions so we're giving you the best path forward but if you do want to make your own workaround uh we give you kind of a uh preview uh version that might be

Speaker 2: you know, merged into main later or or or actually excuse me documented more thoroughly later. And I can totally see a world where we allow um easily the the JSON file format. But I wanted the same path forward for GLB GLTF to start and then also to support objects which I think it might be easier and then the some of the more advanced file types and I want it to be It's named GLTF Plus because I want to be able to use the same image file format for all images and to just make uh use a switch case, for example, probably not literally, but uh a switch case for um what file type you end up doing and then going that down that right.

Speaker 2: So I want one tool to be able to handle it all. And that's why I went down with image. Uh

Speaker 3: any more questions? Sweet. Well thank you.

Speaker 2: Thank you for the time.