Here Come The Robots - Django and Machine Learning

Hi everybody. So uh this is me. I'm uh I'm also from Torchbox. It's a kind of one-two thing with Neil and Meetant this morning. I'm a technical director at TorchBox. We're a UK agency. We've been using Django since 2007, I think, and we're perhaps best known for Wagtail, the CMS. That we made uh we uh five years ago. Um I want to uh apologize for my um clickbaity title. Uh this talk isn't really about robots. Um or uh or robots taking over the world. But um uh and in in fact it's not really even about artificial intelligence. Artificial intelligence is just uh is is a kind of despite the way it's mixed up in the media

is uh a kind of superset of of many technologies including machine learning so uh Machine learning is one, but also natural language processing, translation, robotics, and um It's it's it's a big area, and uh but I do want to s to to spend just five minutes talking about some of the big questions around artificial intelligence. Have any of you read this book? I really really like this book by Max Tegmark, he's a physics professor at MIT, and he talks about these three three phases, like three releases of of life. The first one is version 1. 0. This came out about four billion years ago, I think, and uh

this is like uh bacteria or even chickens. And um Two of the characteristics, uh kind of the main characteristics of uh of release one point oh of life are that uh they are unable to um to change their own hardware, their bodies. all their software, apart from by evolving, which which takes many generations. And then version two Version 2 is uh what what uh Max Tegmark calls the the cultural age. And uh with version two um Th life still has the life forms still have this limitation of um not being able to manage its own hardware. Uh but version two is humans and uh humans have been able to

to manage to upgrade their own software. So uh in this room we probably speak you know fifteen languages and play 20 different musical instruments and have you know deep knowledge of of of of many many various skills. And these are things that we've in many cases decided to do. So within our own lifetimes we can decide to change the way that we operate and pick up new skills and then affect our lives using those skills So that was a big change, version 2. 0. And actually, there's a kind of side point that uh there's also a quick 2. 1 release, which um which kind of maybe came out in the last uh 50 years or so. where humans are starting to kind of upgrade their bodies a little bit. So um you know

false teeth or uh new knees, that sort of thing. But still it's minor. Generally our hardware only happens, you know, the changes are happening pretty gradually. And of course the you know the subject of the book is uh version 3. 0. Uh and with version 3. 0 the um the this this this life form will also be able to to manage its own software but uh also its own hardware. And once that happens, then you can get this really rapid, uh really rapid development. Unfortunately, Life 3. 0 is not human. Life 3. 0 is this is the robots who are uh who uh will be able to upgrade themselves. And um and that's the kind of you know the premise of all the kind of science fiction films that we've seen about this stuff. Uh I find it really interesting that uh

we seem to be right on the very cusp of this this massive change between version two point oh and version three point oh. So the first one, f like I said, four billion years ago roughly, and then two point oh came out around uh 100,000 years ago. And then this new phase, and and I should say this is speculation, right? So there's a there's a there's a lot of um AI experts who have very different opinions about this. But um someone did a study about the the the mean point at which the like the the average point at which the experts think that uh we will reach this thing called artificial general intelligence where where um computers are uh are able to kind of have the sort of generalistic s uh general skills that we have and the mean point is in about 30 years

time. About 2050. And um it kind of feels extraordinary that we, you know, this generation is right possibly on the cusp of this this kind of like extraordinary change. I don't know if it's uh and also, you know, uh is it I don't know if it's a coincidence that uh around the same time is uh possibly the you know where we're starting to feel like the most immediate effects of cataclysmic climate change And um this again this could just be coincidence that these two things are happening at once or it could be just a kind of replay of this uh like apocalypse uh messiah kind of um fantasies that people have had for the last m for many millennia Anyway, I'm not here to talk about uh apocalypses. Uh I want to talk about machine learning and uh and uh so just a subset of AI and particularly within machine learning.

Uh I I like this this definition by uh Francois Cholet, um that um classical programming, so I guess the programming that uh most of us have have have been learning, have been working with uses rules and data to produce answers. So we know what the we know what the rules are and um and we'll we'll take some data and we'll make a decision based on that. Whereas machine learning uses data and answers and you give it to a computer and the computer creates the rules. I think it's a really nice simple defin definition that really kind of triggered, helped trigger my understanding of this. Instantly I re I recommend if you if you're interested in learning about machine learning from the kind of fundamentals, I really recommend Scholar's book. He's also the author of Keras, which is um perhaps the the

the best known. Python library for machine learning. And this is an excellent book about the fundamentals and the kind of kind of the basic maths. Incidentally Sholley is one of the people who are skeptical about uh the artificial uh uh general intelligence. He thinks that it's um he can't see uh the possibility of making the leap between these very specialized skills that computers have now and the general skills that that humans have. But on the other hand, he's very optimistic about the immediate effects. that um the uh uh uh the beneficial effects of machine learning. And that's what I'm going to talk about in this talk. And so I I want to talk about some really um practical and simple ways that we as Django developers can take advantage of this kind of extre

amazing explosion of uh of techniques. And the first one is um image recognition. So uh some of you may have been to the uh the workshop yesterday on uh computer vision and um and you might have learnt how to identify shapes in a in a picture and and to make um uh to decide whether a picture is is about uh you know matches a certain set of characteristics Uh but the idea that um you could just get, you know, uh a million megapixels and a million pixels and send them to a computer and th and the computer will be able to tell you what the contents are or of something of any angle, even seven or eight years ago would have seemed like a futuristic impossible possible task. But now it's something that we can do in uh about fifteen

lines of pretty cruddy Python. Um so I I've built a simple Django app that I'm going to show you and uh which which hooks into some of these services. And I guess the the kind of key point of my talk is that You don't need to read Cholet's book and you don't need to have a have like undergraduate level maths in order to do this stuff because other people have done it for you and you can hook into these serv these cheap services which are getting faster all the time. I'm not saying you shouldn't, I'm just saying you don't need to. So in this case, most of the work is around like reading the documentation and working out how to do authentication in different ways So we're just taking an image and posting it off in this case to Microsoft's service. And I'm going to attempt a live demo here. So this is at uh robots.

tomd. org. You can try this later. And um I'm gonna start by what this is an interesting image I found. What what do you think this is? You're all muttering, so I can't quite hear you, but you're probably right, whatever you said. So I'm gonna paste that in here. And ask the computer to describe it for me. And a person holding a bird, pretty good. Pretty accurate. Maybe, you know, maybe not such a difficult one. This is my colleague Colin. He doesn't actually uh he does have a beautiful red beard, but he doesn't have those ears. Um let's uh let's try this. This is gonna be a bit more of a challenge.

Dog in front of my room. Okay. I mean you can see what Uh I can sort of understand that. Like the the bathroom sign in the back looks it makes it look like it uh you know it's one of those sort of classic bathroom selfie shots. Um and uh you know it's it's done a pretty good guess with that. Okay, here's um here's a photo I took earlier. Django Pony, is it a close up of a logo? No. That's not not accurate. That's just a distant picture of a light sign. So I'm going to try something a bit uh bit more risky now. Okay, take a picture of me.

Click A man and a woman, so that's interesting. So I guess I'm not sure which one's the man and which one's the woman Um I only tried out on you down. This is the hot for me this was the hardest bit of the whole uh this whole site is working out how to get JavaScript to uh understand different camera types. Right. Everybody wave. Okay. Are you also a man in all pretty good? Group of people sitting in front of a cloud closing for the camera. All right.

So that was image recognition. There's some practical applications of this. Um one is uh content management systems, and that's the one I'm going to show you next. So actually This is a demo that I built of some code that someone in this room wrote who I haven't met. Martin, are you here? There he is. Martin's put his hand up. So uh this is again this is Wagtail and um I'm uploading a little set of images from my uh from my machine and but this moves quite quickly but what you can see here is that Wagtail has taken the title and now already knows the titles of these images. So that's my dog, a brack and brown dog, Nigel Farage wearing a bow tie. Uh this is some sardine heads in my sink. That says a white plate covered in snow, so that wasn't so good.

Um so clearly, you know, you can't rely on these completely. This is this is not terribly accurate, and that's because Uh again I use the Microsoft Vision Service for this. Microsoft probably hasn't seen, doesn't have many reference images of uh of sinks filled with sardine heads yet, but it you know it will get better. But I think this is a really use good way of using it because you don't need uh in this case we're not we're not c relying on the machine to make exactly the right uh guess, we're using it to to help us to sort of augment the editor experience. And I think a lot of the time this these machine learning tools can be used in that way, is it's to currently to to augment the ways that we're working already This is something that's happening more in the last uh I've seen more in the last couple of months. So as well as recognition, we're also seeing image generation systems.

And some of you may have seen this site that uh that was quite popular a few months ago, uh a couple of months ago about thisisnot a real person. com, I think. So here's an example of someone who is not a real person. And this is not this is not um someone like This is not a computer program piecing together different parts of a person. This is like building it up from the fundamentals. And each time you refresh it creates a new one. And it looks pretty convincing. You start to recognize the ways that it isn't quite right. So uh generally you focus on the face, but you can see around the edges there's something pretty crazy going on. Also apparently teeth are hard to to for computers to do at the moment. So you often get these kind of slightly weird central teeth. I mean normally I would feel rude about saying that, but this isn't this is not a real person. Um someone quickly uh afterwards made them this is not a real cat, which I'm happy to say has not been so successful.

Uh so you know the the computers uh we're we're safe from the robots for the for the moment. Right. The next one is about sentiment analysis. This is another useful tool that you can you can use to to add up to to augment your uh your Django apps. So sentiment analysis is a fancy way of saying what is the author feeling? And again, here's some like In this case I use the IBM's Watson service. And I'll try a demo of this. So I will take some uh feedback. And last night we went to this beautiful

uh Ethiopian restaurant. Thank you very much to the organizers for that. And uh so I looked at some I looked at some of the reviews and uh there are very few bad reviews because it's such a great place. But here's one that was the the uh author was furious that they only took Danish credit cards. So we can try seeing what Watson thinks about this. So Watson has uh understood that there's this, you know s quite a bit of sadness in this statement. There's a bit of joy which is a bit surprising to me. I suppose that the joy is the the city full of wonderful restaurant. And it's uh it's analytical. On the other hand, if we take one of the many positive reviews, interestingly there's a there's another one here

which is positive and just says Except danger because it goes, but there is an ATM just down the road, so you'd think the first person could have like just walked down the road. In this case Joy is is stronger. It's a bit tentative. I must say I haven't been that impressed with the uh with the Watson scores and uh I've just m last night did a a second one using the Google tool which is simpler. So Google doesn't try and work out um uh different tones it just says whether positive or negative and it looks at magnitude as well. So So magnitude is like the the strength of feeling. So you might have um a paragraph where you say, I really hated this, but I loved that, and then that you'd have quite strong uh uh high levels of magnitude but but the overall message would be

neutral and uh so the Google results um are able to express that. So what are some what are some practical applications of this? So customer support is an obvious one. So what uh if you get a lot of feedback um and uh you want to know you want to maybe be get a trigger when there's something that's like a particularly angry message so you can deal with personally. Um handling fake reviews could be good or better bots so uh you know there's an increasing focus on bots and One of the things that makes a realistic bot is if you can detect the tone of the person who's asking you questions. And that means that you're more likely to be able to give a kind of more human answer. We're using this at the moment for one of our clients, the Samaritans. They're um

in the UK a very well-known organization who help people in in crisis. Um people people who are you know thinking of hurting themselves. And uh we are then this all done over the phone at the moment, but we're building an online chat service for them. And one thing that we want to do is to be able to measure and and I you know this is e each message is not going to be very accurate but I think over time we we hopefully we'll see trends and see and and be able to and start getting some data that we hope will help them improve the service about about the measuring the the the the state of feeling, the the the the the sentiment during this journey. Next up is entity extraction. And uh this is really about identifying proper nouns in a text, which sounds like a simple thing to do, so you could write a you know a regular expression that looks for capital letters and tries to work out what your

proper nouns, but turns out to be harder than that and you know different languages have different ways of uh of of handling of of understanding what a proper noun is So let's take some text from this one. And this one I'm using again the Google Natural Language Service. And I'm going to take the first paragraph from uh the lovely website for this conference. And Google's going to tell me what it thinks it's about. So uh accurately understands that it's about DjangoCon Europe. That was g that was good. And that it's, you know, it's very I'm not sure why the two communities, that's probably something in my code. There's a mistake, obviously, the uh the Django on unchained effect. So it's uh it's identified the wrong sort of Django, which is definitely what we don't want.

But interestingly, if we then include The second paragraph, which talks about Django as a technology of conference values And extract again. This time it understands that Django, this is Django the web framework. So I've I've pulled uh uh Google also gives you the Wikipedia links for each item. And I think that's that's impressive. And that's something that would be hard to do with a regular expression, or hard to do if you were building the rules yourselves. It's using the context around the paragraph to understand the uh the what those entities are. I've got some nice demos for this, but I'm I'm short on time, so I'm gonna skip this. Again, there's a content management. uh tool and we've done an integration with Wagtail

that that uh tries to extract the kind of the themes from each piece of uh each each article and that means that you can start Allowing users to uh if if you're an editor of a website and you've got a hundred pages, you probably know how those pages link to each other, how what the relevant themes are. But if you're running a big news site with a million pages, then it gets harder and harder to to establish the connections between all those pieces of content. And but using tools like this, auto tagging using entity extraction, means that you can start creating this more kind of thematic natural way of browsing. And the last one I'm going to talk about is outcome prediction. And this is the kind of the lowest level one. This is the more like the basic tools that you would use for building your own machine learning models. And a way of thinking about this is just is uh kind of prediction. So what will happen now given what we know about the past? It's a bit like Scholy's rule, uh

his definition of machine learning. And you need to carry out these steps. So this is this is not just firing a bit of, you know, your a blob of text within ten lines of Python. You need to do a bit more work. You have to prepare your data. Then you have to train a model, and then you have to evaluate whether or not that training is accurate. And once you're happy with its accuracy, then you can use it. For this one I used uh the Amazon service and uh uh for the test I used this quite well-known data sample. So this is about 100 creatures. And uh we can see their names in the first column, and then we are seeing their it's called features, with basically their attributes So we can see that the blue is the hair column, whether or not it has hair, feathers, so on, whether it's airborne.

And generally when you are creating your training set you want to try to get the features in this in into this state. So you um where possible you're you've got binary uh binary data for each feature. And sometimes that's not possible, but that that's going to make it quick easier and quicker for your training. While I was while I was doing this, uh I the most of the work is around kind of working out how um Amazon, you know, reading all the grim Amazon documentation. And I I came across this uh tweet from Vicky Boykes who's uh I very very much recommend following. developer in Philadelphia, uh who said about hottest programming skills, getting info from AWS. But I I liked I the fact that uh uh I managed to include this in a conference talk about AI, so I felt like I was sort of really, really hot.

So this is how it looks in AWS once you get the right features in. And then you create uh a model based on that data and you test it. and then you are able to demo it. So here's this is the last one of our slots. So someone someone suggests an animal. Animal? A giraffe, okay. Does a giraffe have hair? Yes, yes. Feathers? No. Eggs? Milk? Is it airborne? Is it aquatic? Is it a predator? I don't know. What is it a predator? No. Toothed? Backbone? Does it breathe? Is it venomous? Does it have fins? Does it have a tail? Is it domestic? Is it cat sized? This is a weird one, the cat size.

Alright. Let's see. Alright. 98% confident that you're a mammal, and it's right. And um uh last time I did this um someone someone's C said, well, you know, that's that's stupid, obviously, if it's uh if it's milk, if it's got milk, then it's a m mammal. We know that, right? That's in a kind of you know, we don't need computers to tell us that. But I think that's a really nice uh example of the the Cholet thing, right? So that's the that's the like the classical programming version where where we know the rules. We know that if it's uh if it is milk then it's a mammal. But in this case we don't we don't know that. We're just we're just giving it the data and it's working it out for us. But giving it the data is something that you have to be really careful with. And this is

actually this is a really important point. So here we can see that uh we look down the the left here, most of the bees are mammals. Most of the the letters the the the the creatures that begin with B are ones And uh you know, it's possible that the rule that the the com the engine could try and uh uh uh establish a pattern from that. Right? And um this is this is known as overfitting in machine learning. And actually it turns out you had to be quite careful not to let that happen. Um and this this this starts playing out in some pretty unpleasant ways. So there have been a few stories about this recently. Um you might have seen this one that uh Amazon weirdly that Amazon would make this mistake, um created uh a recruiting tool based on um Uh AI, machine learning. They wanted an engine where you it's gonna give you a hundred resumes and it's gonna spit out the top five and then we'll hire those.

But basically but th they quickly well not that quickly, they they realized that the system was uh was becoming uh prejudiced because of course it was observing patterns in resumes that had happened before. And this is you know a general point that we all have to be really careful about when we're training models The models that we use are going to include all the prejudices and mistakes that we of humans have made in the last in the last decades. And If we just give that raw data to the computers to make new rules out of, they are going to inherit those biases and those prejudices from us. So this is something that we have to be really careful about as we're creating these models. This slide came out yesterday at Google Cloud, and it's interestingly that interesting to see that there's more and more awareness of this.

So they have this fair aware idea that uh and and and and each point of their machine learning that point of their documentation they they're pointing out ways that you need to be careful about this bias, which I'm really pleased about. Okay, I'm nearly uh I'm nearly out of time. I'm just gonna talk about what I think might be coming next in this world And the first one is around reduced complexity. So this whole talk has been about ways that you can use these tools, which are all, you know, they're all super cheap. They're all like, you know. three tiers and then like a thousand requests for a few dollars. Um but you can use these tools to start injecting um amazing abilities, giving your giving your apps amazing abilities. But I think the complexities are going to come down more and more. So the Amazon one was the most complicated, but that's getting simpler and simpler. Last month Uber released this Ludwig tool. And uh again it's in Python.

I mean we'll we're lucky in Python because uh because this is where all the action's happening. Um and with this tool you you can just you just uh provide a CSV file and it will try to do the modeling for you. So it's kind of just removing all these steps. Last night Google uh launch something similar, Auto ML table table. So this is similar. uh imbuing your past your your your models with the biases of of past generations. Also the generation thing so we looked at image generation but text generation is uh is happening too This is the OpenAI group who quite controversially didn't open their last model

called GP22. because of their ethical concerns about it, because it says it's too good. And uh as as an example, so they took forty million articles from the internet and then they start giving it some sentences and then it and then it spits out more sentences. And it they gave it the sentence Um recycling is good for the world. No, you could not be more wrong. And the and then the computer came back with recycling is not good for the world. It's bad for the environment. It's bad for our health, it's bad for our economy. I'm not kidding. Recycling is not does this sound like anyone to you? Um uh uh you know this is scary and uh and it's something that we need to start dealing with is like is understanding the fakes And finally, I'm just going to talk about machine learning at the edge. I think this is going to be a big trend in the next couple of years as well. I have a Google Pixel phone, which is And it's got this amazing camera. I don't know if any of you have got this, but the the most amazing thing about it is the night

sight vision. Has anyone experienced this? And uh you take a picture in almost dark And uh it seems to kind of reveal stuff that the the the eye can't see, and certainly the lens shouldn't be able to see. And this is a combination of um uh of like computer vision but also machine learning. So it will take lots of little pictures, it will it will adjust for the movement in your hands, but then it will use a machine learning model. on the device to work out what the lighting should be based on all the other images that that's that the model has had learned from. And you come up with these extraordinarily realistic and believable pictures that the eye can't see. Alright, so next steps. If you want to learn machine learning, I really recommend Francois Cholet's book, Deep Learning with Python. There's an amazing online resource called Kaggle. If you want to do something with machine learning, then I should just read the docs of these various services and build something amazing.

Thank you very much