Connecting Patients to Doctors in Real-Time Using Django

Speaker 1: So my name's Jacinda Shelley. I work at Doctor on Demand. Um So what is Doctor on Demand? The rest of the talk will not make a heck of a lot of sense if you don't know what the product actually does. I could explain it to you myself, but shortly after we launched we got a bit of free publicity that we could never have hoped. to have asked for in a million years. We got picked up by the Colbert show, so before I start, I will let him explain exactly what we do. It was like a free advertisement. Oh. And you can kind of tell when this was released because of the reference to uh

Speaker 2: doctor without the crippling having insurance. Jim?

Speaker 3: Welcome to Doctor on Demand. The Doctor on Demand app lets you have an immediate face-to-face consultation with a real board-certified licensed physician who can diagnose you and prescribe medication. Dr. Tong is one of about 1,000 doctors nationwide.

Speaker 4: treating vacations via app.

Speaker 2: This is a major breakthrough. Medical expertise with the convenience of a smartphone. Why waste time? Getting an exam, we can just shoot your doctor an emoji of your shattered femur. Got a little sunglasses, isn't that cool? With doctor on demand, you FaceTime with an doctor from your computer, smartphone, or tablet, which is great news for anyone with an iPhone, an Android, or a public library where they can press their junk up against a webcam. gonna revolutionize medicine. It's designed to modernize the healthcare system by using mobile technology to make a simple doctor's visit stress-free for patients and doctors alike.

Speaker 2: Yes, it's less stressful for everyone You really think your doctor enjoys cupping your balls? No. Now he can watch you do it yourself. And hey, if you get really good at it, I know a site that'll pay you $40 to do it on camera. Once you connect with the doctor, you can even upload photos. for them to examine. In fact the only thing you can do is send a stool sample. Learn that one the hard way. Ruined a perfectly good headphone jack. Clearly app based Healthcare is the future of modern medicine.

Speaker 1: And we'll just cut that off right now because it automatically goes to the next one. On our website, this is what it actually looks like. You go in Put in symptoms, um, fill out a few basic pieces of information like you were at a doctor's office and The system connects you with a doctor in minutes if everything is working correctly, which it normally does. So That clip was actually a little bit scary because we were on the West Coast and all of a sudden we had friends on the East Coast calling us up and being like, hey, did you see you were on the Colbert show? And we're like, what? um what happened there. But it turned out to be a really good thing. And it's a lot easier than me

Speaker 1: trying to explain it because he does it a much better than I ever could. Doctors get paid per call. So lots of stuff with the website is the same as you might expect. We have all of the same issues that any web app has. You have data that people need to access. Lots of it is static or custom. People after they have visits, they have charts associated with those visits. We have access controls. We have all of the standard stuff But um when I was looking at Django Khan I was like, oh hey, they're still accepting proposals. One of my coworkers was like, oh, you know that thing that you built that does the matching? You should talk about that. And I was like, oh, well what's the worst they could do? Just not accept me?

Speaker 1: No, the worst that they could do is accept me. Um So there's there's lots of cool stuff. There are lots of other talks about how to make that stuff work well and frameworks and APIs. That is not what I'm talking about today. Not everything is the same. The biggest thing that is different is the fact that we have to keep track of where people are in the call process. So people want to initiate this matching process and you don't know exactly how long it's going to take because you don't know exactly which doctor is going to pick up and you don't know um if one of the doctors is asleep at the wheel and you have to call other doctors um to make sure that it connects appropriately. So

Speaker 1: We also have to deal with the fact that regulations say that I can't just route you to any licensed physician in the U. S. Oh no. No. My life would be much simpler if we had one indivisible country that did not involve states Instead, what I have to deal with is the fact that Dr. A has licenses in these five states, Dr. B has licenses in these seven states, and I can only connect you with a doctor that is licensed in the state that you are currently located in, assuming you have never seen that doctor before There are other rules if you have seen that doctor before. But if this is your first time, I have to connect you to a doctor that is licensed in your state. There are 52 cues in that case because there's all of the US states, there's

Speaker 1: DC, there's international calls that can actually go to any doctor. And there are also some other territories. So there are over 52 different cues that you have to deal with. Um so One question you might ask is this sounds like it's been solved before, right? Like does this matching people who need help to people who can help them sound at all like an application anyone else has ever dealt with in the past? Maybe customer support? So why not just use an existing customer support system and do the routing that way? Um there there are a number of reasons. Um partly why I'm here is if anyone knows something

Speaker 1: that would mean that I built this system in vain and I don't have to maintain it or make it work better. Please tell me. That would be awesome. But assuming that that is not the case, I we looked at several different systems and none of them could meet the requirement that we had where When patients are calling in, it actually has to be a video call for most purposes to satisfy state requirements for a legitimate telemedicine visit where a doctor could prescribe medication if appropriate. Actually, a lot of people ask if we're a pharmacy pill shop. We're not. There are a significant percentage of our calls that don't involve prescriptions. That's a question I get asked all the time. But

Speaker 1: in this case, it has to be a video call, so we would need something that can support the support iOS, Android, and web clients in terms of connecting their customer support routing system with our video, the video technology that we use. And there was nothing that was flexible enough to drop in in that manner that I found. There there seemed to be mostly monolithic solutions. for call routing where if you have a customer support system and you have a voiceover IP or traditional PBX that they support, then they will do everything for you. They will handle all the scheduling. There are massive amounts of ways that you can configure it. But if you just need something simple and you just want to plug it in, not so easy.

Speaker 1: So I got the opportunity to look at a bunch of different things. And again, I may not be the unique snowflake that I think. And I would love for someone to tell me that I can just throw all this code out and never have to worry about it again. So without something off the shelf. What did I get to do next? I got the chance to use a lot of technologies that um might be common in More Python backends development, but that you often see as kind of side notes in Django project. So the first option that we needed to uh look at was well before I get into the first option of what I considered um

Speaker 1: the data that you need to keep track of for this is Pretty straightforward. You have all of your patients that are calling in and what state they're located in. And then you have all of your doctors that are currently online and available and what their current status is. So to simplify things a bit, the current statuses, there are more than these in reality, but current statuses can be considered available for a call, currently being paged for a call, or actively in a call. Um and what states they're licensed in. So that's all the data that you need to keep track of. And First option is to just use long-running HTTP requests.

Speaker 1: Now, this has a number of issues. Um so one thing that it does work for is a very Quick proof of concept if you just need to get something running and show people how this is supposed to work. So this is actually the first thing that I did. You set the timeout to be really, really long on your web server. You allow the request to go on for minutes at a time and assuming that you know that your doctors are generally behaving correctly, which they are in a proof of concept setting because it's all staged. then everything is fine and dandy. It's simple to implement. It's good for prototyping. You just kind of use a database and check what's going on.

Speaker 1: In terms of so you have a a visit object and that has a status of whether or not it's been connected yet. You can use models for everything and it's all fine and dandy when you have a few demo patients and a demo doctor. But it does not scale. And worse than that, even if it did, it's flaky and unreliable because if a client loses their connection. They don't necessarily have a way to get that status back. If they did, how would they find out the result? So you end up Starting with something more like option two, where if the client's flaky anyway, you do need a way for them to recover from that and see where they are in the current match process.

Speaker 1: and get reconnected to a doctor. So you might as well start having the ability for the patient's client to ask what its current status is and whether you've found a doctor yet. So Ajax polling and then on the back end because you don't have this long-running request for the match, what I did is use Celery, which is an asynchronous task queue written in Python. It integrates really well with Django. Another talk would be on like all of the configuration options for it. I will not go into that. There are many. It works well out of the box, but there's lots of stuff you can do to tune it. And that kind of fell naturally into this use case

Speaker 1: because I could have celery, lots of workers running in the background. Um it scales really well uh as we have more patients coming in, they're just able to uh check the status of the doctors and page a doctor one at a time. We have like a built-in business requirement that if the first doctor that's called doesn't answer within a minute, then you start paging another one and another one and another one. And these can just go on for as long as they need to with some arbitrary timeout. Um and so that's all great. So the pros for that, asynchronous task, you don't have to worry about request timeouts. Um cons for polling. It's no longer real time in the way that your long-running HTTP request is.

Speaker 1: So with the long-running request, you just respond as soon as you have a match and you're great. With polling , to simulate real time, you have to keep making this request to the server over and over and over again. There's gotta be a better way. And there is. So you can still use polling as a backup, but the optimization that I went to was using WebSocket, which is an HTML5. Technology that allows clients to receive pushes from the server. So they open up a channel, it's lightweight, it's perfect for this. And then on our mobile devices, you also add in push notifications. So if the user has enabled push notifications, uh

Speaker 1: you can use that to as soon as a doctor accepts the call immediately push that information to the clients. And then even if you have polling as a background backup, it doesn't have to be as frequent for it to appear real time to the majority of the patients calling in. The cons, um, it has a few extra mu moving parts. Uh you have to worry about setting up this extra service. If you don't want to set up your own, there are now SaaS providers who do WebSocket for you. I think the biggest one is Pusher. And I'll help you with that. So stage two in making this thing work well. Tracking doctor

Speaker 1: status. We have a lot of doctors that can be online at any time. They all have licenses in a variety of different states, like I've said before. The first option would just be to have all of this be in the models and then every time someone makes a call in make the appropriate queries to figure out who's online, who's not, that's slow, it could be out of date. Um the pros of this approach of just using the database are that there are no external dependencies And I think that is really the only pro because it's slow, the data can get stale really fast, you have to make all of these extra reads and writes to your database. It's inefficient, it's not good. Depending on how your models are set up and the way that ours are set up, it involves quite a few filters and joins

Speaker 1: to do this just using the database. So I found that Redis worked as a really good solution to this problem because I needed the data to be persistent. Did I spell persistent? Right. Um I think so. No. I will fix that later. Uh the Uh and that's gonna be on video forever. All right So uh Redis is really good for persistent data structures, and the data structures that I used are the obvious ones in this case. We have a data structure that has the doctor's ID

Speaker 1: and their current status, and that's what we update so it's used as a cache for this ephemeral data of what status the doctor currently has because we only care about it while they're on shift. It's not something that we need to store long term. And the other thing that is useful in Redis that I found is uh cues. So the first link there is just the official Redis client. Well, not official, but the biggest Redis client available in Python. The second one is uh a Q package that I use, so uh what you can do in this case is instead of storing everything in the database um have cues in Redis of the doctors that are licensed in each state

Speaker 1: when they become available, push them into these cues and then as patients want to see doctors in these states pop off the next doctor, check their status, and then if they're available, match them. If they're stale, just discard them at that point and you can keep using cues this way. One disadvantage of this, so if you start with just a FIFO queue and you have to have a separate cue for every state, you can end up with duplicates depending on how you do it. where the doctors that have licenses in more states will get richer and get to see more patients uh because they'll end up in more cues more frequently. Uh so one of the optimizations you can do instead is to

Speaker 1: uh use A priority queue, which won't have duplicates, um the way that a FIFO queue would if you just have IDs and it's kind of appending to the end. Um and it has a score, so whoever's seen patients the least recently can get the next patient available in that state. The last thing that I came across that I thought was interesting in this process of developing this architecture was atomic call acceptance. In February, the last half of February, we decided to do this wonderful thing where we made all calls free for two weeks. Um that was painful for me. Uh everything

Speaker 1: worked, so it was actually really great. But uh one corner case I discovered at that time was that it was possible to have multiple doctors accept a call simultaneously. I thought I had protected against it using pipelines, but I hadn't done it correctly, so I actually switched to something that was a lot easier to read. Uh and that was just to use a simple uh lock. And this is just kind of a way that You can do distributed locks and resists in their official documentation. What you do is you set a unique ID. That second part that's the value is It doesn't matter what that is if you're just using it as a lock. Um if you're doing something else with it, uh

Speaker 1: this is a standard key value store. And then the two options are that it expires after a certain amount of time and then you only set this if it's not already set. So what this does is um Say I set doctor two um and or actually say I set call attempt So this patient is calling in, their call is number two, and both doctors try and accept at the same time. I mark call two, I set the value as uh some random value and only one of them the o only the one that gets the lock gets to accept the call so you can't end up with two doctors accepting the call at the same time. And that's just a neat trick if you're using Redis already and you

Speaker 1: come across Something that would use this. So in the end, we have an improved system where the database does what it's good at. We have celery taking care of asynchronous processes that can end whenever they need to. Specialization improves our efficiency. We can at some point in the future offload All of the salary tasks to other servers, it clusters well, it's very scalable, it's nice. Um and Redis keeps track of our ephemeral data and lightweight persistent cues. And that is the end of my talk. So there's now some time for questions. Thank you for listening to me. This was the first talk I've done at DjangoCon, and I'm really glad to be here.