I Can't Believe It's Not Real Data! An Introduction into Synthetic Data with Mason Egger

Hi everyone, my name is Mason Eger and I'm the lead developer advocate at a company called Gretel. Thank you for attending my DjangoCon talk today. Uh sorry I couldn't be there to see it in person with everybody. Was really looking forward to it, but just wasn't in the cards for me this year Today I'm going to be talking about synthetic data with my talk, which is titled I Can't Believe It's Not Real Data, An Introduction to Synthetic Data. So the first thing I want you to do is kind of set the stage, just kind of imagine some scenarios, if you will. Say you're a developer working on a web application. Maybe it's a Django application. We are here at DjangoCon after all. Um, that manages uh students in a classroom, whether it be grades, attendance, you know, things that students would need in a learning system. So you're building out your application, you're working on it, and now it's suddenly it's time to test. So you're ready, you know, hey, let's go ahead and test this, make sure there's no bugs.

Um let's maybe test it on some production data. Let's test it on some data that we know is kind of already being used. We're gonna have to migrate off this old system into this new one. So let's test it on that data, but then you kind of realize that you can't. For security reasons, you're just not allowed to have access to the production database. This is good for security reasons, but there's also like things like FERPA, which is a law that protects students' uh information. So not not only can your boss not give you access to it, it's actually maybe even illegal for you to have access to any of this data. Um so you end up having to test uh with a database that maybe only has a handful of records. Maybe you pre-populated databit yourself, maybe you write some fixtures. This is all sample data, it's all guessing data, but it doesn't really represent um the population that you're working with. And because of this, maybe an edge case slips through.

Maybe something wasn't represented in your test database that might have been caught in the real database had you had access to production data, you know, but you just you didn't have access to it. So you were insufficient with the data that you had to test your application. Imagine that scenario. Another scenario you can imagine is imagine that you're a data scientist. And you're trying to build a model. In particular, you want to build something that figures out and predicts a rare disease. Maybe it's heart disease, who knows? But maybe you're trying to predict some sort of illness or something based on parameters. So you start looking for relevant data sets. You know, you go to uh you know your database system and you look around and you find a little bit, but you really don't find a lot. There's not really enough. It's not a good sample size. It's not a good representation of the data. But you know, you still have to train this model.

Like you've spent all this time kind of working on the model. And you so you you train it anyway. You know, like let's just see what happens. And As you kind of were suspecting, the model is unsuccessful. You didn't have enough data that the model really couldn't learn um anything. So you you really are kind of you don't have anything really more to do here. You don't have much to do. Um, but wait you overhear from a friend of yours at another hospital um that maybe they have some similar data, maybe they have a larger data set of this that you could use. And you're like, oh, you get so excited, like yay, I'm gonna be able to, you know, combine this with my data and actually test my model. And that is a very short-lived happiness because you can't get access to this data because of PII. It has personally identifiable information and it patient records are very well guarded and protected by law as they should be. So you're just not allowed to have access to it.

So it kind of just dashes your hopes right there. Um and I'm hoping that as you listen to these two narratives and you kind of Think about like what was the common theme between the problems in these, you come to realize something that I've realized, and which is uh in the reality is data is a scarce resource. It's not something that everybody has access to, but it is something that most developers need. Almost all developers, all data scientists, machine learning engineers, people need data. So it's a really scarce resource and there's just some problems that, you know, most of the time we don't have access to usable testing data. We need access to, you know, this data to be able to test our models or our uh programs effectively. You know, 35% of all data science time is spent in the data gathering stage, as you can see from this graph here on the screen.

Um, you know, it's not even really in the fun parts. You spend all this time gathering the data, you get to build the model and productionize it, but majority of the time. You know, over a third of the time spent is spent in the data gathering stage, and data is often inaccessible due to PII. So even if you are able to get it, you can't really access it. Um, you have limited data sets. This is another problem that you have. Um, as we saw in one of our previous examples, lack of quality data can affect model training results. You know, it just doesn't, we weren't able to garner anything from that. um from that machine learning model because there wasn't enough data for it to learn and be able to accurately predict something. It can also even be prohibitively expensive or even some cases impossible to collect more data. Maybe the situation that you were monitoring was like, you know, maybe it's a comet that passed through the sky and that happens every 2,000 years on Earth.

Well, you know, it's passed through. You're not getting any more data on that, at least for another 2,000 years. the likelihood is none of us will still be around at that time. So it can it can literally be impossible to gather more data on a specific subject. And then you even have biased data. Biased data is a real big issue as we all see whenever we read any news news article about AI or machine learning doing something heinous. Um, data sets can be skewer towards representation of subjects in a general data set. It could just it doesn't lead to a good experience. So If this is the reality that data is a scarce resource and we have all these problems, how do we solve this problem? How do we make it where we do have use access to data? How we aren't so limited, and how do we handle bias in data sets? And the cool thing is there's actually a solution to this, and the solution is synthetic data.

Um, so you may be asking, what is synthetic data? I've maybe have never heard of that. I know I hadn't as of like a couple months ago before I joined this job. I had no idea what solution data was Um sorry, synthetic data, not solution data. Um synthetic data is, and this is a good quote from um our Chief Product Officer at Gretel, synthetic data is artificially annotated information that is generated by computer algorithms or simulations commonly used as an alternative to real world data. That's kind of a mouthful. I've kind of broken it down here too. Synthetic data is created by training a generative machine learning model on your data. So you give it your data into a machine learning model and it creates more data that looks like the original data. Now your first question may be, and my first question was, well, isn't that just fake data? Um and the you know the answer is actually no, it's not.

Synthetic data is different from fake or mock data. You may the first thing that may have popped into your mind may be the The library faker, where you're just, you know, generating data. Um, faker mock data may not be representative. It's purely random. And if somehow you do come across fake data that is completely representative, um you manage to make random the random number generator be completely representative of your data, go buy a lottery ticket. I don't know how on earth you did that. You have really good luck. The likelihood is is that fake data, which is just completely random, will not be representative. Um, another issue that we've often seen is that fake data can very often be too clean. You know, for anyone of us who've ever actually worked with data or if you've ever seen it, you know that nobody fills out the form you want them to fill out the form. They send you weird stuff.

Some people's into SQL injections. Hopefully you're handling that. So the data can come in too clean, and that doesn't really set an expectation because we have to test our code for dirty inputs as well. So, you know, if you have those dirty inputs, you can synthesize them. But the faker, the fake data doesn't really come in and doesn't give us the chance chance to test on those opportunities. Synthetic data, however, is generated from existing data. It will look and behave like your initial data set. A very common question I get is well what if I send it all garbage? And I'm like, then yes, you will get it. That's the whole point, is that It takes the data you have and it makes it look like the data. It can't magically fix your data. If there was a solution that magically fixed everyone's data to where it was perfect, that would be, you know, amazing. People would be all over that. But You have to still do the work on your end to ensure the data that you have is, you know, good data, representative data of the population.

Um, and in fact, synthetic data can be nearly as representative as the initial data set. So, like when you have this synthetic data You're like, well, how do I know that it's, you know, still like that it keeps the correlations? Well, it definitely can, and let's talk about that. Um, so let's talk about how accurate is synthetic data. Synthetic data accurate actually is really fairly accurate. Unlike fake data, it can be nearly as accurate as the real data. In some cases, the accuracy is improved. So there's a lot of different ways of measuring quality. Many different projects do it in many different ways. um doing some sort of uh mathematical quantitative analysis of it or just checking to see like you know how far is the data skewered using different mathematical models and such. I'm not gonna get too much into that today because that we could spend the rest of the talk talking about that, but it definitely is a thing.

However, the cool thing about synthetic data and what where synthetic data, you know, as it is as accurate as real data, it means that downstream data consumers can readily make use of synthetic data. So one of my colleagues at Gretel um trained a classifier that uh you know on some synthetic data that was essentially a project that was trying to see out if the user bought pizza or not. So it had your shopping history. And instead of giving that shopping history to just a company and then it has your information, like you know, it has Mason's PII, Mason buys pizza and funyans and Lord only knows what else I buy, instead of giving that data to the model Um, we were able to synthesize it and then you know basically remove my PII and then see did this user buy pizza or not. And as you can see from this chart here, which is this is the accuracy.

The logistic regression of the synthetic data was actually slightly higher than the accuracy of the initial data set. So synthetic data very much Can be as uh accurate as the initial data and we can prove it, which is fantastic. Good science is being able to prove your science. So you may be asking, well, what can I use synthetic data for? Like this sounds like a really cool tool, but how does it affect me? Um, and in reality, synthetic data acts as an alternative to real-world data in almost every situation. Any task where you need data, you can in theory use synthetic data. You can use it for training models. You can use it for testing applications. You can use it for creating sample data for demos. We're seeing this a lot right now You can use it for anonymizing data so that way you can share it or protect people's privacy.

And more, there are so many different use cases for synthetic data. Unfortunately, today we only have time to really talk about the three that I really like, the ones that are the most that I think are the coolest, the most important to me So, how can synthetic data help you? Well, number one, it can help you make private data accessible and safely shareable. So you can take data that you know you shouldn't be able to share and share it. Number two, it can generate more samples with limited data sets. So if you don't have enough of one data set, you can augment it to make it larger. Or you can reduce bias in machine learning data sets. So we're going to go over that. So let's talk about the first one: making private data accessible and shareable. So data often contains PII, personally identifiable information, making it very risky, or in some countries, even illegal, for developers to work with this data.

Um and let's let let's be let's be real. Developers don't want to work with data that has PII in it. They don't care if it's my name in the data set or if it's some other random person's name in the data set. They care about the correlation between the data. They tell They care about the story that the data tells them, not necessarily the personally identifiable information in it. And more often than not, developers don't want to touch this because now they're liable. for this information, it makes them accountable for it. It would be so much better if they could have all of the data and none of the responsibility. You know, we don't have to worry about protecting someone's privacy because there's no private information in the data to begin with. And that's what this can do. So you can generate synthetic data sets allowing you to have statistically similar data sets. So like the the correlation between them is the same, but you can remove the PII.

So it will completely generate an entirely new row or however you whatever your data has looked like. But then we can go back and look at it and say, yes, this has the same correlations and the same relationships as the initial data set. This will allow you to not only share your data internally. So now imagine, you know, in that first example, I could just access a synthetic database that was trained on the product database. The data is will look and behave almost identically to the initial database, but none of the risk is there because none of the real personally identifiable information is in there. So that's great. But then it also solves the second case problem that we saw scenario, which was now I can share this with my friend at the other hospital because I can synthesize the patient records at one hospital. and then send them to the other. All of the data, like this, you know, this heart rate, this blood pressure, this, you know, I don't know, eye color, for some reason

indicates a disease or something. Now we can do that and we don't worry about compromising patient safety So we can now share data in ways that we were never able to before. Another really cool example that I like to talk about is imagine you have all of your data in an S3 bucket. It's a very common place for data scientists. to store their data. And instead of going up and accessing the data and pulling it down, every time you went up to access it, it just synthesized you a new set. So so nobody would ever have the access. to pull the real data set again, pretty much ensuring a really good data privacy uh story and reducing attack vectors, which is awesome. So making private data accessible and shareable is definitely probably my favorite thing that I think you can do with it. There's so much cool stuff. But I I just I sit here and I think about all the cool things we could do if we could share all of our data with everyone in the world.

Think of the things we could accomplish. It would be really cool. The second one, the second really cool superpower of synthetic data is augmenting small data sets. So a big problem, as we saw in our initial one, our our our initial example was not having enough data and not having the right type of data. It's a serious bottleneck. It's very often your most valuable asset. And as we said earlier, collecting data can be expensive, it can be difficult, it can be apps, actually, it can be impossible. um to collect more data. But what synthetic data allows you to do is it allows you to generate an unlimited amount of data based on that relatively small data set. So a good example of this is you have a machine learning model and a small amount of data. You want to you you can use synthetic data to regular

regularize hard word to say your model training So a good question that I always get kind of asked here is, well, what if that initial data set was like terribly biased or you know not representative of all? Will that just will the synthetic data create more of the same? And the answer is yes. That's the whole point of synthetic data, is it creates more. Now, what really helps with this though is say you have that really small data set. And you couldn't really determine if it was representative because it was too small and the model wasn't learning. Well now you can synthesize as much as you want, so you know that it's not a lack of data problem. And then you can use that as the next step in your debugging. Okay, we know it wasn't a lack of data because we literally generated a billion records and sent it through. Now we know that this data set is

not representative. Maybe our model math is wrong. Who knows? But now instead of being stuck on data, you can be. But well, maybe you can solve the problem, or you can just be stuck somewhere else. It's kind of like when you're programming and you have a bug and you fix the bug and then another one comes up and you're happy because at least it's not the same bug again. It's kind of a similar thing here. And another really cool thing you can do with this is from a testing standpoint, you can load or stress test your application. Like imagine you want to be able to recreate traffic to your website. Um, but you know, it's kind of difficult, or you maybe you aren't like storing it in like maybe like a Kafka Apache Topic and you want to replay, you know, eight months, eight years and three days. I've seen that happen before. But now you can just generate a whole lot of data and you can stress test your application and it's gonna look like real traffic.

And that's really useful there too. Now, this is also a really fun one, reducing bias in data sets. This one takes a little bit of work on the on your side as well, but it's really cool to see how it can be done. So, as we all know, biased data is a very big problem. On the best case scenario, biased data um just leads to a unfair model. And the worst case scenario, the bias data is leading to maybe a model being trained that causes actual harm in the world. And that's a terrible thing. Um, ethically, that's awful, and we should do everything we can to stop that. Um, but what we can do is we can reduce by in the data sets. um with synthetic data by augmenting the data set towards another uh like to kind of balance it. So then an example that we have here

is is if you can identify the bias in your data sets, then you can use synthetic data to balance it out and then regularize regularize. That is gonna be a word that I'm gonna trip over for a while. regularize your data set. So there's a really cool blog article you can read here. I also have these slides on my website they're at mason. dev. You can find them. Um, so you can get all these links and read these blogs, but basically, this synthetic data article was able to reduce AI bias with synthetic data in heart disease prediction models. So there's a heart disease prediction model that's pretty famous. I think it's on Kagel. And it was 68% male data, 32% female data. So it's a two to one ratio bias towards men in being able to predict heart disease What we were able to do is we were able to use this synthetic data to generate more female patient data to balance the data set.

So instead of it being 68% male, 32% female, we were able to balance it to where it's a 50-50. Where it's no longer biased, it's no longer skewed in one direction. And it allowed us to increase the accuracy of this model from 88. 5% to 96. 7% in predicting heart disease Which in turn led to 6. 17% more females with heart disease accurately being diagnosed. Now you may be looking at the math and you'd be going, well, Mason, that math doesn't add up. Well, this is where it got really interesting is not only did when balancing the data set did the female prediction rate increase, but the male prediction rate rose rose also because the model wasn't guessing. It was regularized and it knew, okay, this is what happens whenever you have this set of parameters. So this is a really cool one.

So if you can identify the bias in data sets, then you can, you know, counteract that by balancing it out and synthesizing more data. I get really excited about this. Um, maybe this is a topic you've never heard of before. It was a topic I surely had never heard of before. Is it is it being used or is this bleeding edge? Well the answer is both. Um it's being used in a lot of places, but it's still a very new technology. Um automotive and robotics are using this all the time. Think of self-driving cars. We can't possibly have enough street data from all this, so a lot of synthetic data is being used. to create uh data to train models for self-driving cars, helping test safety and crash preventions. In the financial services, they're using synthetic time series data to enable sharing that doesn't compromise customer privacy. In the cybersecurity infosec uh areas, their

synthetic data is being used to detect rare events, including fraud and cyber attacks. In the uh healthcare and life sciences, this is a really fun one you should click on this blog. They're literally synthesizing human genomes and doing synthetic genomic data so that we can use genomes to uh do medical breakthroughs and be able to predict new diseases without ever compromising someone's privacy. And this actually took me a minute for my head to wrap around. It's like, is a genome PII? And the answer is Yes, the genome is literally like the the biological blueprint of a person. Blew my mind. I'm still fascinated by this. So synthesizing genomic data. Manufacturing, being able to tell when complex complex supply chains are going to have issues, and more. There's so much more that synthetic

synthetic data is being used in action. Now I may have got you all hyped up about this and be like, oh, it's the greatest thing ever. It can do no wrong. And that like all things in life, that's not true. Um, there are currently some challenges in synthetic data, and I like to call them out up front and forward so you know what you're getting yourself into. Highly dimensional data sets with hundreds of thousands, hundreds or thousands of columns can be uh very compute intensive. These are still machine learning models, they still take uh you know clock cycles and resources to to train. So the bigger the data set is or like the more columns it has, the more complex it can get. Synthesizing relational databases can still require some manual configuration to be able to keep the relationships between the databases uh intact. There are different privacy preserving technologies that are being done in synthetic data, such as differential privacy, which is basically just adding a little bit of noise to help with the privacy aspect of it.

Unfortunately, these require large amounts of data to provide strong privacy guarantees. So if you don't have a very large data set, you may not be able to turn all of these on in the initial stints. And then synthetic data generation requires time and effort. Like these are still machine learning models you're training. It takes time. So it's it's not instantaneous. Um streaming synthetic data is a big conversation topic, and you know, I think we're one day we're gonna get there, but um it still takes a lot of time and effort. So What is the future hold for synthetic data? Is this a fad? Is this going away? Like, you know, what's happening? And in reality, I think it's here to stay for a while. Um, Gardner is already predicting by 2030 that synthetic data will overshadow real data in all AI models This is already happening today in some very large companies. And it allows for easier compliance under data protection laws such as GDPR and CCPA.

So This is another big thing is that if this data is not, doesn't contain PII, then data that became protected and unshareable due to GDPR and these very necessary privacy laws, well now we can share the anonymized version of it and we can do a lot more with it. It also reduces the attack vector on data. Also, one of the really cool things that's coming in the future is eventually going to be able that synthetic data will be able to solve the cold star problem, which is What do you do when you build something but you don't have a model to start with? And then generating synthetic data from thin air instead of training it. That's still No, I don't think it's as far off as I as as we think it would be, as with all things in tech, things that we think are very far off will be released in beta tomorrow. Um But yeah, it's really cool time to be in the synthetic data space.

So how can you get started using synthetic data? Well there are many, many, many resources available. You can check out a few of them here. Open source agenda has a synthetic data section There's an offense awesome synthetic data list that we have curated. There's lots of different open source options available. There's Gretel Synthetics, there's Synthetic Data Vault, there's Stable Diffusion, which is if you want to do image-based synthetic data. So you can go and check out all of these. Um the one that I'm going to talk a little just very briefly is Gretel Synthetics. This is the open source project that's managed by the company that I work for, Gretel It's an open source project, as I said. It's got multiple models within it. So we have an LSTM long short-term memory model, a GPT model for text-based. I believe CTGAN is a better I don't remember what CTGAN is.

No, that's not the time series. That's doppelganger. CTGAN is like another vert like another good thing for tabular data. And there's more to come. The thing that Gretel 's doing is adding more and more of these models. So these will constantly be added to our open source project. You can train the model yourself. If you are doing it, you're probably going to need a GPU. This is training a data science model. You can do that with Google Collab. Check it out here at Gretel Synthetics. um if you want to and here's some docs on it. Now if you want to get started with it like right now, like you're super excited, but you don't really feel like training a synthetic model. There is a cloud. We have a c there's a a cloud-based option. So if you don't have a G or if you don't have a GPU, that's another big thing. I would my GPU is like a 1070. It's right here on the side. You can't see as I'm touching my my my desktop. Um, and I don't know if I would want to train a model on it. I bought the thing like in 2016.

I think it would still be good. But who knows? Um, so if you don't want to have a GPU or you just want to try it out really quickly, you can totally try out the free tier at Gretel. ai. Just sign up for it with your email address, and you'll get uh enough data to train a couple data models, train enough data sets. Um you can train synthetic data in three lines of code, and here it is. You literally just import the model, you specify the data set, or I'm sorry, you import the library, you specify the dataset. You tell it, hey, create a trainer instance. Here's the data set. Generate me some synthetic data and show me what you got. Um really cool stuff. I love this now. Some other resources you can talk up you can look up to is you can look at the the Gretel AI docs. Um Gretel Blueprints is a really cool place where we have a lot of um already like production ready synthetic data notebooks that you can use to kind of like jit

look at generate some clone them whatever you want and there's also a fun with synthetic data repository that I've been doing that you're more than welcome to contribute to if you want. Um and this is where we just do goofy things with models. I have one that generates synthetic cocktails, it takes a whole list of cocktails, feeds that into GPT, and then Spits out, you know, a synthetic cocktail. Um if you want some free swag, you can scan the QR code there or the one on the screen that's right there. Haha, I got it Um, and we'll send you some free stickers. And if you want to learn more about synthetic data, we'll send you like I'll send you an email with like some of my favorite resources and stuff. The form will close a week after the premiere of this talk. I should know when this talk premieres, but I don't off the top of my head. So a week after today at noon central time is usually when I close it to give people the opportunity who watch this after the fact on YouTube. To be able to get some stickers as well.

And if you end up wanting stickers and you don't get this, just tweet at me. Like you can just tweet me at Mason Egg or even tweet at Gretel AI, follow us on Twitter and tweet at us, say, hey, I want some stickers. I'll probably give you some. But yeah, thank you so much for your time. I really appreciated being able to present to you today. If you have any questions, go ahead and ask in our Slack channel. The slides are on my website, mason. dev. Thank you so much. Don't forget there is swag, so get some swag, fill out the form. It's like what's your email address and then we'll get out to you. Thank you so much. I hope you're enjoying DjangoCon. I hope to make it out there next year to see everyone and have a good day. I'll see you later.