Python for Planet Earth: Climate Modeling and Sustainability in Action with Drishti Jain

Speaker 1: Yes, this is the only and the best Earth we've got. My talk today is going to be about Python for planet Earth, about how Python is being used for climate research. A lot of open source projects that you all can contribute to as well. We, as developers or technology enthusiasts, do not need to be researchers or scientists in order to actually make a mark. in research in the climate and environment space. So let's dive right into it. Okay, as we dive into it, uh a little about myself. So I'm a software developer and a machine learning engineer. Uh I'm based out in San Jose, in California. I'm a published tech book author. My book is on Larval, which is a PHP framework.

Speaker 1: So yes, I'm somebody who dabbles across different technologies. I'm a social entrepreneur. I have a nonprofit organization back home in India. We have 400 plus active volunteers working across 11 cities in India. I'm also an international tech speaker. Being avid in the tech community, I've received so much from the tech community, especially through open source projects. That is what is helping me day to day. I always find ways to give back to the tech community and speak and share my knowledge at different tech conferences. And I'm a career coach. I have my own initiative called Skill Up with Rishti, wherein I help folks transitioning roles between tech Being able to upskill in tech, switch roles, find better opportunities, find the right fit for their skill set in the industry.

Speaker 1: If you scan the QR code, that is Everything about me, different links that you'll find. You can find me across different platforms. And my username generally, the one that's on X or Twitter, is the same across different platforms So feel free to shoot a DM if you are ever exploring this space. Now, as we talk about climate change, firstly, why does it matter? The very first point as uh my gracio host uh mentioned while introducing like this is the best earth we've got we see climate change on a daily basis um with warmer summers with more colder winters that's a direct impact that we see Even when we are not really thinking about climate change, we don't have to go ahead and think about it in order to face

Speaker 1: it. We face it on a day-to-day basis. Uh so climate change is one of the most critical challenges today. Everybody is trying to solve it, trying to mitigate it. Companies are trying to be carbon neutral. But still, we see it going towards the wrong direction of it still being impacted. So there are a lot of data-driven insights which can help combat a lot of climate issues And as somebody who's well versed with technology, all of us, we want to look at data and find out ways in which we can help improve it. Anything can only be improved if it is measured and acted upon. There's a lot of data that gets collected and is out there in the open source world. But not a lot of folks put in a lot of time or effort in order to derive insights and make the best use of it.

Speaker 1: And in climate science and sustainability efforts, Python is at the forefront of it. A lot of folks I was talking to yesterday and today during the conference think of scientific research to still be happening in R uh the language are but that is not the case a lot of open source python libraries are out there which help you do a lot of climate modeling um right at your fingertips. And if you know Python, then you will be able to contribute in scientific research. So Python and climate science. Python being open source, flexible and is widely adopted in research, you'll be able to see a lot of code already out there which we can build upon and help improve in There's a rich ecosystem of scientific computing libraries and a strong support for geospatial and environmental data

Speaker 1: Whenever you think of climate change and sustainability, think of it just to be cases wherein there's a lot of data available and it's just sitting there. Not a lot of folks know what to do with it So let's look at climate modeling. Climate modeling has computational simulations to study climate systems. It helps predict long-term climate trends and impacts. A very intuitive use case is the depletion of the Amazon rainforest. With data, if you're able to visually map it, you'll see over the years the trend of The decline in the rainforest area. That is a use case wherein you see long-term effect of climate change, along with what is the cause of it. We are killing the lungs of the earth.

Speaker 1: And if we are not actually measuring it on a day-to-day basis, we think, oh, it's still there. There are so many trees being planted. But is it still impacting the combined effect that a huge rainforest has or the ecosystem that dies along with the rainforest being um Broken down into no. So climate modeling is what helps us predict long-term climate trends and impacts. A few of the key Python libraries around climate modeling Are NetCDF4 , XRA, and Climate Learn. So Climate Learn is a very good library which has a lot of machine learning models for climate predictions. The the parallel I always suggest folks is uh just like in SK Learn, we there are a lot of um

Speaker 1: inbuilt ML models that you can import, modify, fine-tune the parameters and use it for your use cases in your business activities in your personal projects. In case of climate modeling There are specific libraries which know what weights help improve the predictions that are there for climate use cases. So a lot of libraries are already out there. It is just you who has to go and like look at what parameters I can fine-tune to find the right predictions. Like for example, uh NetCTF is very good for handling multidimensional climate data. With climate data, think of climate data like It is a real-time series data. You're collecting it every instance and it is multidimensional. There are multiple ways of collecting the same data.

Speaker 1: So you'll have different sources of data, you'll have multiple dimensions in the same data. All of that, it is a complex problem to solve when you have multiple dimensions, multiple sources of data, along with the big data scale of the data that is being collected, you need the best engineering techniques in order to extract the most relevant information out of it. If I show you a simple example of just analyzing climate data, so you can download a net CDF file for temperature data. I've included the GitHub repo which has some data for you to play around. And if you just load the s if you just load the data set, calculate the temperature anonymity that is there, and a 10-year moving average, and plot it

Speaker 1: Now that you see it, it is very straightforward in Python. You will be able to have a graph which tells you what the temperature anomalies have been on the sample data that we worked upon And this, along with other trends that you're studying, say rainfall uh impact on a particular area, this has a lot of business use cases as well. If you're in an industry, um involving anything to do with water, with electricity usages, impact areas, would there be load shedding at some areas? Are there possibilities of any natural calamity due to rainfall patterns which will impact a energy company in their day-to-day operations? So, from a monetary standpoint as well, these have a lot of use cases in real-world business scenarios, money-making companies

Speaker 1: as well. But on the side, there are a lot of open source projects that can be used in order to study these things so that you're able to contribute better. Another um Interesting use cases, predicting climate scenarios with AI. We use AI for a number of use cases in order to make them useful and easy for us to use. But why don't we use AI for actually something that matters to each and every one of us on an everyday basis? If you look at climate forecasting, this is a very important use case, is used by a ton of Different industries and organizations to make the most of data. Regression models for temperature predictions. So, temperature prediction one is very important whenever you're dealing with a climate scenario

Speaker 1: But climb uh temperature prediction is there are um like business units inside big tech companies of electronics, uh like fan-making companies, AC making companies, they need to Parallel this with the sales that they'll have for the year and then backtrack it to the manufacturing unit of how they will produce things in order to meet the demand and supply model So, different regression models for temperature prediction is very useful for climate forecasting for any kind of use cases. And time series forecasting using profit is very effective. When you're working with time series data, there are different industries like the finance industry, which makes use a lot of time series data to make decisions.

Speaker 1: Similarly, in the climate forecasting space, time series data is what helps you to predict things, forecast things ahead of time. And with time series data, you're also dealing with the issue of Velocity of data with which it is being generated. All of that, a good use case, is profit. Environmental monitoring with Python Um, if we look at real-time tackling of environmental changes, which includes air pollution, deforestation, ocean health. There are a ton of Python libraries which help you make use of tracking these environmental changes and then making decisions based on it So even if you are a no-voice in the environmental sciences field,

Speaker 1: but you're good with technology And you understand basic concepts that are associated with environmental sciences. In that case, you can make use of These libraries to give you a kickstart, it'll help you solve your own cold start problem of not knowing about the industry but still being able to understand data, play with it, and make the most of it So a few libraries like GeoPandas. So this has very good geospatial data analysis. Another way is you could use Normal pandas that we use for other use cases. But GeoPandas has special functions, special implementations, which will help you save a lot of time in doing a lot of things That way, you're saving a lot of time. It is helping you understand the data better

Speaker 1: and you're able to contribute better. Rasterio is a satellite image processing library. With satellites, there's so much data that is collected and the same data can be interpreted for various scenarios. Now, all of that data that a satellite captures, you need a way in order to actually identify if this satellite image can help you identify different terrain pattern shifts. If it can help you understand better about the climate of a particular position, of a particular location on the map, and also just understanding the multidimensional data you're getting with satellite images. And Earth Engine API is the Google Earth Engine for environmental data. So a lot of folks when they're talking uh or when they are starting about environmental changes

Speaker 1: Uh go on with Earth Engine API. It has a very good documentation. It also has a lot of use cases which you can play around and then Use it with real-time Google Earth scenario and build small apps which will help you in your day-to-day use cases. I know a few folks who are using it for as simple as the plants that they have at home. A lot of us are now taking pride in being plant parents. There's so much happening. And we want to keep our plants alive and healthy, right? So small apps like these, there are a lot of libraries out there. And with the help of a ton of AI coding agent tools that are out there, you can create a quick app for your use case. So you have like a custom way of identifying your plant needs

Speaker 1: And then this expands into a much much wider application as well. Say you live in an area which Uh say you're like 10-15 minutes away from a farmland. You can help out your local farmer by building a small app for them. You are making use of your technology skills in the environment space. You're monitoring um Different pollutants level pollutant levels in the air and helping your fellow farmer who's providing you like good vegetables and fruits in the farmer's market. Let's dive a little deeper into Rastereo, which is the satellite image processing library. So it is a GIS, a geographic information system which uses Geo TIFF and other formats to organize and store gridded or rastered datasets.

Speaker 1: So with Rasterio, it reads and writes these formats and provides a Python API based on ND arrays So, making use of our existing knowledge of NDRS and Python, but adding the complexity of handling satellite images The very first thing is that raster data represents spatially continuous phenomena. So you'll have elevation, temperature, satellite imagery. It is composed of grid cells, which like pixel values, which represent real-world data. So each pixel of a satellite image that you get is going to have a ton of information about real life. And there are a lot of common formats. With satellite images, you'll find satellite image formats to be in Net CDF, HDF, GOTIFF, and

Speaker 1: Then Rastereo helps you make use of it. Why do I suggest Rastereo for satellite image data processing? It simplifies reading, writing, and processing raster datasets in Python It supports different geospatial formats. The challenging part with climate science is that you cannot dictate the way you want your data format to be. Satellites are expensive The way you're receiving data is the way you have to consume it in your pipeline. You cannot modify the model or the source of getting data. You have to work with the satellites that are already up there. And the way in the format they are sending data to you too. So Rasterio is a good way in which you're able to handle different kinds of data formats you receive.

Speaker 1: It is very efficient in handling of large raster files and it has integration with NumPy for analysis. It is as easy as pip install rastereo to install it. And if we look at a quick example, uh if you have to open a sample which is in uh TIF format, you use it to open and then using uh NumPy array you directly have it in your image data. Um and if you when you have satellite data there's also a lot of brightness uh issues that you might face because a lot of it it has multi-dimension there are going to be multiple colors you want to highlight a few colors over the others All of that can be done very well as well.

Speaker 1: To visually show you the kind of data set that I pictured is like this: it is along the coast. There's like ocean on the one end and like habitation on the other side. Then I'm trying to crop raster data for that. There's something called mask inside it, so you don't have to worry about that image processing And a GeoJSON-like DIT , which implements the Python geointerface protocol, it defines the area you are interested in. So, and ROI is the region of interest. So, there's a region of interest that I've defined of what ROI means, and that is what I want the mask to be applied to. Internally, another file that I have looks like this, wherein it is a polygon of

Speaker 1: the four coordinates that I want of the mask data. Now the cropped result looks like this. For context of the initial photo that I showed you to this, out of the whole set, this is the part I am interested in So you see a satellite sends you a lot of information. Now I'm focusing on the farmland around a particular area and then I can perform more analysis on it based on If I know the flower that is growing, that is why it is showing up as red, I can perform more analysis on it. So satellite data is huge. You need to like mask and find out your own things So if you look at it, this is like just a few lines of code and you're able to extract out important information right from your satellite data. This is an advantage of Rastereo library for you directly.

Speaker 1: Similarly, there's a lot of application of Python in sustainability projects, right from forecasting solar and wind energy production to optimizing resource usage for sustainability products. And the relevant Python libraries for it are PVlib and skln for any kind of prediction models that are there. I'll quickly go over what Pv lib looks like. It is basically a community-developed toolbox which provides a set of functions and classes. That simulates the performance of uh photovoltic energy systems and related tasks. Um, I'm running short of time, that's why I'm rushing it a bit But uh the primary focus of PV lib is uh for handling time series solar resource data.

Speaker 1: A lot of research goes into studying the solar energy patterns that we receive on Earth And that helps us do a lot of predictions for climate science. The functions in PWLIP provide a uniform interface for accessing data across many formats. The Pvlibs has. io tools which returns two objects which is a pandas data frame plus a metadata dictionary. So you see the original data you would get would look a lot complex. Once you are importing it through Pvlib, then it is your regular pandas and uh some metadata So then you can do it very well. So this is a sample code I wanted to show all of you is to generate a sunpar diagram to show the solar position over a year.

Speaker 1: You can look at these during the slides later on, but the polar plot of what the Arly solar and Zenith and Aesmith locations look like looks like this. So, this is used by climate science research to study sun patterns over different years. And even though this looks a little complex, but It was written using just those like about 20 lines of codes, the ones on the two slides that I had. That's it. So Something like this is also achievable because you're making use of Pv Lib, which is already handling the complexity of understanding solar image data directly, and you don't have to worry about that There's also another field of study inside climate science

Speaker 1: of climate risk, and there's a lot of analysis you can do with Python. You can quantify climate risks. One is saying that yeah deforestation is bad, but till the time you don't have a quantifiable number to it. How would you take an action on it? How would you measure the impact your action is doing? So, for example, something as simple as carbon-aware computing All of us write code. At the end it re it has a carbon footprint. So how do you make your code to be carbon carbon aware? A lot of companies are shifting towards carbon-aware computing as well, so that they're able to meet the clean energy standards set by the government. Also, real-time air quality pipelines.

Speaker 1: So if you have a simple sensor which is just recording the air quality which a lot of us would have. Even something as simple as air purifiers in our house. If you buy an external one, you can use that data and do predictions and analysis on the air quality you have in the surrounding area You can do a lot of geospatial analysis. So, for example, deforestation in Amazon. So, you can make use of different data sets. And if you see here, I'm making use of Geopandas and Shapely. So these are again climate-specific libraries uh which are open source and python. So things like this make it very easy for you to handle data, understand data, and then create um different action items out of the data.

Speaker 1: A quick call to action before I come towards the end of my talk is to contribute to open source climate initiatives. As you saw, there are a ton of libraries. All of these libraries are open source. If you make use of these libraries to study the data that's readily out there, you as a developer would find in use cases which can also be implemented. So I would say go ahead, submit a PR and uh Help the community by improving these open source libraries for climate initiatives. And use Python for environmental data analysis analysis. If you make use of R, uh I would say give Python a try. Uh there are a ton of open source libraries which make use of a lot of models, which then

Speaker 1: indeed Are very very useful for the community as a whole as a lot of technology folks are entering uh completely into environmental spaces who need not have environmental sciences as a background. So Using Python helps also bring advances in the regular tech ecosystem to research in climate science and advocate for sustainability in tech Something as simple as being aware of the carbon footprint of the code that you are running is a great place to start. And you can also Uh this is something that I did personally and that is what got me intrigued into climate science on a day-to-day basis for me as a developer who doesn't work directly in climate science Is I started checking the carbon uh footprint of my code and running it.

Speaker 1: And then when I optimized a few of my loops, the way I was Making my code efficient, but that also reduced the carbon footprint of my code. So that is an interesting insight that you're not only optimizing your resource usage, but you're also making it more efficient from a climate standpoint. And um I showed it in one of the slides. It is as simple as this. So Eco2 AI has it in built. So you can do carbon-aware computing With a few lines of code and see how your code base has progressed over time. That will be another interesting metric you can like talk about at work if your code is improving even from the carbon footprint standpoint So yes, uh those are three calls to action from my end. Uh thank you.

Speaker 1: Um and I'm open to questions.

Speaker 2: All right, let's go to round ahead. And we do, we've got a couple minutes for questions.

Speaker 3: Thank you, that was very interesting.

Speaker 1: Thank you.

Speaker 3: You mentioned getting involved. Do you have recommendations for online spaces or communities where those people who work in this with Python hang out?

Speaker 1: So there's not a lot of say meetups that happen, but I've seen the Eco2 AI library to be very active. The one that helps with carbon-aware computing, as well as PBlib. So on the forums you'll see that if you go to a few of the closed PRs, you'll see a lot of discussions and generally those folks are similar The ones who are active are similar across and then maybe you could connect with them over Mastodon or Twitter or like find which platform they are active on. That would be a great place to see start.

Speaker 2: I have a question.

Speaker 1: Yeah.

Speaker 2: First of all, I really appreciate this talk. My background's in GIS, so it's fun, refresher. I I was kind of You may not have an answer to this question, but I'm kind of curious where what's involved in the preparation of those data sets and like making them available. Um and how you know data quality might be impacted by, I don't know, a tremendous amount of defunding for environmental research in some countries in the world.

Speaker 1: Yeah. Uh so the open source data sets that I have seen across, like the Google Earth dataset, I think that is pretty well maintained. They keep updating it. Um and that is directly Accessible as an API. What I also do with Google Earth datasets is I apply a filter right before I download the dataset So that way if I'm say focusing upon forestation and deforestation, then I just look at those data because that's again satellite data and I would say not want um like Other dimensions of the data to get downloaded. That is one way. Overall, about data equality, I think that is a very wide topic based on um Way different organizations are handling it.

Speaker 1: I haven't seen a lot of documentation around it. It's a good point. Like I think as more folks come into it, raise questions, that is something Dataset providers would look into even satellite companies who are providing a lot of these open source data sets. I personally haven't seen that being documented anywhere. So yeah Thank you.

Speaker 2: Okay, all right. One last or output re question and see if I can sorry

Speaker 4: My question has to do with R and Python. So I work with a lot of people who do environmental science work in R. And what would be the best argument you could give to have them try and Python a little bit.

Speaker 1: Oh, the very first one would be to watch a recording of my talk right now. The second one would be um The support that Python has, the ecosystem around Python is much more stronger, uh, I feel, as compared to R, because R is mainly used by researchers and scientists, whereas Python is used by the wider tech community as well. So any updates to Python as a language or improvements in it also by default will get reflected in the existing applications written uh with Python. So I think The continuous improvement in Python plus the community support around Python. Like it it, in my opinion, it should be easier for somebody to start with Python and keep improving on it as compared to R because of the community support

Speaker 1: Yes, thank you. All right.

Speaker 2: Thank you very much. And uh that concludes the talks in this room for today.