How do debug tool bars for web applications work?

Hopefully you can understand me. I've came over with a flu on the flight over. I'm a lot better right now than I was earlier there. So web application debug toolbars exist for various web frameworks. Solutions exist for frameworks included in Python, as well as Ruby, PHP, and port. Now I have no idea where that idea first originated, but for Python at least, the first example I know of is the Django Debugtuba. Although I'm sure if we have some Zote fanatic in here, he'll jump up and down and say, no, no, you're wrong. We had it years earlier But but in my research I could only find Plone and that was only implemented in more recent times. So I'll say Django wins. As to what they are, the actual web application debug toolbar is a piece of code which is embedded within your web application. Once enabled, when you then access your web application from a browser, a toolbar or sidebar

will be presented along with the normal page response for your web application for the URL which was visited. Through this toolbar you can then access information about the specific web request which was just made. A range of information about the requests may be presented, generally within separate panels selectable from the toolbar. This can include performance data such as how long the request took to execute, the CPU resources consumed, or other operating system parameters. The request headers received by the web application from the browser, as well as the response headers that were returned by the web application, would also generally be available. As with details of any request parameters and cookies, an arrangement framework specific information could also be available. This may include session information derived from cookies, the URL route

match And the name of the actual handler function which processed the request. Even more detailed information about work done within the context of a request for database queries and template rendering might also be available. For the case of the Django debug toolbar, as I've been showing, it provides a set of quick setup instructions. First, you need to get the Django static files feature working. After having done that, add the Django debug toolbar itself to the list of installed applications in the Django settings module. You'll also need to ensure that Django is running in debug mode. Is now just a seal member of running up the Django development server near all ready to go. Or at least you hope so In the quick setup instruction for Django debug toolbar, you'll find this very nice, very nice and cheery message warning you of possible impending doom.

So your luck may vary, but this obviously raises the question of what is Django debug toolbar actually doing on startup that could cause such a mess. including the possibility of circular imports and errors telling you that you've configured things incorrectly. The answer to this lies in the note in the documentation preceding this warning. And that is that Django Debug Toolbar is using an evil little hack to bootstrap itself. Specifically, Django Debug Toolbar wants to monkey patch additional configuration into the Django settings module and URL views. To do this, it is hijacking the validating of database models when using the development server to run code which has nothing to do with the actual database. Now I have got wind of

some changes in Django 1. 7 related to application initialization. So it is possible this may no longer apply in Django 1. 7. But what do you do if what do you do if the server does blow up on you because of this? Also, since this evil workaround is dependent on using the Django development server, how can you use the Django debug toolbar with an alternate web server such as ModWisby? For this eventuality, Django's debug toolbar provides you with additional explicit setup instructions as well. The first step in that is that you need to tell it in the Django settings module to not do its evil punk monkey patching. Now we already added the Django debug tool out to the list of installed applications. This is still required, otherwise Django will not know where to find the static file assets.

and template files that Django debug toolbar view handlers require, leave it out and you will immediately get a template does not exist exemption from your web application when access. And we don't want that. Next we need to change the Django URLs configuration. This is to insert special Django debug toolbar view handles to process any requests which are received under the underscore debug sub-URL of the Django application. Wisdom is that Django debug toolbar should never be used on the a production system. So this is also gated by checking the Django debug flag This is of a course assuming you know why you should not be running Django in production with the debug flag enabled. Although I was a bit worried about the talks and security talk the comment and the security talk suggesting that a lot of people do, which is

don't worry. So after adding in the URLs for the Django debug toolbar, we need to add in a middleware class. This is a quite important part of the Django debug toolbar code, and its placement in the middleware classes list is also very important. It must be placed as close to the start of the middleware class list as possible, but after any middleware that encodes the response, such as GZIP middleware. Finally, although you should only be running the Django debug toolboard in a development environment, you still need to protect who can access it. The last thing you want is someone being able to access it over the conference network while you're watching my talk here. Now the documentation does say that if you don't set this, it will default to 12701 and colon colon

one, which is the IP V6. But on macOS at list, I found I've also needed to add in FE80 column column one style of address. I I I don't really understand the difference to this stupid these stupid IPVs extra. I think I have to do it when I've got my VPN going or something. If someone can tell me, please do. So back to the Django development server, and we should be all good again. The evil hack of bootstrapping via Django database models. Verification is gone, as should those risks warned about of module import cycles and strange exceptions. What though if you want to use a real production grade whiskey server, such as Mod Whiskey? In this example I will make use of ModWhiskey Express. You start off by installing Mod Whiskey by running pip install mode

If using Mac OS, it should all install with no issues. If you're on Linux, then you may find you first have to install the Apache development package for your specific variant of Linux, because that isn't generally installed by default. As long as you've got your package package in, that should just be all and you've got it mod dod whiskey then installed. So once we install, we can now test that the package was installed correctly and that Apache can start up properly For this we run mod WISGIEXPRESS command and give it the start server command. Normally we would provide an additional argument of the Whiskey script file, but we leave that off here because I just want to test the installation worked. So in a separate window we can then open up the browser on the URL for the instance of Achi Mod Whiskey we have just started. What we should get is a helpful splash screens

so you know it is working okay. As I already mentioned, to run it with an actual WISGI application at this point, you can supply the path to the Whiskey script file. You can also use the minus-minus help option to get information on all of the command line options in case you want to change the the listener port or specify a directory full of static assets. If if you want some high level uh information on ModWizGESPRS because you've never seen it before, go look at the PyPy, uh PyPI, sorry, I'll get killed if I say PyPy PyPI and look up modWSGI there and you'll find some information there. Now because we're using Django though, rather than pointing ModWisky Express direct at a Whiskey script file, it's better to integrate ModWhiskey with Django itself. That way when we run ModWISG Express it will also know about the Django specific requirements for static files.

What we therefore do is add modwisky. server into the Django installed applications list of the Django settings file. Django does these days have the static files application. However, if using a Whiskey server integrated with a real web server, such as the case with Mod Whiskey, it's really much better to use that real web server We therefore set up a location for the static files to reside and run the Django Collect Static Management commands to collect together all the static files from the different installed applications specified in the Django settings module, including Django debug toolbar. Whatever you do though, do not remove Django control static files from installed apps. This is because the Django debug toolbar code doesn't want to trust the 9-0

on doing. And will cause everything to fail if it doesn't think that the static files app will be serving up those files of its its own. So we've got to trick it a little bit and just not take it away. Now finally here is the cool part. To actually run Mod Whiskey with Django, and this is going to run up a full Apache, we simply run the run Mod Whiskey management command. So this will automatically generate an Apache configuration for you on the fly, dynamically, for this instance. This will include mounting the Django application, setting up static files, and so on. Because we're in development environment, we can even enable automatic code reloading for good measure. And ModWhiske Express will worry about all the details of getting that to work as well.

So here's a bonus for this talk. You get to learn about ModWhiskey Express, but that's not what we're here for. So the apple from running for invoking the modern modules command will give you the URL to then access the site as well as the location of the Apache configuration, which was generated in case you do want to poke around and learn from what I've done in generating it. And the Apache error log in case something does go wrong and you you need to look at the application logs to track down any details or exceptions that were logged in. Most important to what this talk is about, the Django debug toolbar will now also work And that's because we followed those explicit in setup instructions. So what we learned from the explicit configuration steps requires that Django debug toolbar is dependent on a number of key things. In order to work, there are a bunch of static file assets

with HTML, CSS, JavaScript, and images. It registers a series of view handlers which are used to produce data for each toolbar panel. which in turn potentially require page templates to render responses. Finally it requires a Django middleware to be installed. Is this last thing, the Django debug middleware, which is the key to Django debug toolbar working, is the middleware which allows it to hook into the different phases of handling a request in Django? For the Django debug toolbar, it hooks into the process request, process view, and process response phases for Django request handling. This allows it to perform actions at the start of the request prior to the target view handler being invoked to handle a request, and finally allows it to perform actions at the end of the request as well as actually manipulate the response returned from a request.

Note that this is for any request, not just requests made against the Django debug toolbar's own view handlers. The part which is initially of most interest in the middleware occurs in the process response phase. It is in this phase where the toolbar itself is injected into the response that comes back for a request. Before that can be done though, it is necessary to first determine whether the page is in fact HTML. and that it makes sense to insert it. There is no point trying to insert the toolbar into images or a plain text file. We're only interested in HTML. We also want to avoid compressed data or streamed responses.

And this is where it was important to where the placement this was. If we put it after compression has been done, then this won't actually do anything. If it is okay to be modifying the response, then the rendered HTML corresponding to the toolbar itself will be injected into the original HTML response for the request. This injection of the toolbar HTML is done just prior to the existing closing body tag. The combination of that HTML as well as the style sheets and JavaScript code which is in turn which it in turn loads when rendered in the browser results in the sidebar appearing alongside the content of the original request As shown before, the sidebar under the type under the time category will show how long the request took to be handled, or at least up until the time that the toolbar

HTML is inserted into the response When we click on the time category on the sidebar though, only then will the debug toolbar panel be displayed with more detailed timing information related to the request. Although the sidebar itself came back with the original request, the panel displayed as the result of a separate Ajax request made as a result of clicking in that category in the toolbar. Given it is a separate request to get the additional data, where is the data stored and how is it identified? The answer is that any J generated from a request is saved away at the time of the request. In the toolbar HTML that comes back with a response, it has a storage ID field which identifies the item in the cache for the original request. When drilling down, a request is made with that storage ID

along with the ID of the panel corresponding to the category selected on the sidebar, which should then display the full data from the original request. Now we can't save away the full details on every request ever made. So any data stored in the case must have a finite lifetime. By default, the number of requests for which data is retained is 10. This is actually quite a big limitation as to as what it means is that the debug toolbar is not very useful when you're trying to use it on an application which is any traffic beyond just that which you are manually creating with your browser. This is because any data on the request you're interested in would have been thrown away by the time you draw down to it otherwise.

As well as potentially losing data if there are other users concurrently using the same web application. The data can be purged if you are working on a page which is a heavy JavaScript component, which makes additional AJAX requests after the initial page response has been required. These Ajax requests are a problem in themselves. And this is because the Django this this is because Django debug toolbox only provides you the ability to debug the primary HTML request. There is no ability to get details of any of those subsequent Ajax requests made from that page. The limitation on AJAX requests is that the toolbar is displayed in line with the current request only. To better understand what I mean by this, it actually help helps to look at how the debug toolbar for the Pyramid Web Framework works in comparison to that for Django.

In the pyramid debug toolbar, you do not get a sidebar for the current request. Instead you're going to get an icon indicating that the debug toolbar is operating. When the icon is clicked on it, it then goes off to completely separate window. So what the pyramid Django, the Pyramid Debug Toolbar does is not show any details in line to the current request. Instead, you always access details requests from a separate window which shows a list of the last 20 requests. When clicking on the inline icon within the initial page response, it will push you across to the details of that specific request in the separate window. Important is that since you now see all recent requests, you can also see any separate Ajax requests served by the web application for other assets associated with the page.

Having jumped a look at Pyramid, let's look at the history of debug toolbars for Python. As I already said, as far as I know, it started out with the Django debug toolbar. This was summarily copied in creating the Flask debug toolbar. From the Flask debug toolbar then came further copies tailored for bottle, web. py and pyramid. As far as I know, with the exception of Pyramid, they all simply copied the inline style of the debug toolbar as originally implemented by Django. Only Pyramid branched out and tried to address tracking of Ajax requests. The Pyramid Debug Toolbar was therefore an improvement, but in my mind they all have one big failing. That is that they are all distinct implementations. That is that the Django implementation is intrinsically wedded to the Django middleware infrastructure, with implementations of panels implemented as Django view handlers using Django templates.

Flask uses its own middleware and Ginger2, and so on for the others, with each being ported to their specific way of doing things. The opportunity missed here as far as I'm concerned is that all these different frameworks at their lowest levels implement the WISGI specification. Yes, the debug toolbars do offer some panels which are framework specific, but the core functionality they implement, they can be implemented as a WISGI middleware. There's therefore been quite a lot of effort wasted in creating these separate implementations, whereas if it was done at the Whisky level, there could be one debug toolbar to rule them all, which worked everywhere. So imagine it for a moment, one debug toolbar that was best of breed, when someone enhanced it due to what they found when using a specific web framework, all other web

frameworks could benefit as well. Now wouldn't that be wonderful Unfortunately, Python web communities have a history of not cooperating, each believing their framework is the only true way. And Django, unfortunately, is no different. So what inevitably happens, therefore, is that they build things which will only work with their specific framework. Someone annoying, so I think I believe we can do better. So since I think it was such a great idea, let's have a look at implementing the basic mechanisms of the debug toolbar code at the WISGI level using a WISGI middleware. In particular, let's look at how we can manage to inject a toolbar HTML code into the response for a request. If we can work out how to do that, then we're well on our way. This is where we can learn from the Django middleware code I showed earlier.

It may not be obvious, but there are a couple of key things which are going on here. The first is that the HTML code is being inserted just prior to the closing body tag. And the second is that the content length for the response is being updated. In other words, it needs to operate on the complete response. This presents us with a bit of a problem, and that is that in Whiskey applications, the response doesn't need to come back as one byte string, but as a sequence of byte strings. As far as inserting something at the end of the body, we would just have look, we could just look at each separate byte string until we find the closing tag for the body and then insert it. The issue is though that we also need to update the content link And if we can't know the length of what is being inserted until it is inserted, we can't adjust the content length at the point the headers are sent and still stream the response.

Now we could buffer all the response content in the middleware and then operate on the complete response content, then send it on after adjusting the content length Technically though you're not meant to do this in WISCI middleware according to the WISP specification. Plus it would cause applications which slowly stream data over time to break as nothing would be sent until all the content was generated. The actual size of the response content could also be a problem if, for example, one was streaming a very large file that you couldn't actually fit into memory. If having to insert it at the end of the body is such a problem, why are we inserting it at that point anyway? The reason is that part of what is being inserted consists of JavaScript code And reason is that this should always be inserted at the end of body so as not to delay page loading

by doing it earlier in the page. That may well have been the case once upon a time, but but are there now better ways of doing it which would allow us to instead insert it at the start of the response content instead? With the modern browsers we use today, rather than needing to defer things to the end of the body, we can instead load the JavaScript from the head element of the page, marking that it should be loaded asynchronously rather than in line to the request. By being asynchronous, it will not delay the loading of the remainder of the page. When the JavaScript is loaded, we then need to trigger an action to go back and modify the DOM for the loaded page to insert what we wanted there in the first place We have to be careful in doing this though, as we likely need to ensure that the page has actually been completely loaded.

What we can do therefore is trigger any action to modify the DOM off the document being ready in the JavaScript code we are loading. How does this help though? The reason is that we no longer have to buffer up the complete response in order to be able to insert into the end of the body. At worst we have to buffer up to the start of the body. But the amount we have to buffer up is a lot less and so doesn't present an issue with memory. It is also highly unlikely, even with the streaming response, that the head element would be sent slowly in parts. Instead the head would be sent in one go up front and it would only be the body which was sent out slowly. We can handle insertion, but what about the UI itself? The problem with existing toolbars is that the UI is intrinsically linked to the URL routing, request handle, and page template systems of those specific frameworks.

My suggested solution for this is only to have a REST API served up via the WISGI middleware for accessing any data stored about requests. The UI would then be a JavaScript interface rendered in the client by using a REST API in the whiskey middleware to get access to data. We then separated from the specifics of particular web frameworks way of doing things. So you thought in coming to this talk that you would learn about how debug toolbars work. Hopefully you have in part. But what I really want to pitch is the idea that our current debug toolbar implementations could do a big overhaul. In particular, there is a great opportunity to come up with a debug toolbar which could work with any Whiskey

framework. I think we could build a better ecosystem and make our job of debugging Python web applications a lot easier. And more importantly Diverse sets of people from different webworks can contribute to this so that we all get benefit from it. So one thing I'd like I personally like to do is integrate into the debug toolbar features to Allow to view additional performance monitoring data. This is where I'm coming from. So as an example, I implemented monitoring for uh Mod Whiskey, but which used the New Relic platform plugin system to do visualization It is free to use that feature in New Elite, but when I announced it on the MogWiski mailing list, I got zero users. Not very successful. People objected to it not being able to be self-hosted. What I don't want to have to do though is develop a separate UI for half a dozen different framework-specific

debug toolbars to provide the visualization. I've also been working on a side project related to the Neuralic Python agent project I work on as my day job. And Neuralic is a wonderful tool for web application performance monitoring, and you really all should be using it. But what I would like to do is bring some aspects of what we do with New Relic into a local developer mode, integrated into a debug toolbar environment. Because we have a production-capable way of monitoring your web application. So why not try and make use of it in a development environment as well? So in considering this idea, though there is actually in considering this idea though there's actually a bit of a twist. And the problem of how to insert HTML into responses is something that is already solved by the New Relic Python agent across frameworks.

As we use that exact technique in order to implement end user monitoring. The thought therefore is that since we do not do that already, if having a local developer mode for the new Relic Python agent, why not use it as the base for a debug toolbar as well? We could even have a variant of the debug toolbar you could use when using New Relic in production. So there are no other third-party tools which can be brought under the same umbrella. So Simon Wilson of Eventbrite demonstrated at PyCon AU this year, a debug toolbar like tool called Tiki Bar. This actually embodies many of the same qualities I want to achieve in pursuing this whole idea. This includes the idea I have that one could come up with a limited toolbar which works for production systems. Tiki

Bar is though again bound specifically to Django and currently event by its own software, which is exactly which is exactly what I don't want to see. So I don't know yet if I will be allowed to make it available, but what I'm pursuing is a local developer mode for the Neuralink Python agent. It will give you a subset of its functionality for use in the development environment, using its mechanism for HTML injection to implement a debug toolboard. Panels for this debug toolbar system can then provide access to the new relic data as well as other data that debug toolbars presently provide which are framework specific. On top of that, I want a mini toolbar which can deliver up safe information in a production environment. So my reason for doing all this is because I hate to see all that wasted wasted effort being done by different developers for each different web framework or in-house system. This doesn't make a great deal of sense to me.

So hopefully all very altruistic, but I I do have to admit I have uh have to admit there is a bit of a selfish aspect in this as well. And that is what I that is that I think that what New Relic does is awesome. And the debug toolbar provides possibly a way for me of putting your relic in front of you in a local environment as well. So you get an idea of what we can do and how it can then be useful in your production systems. Hopefully you won't concentrate too much on that stuff and you'll say, hey, this is just a great idea anyway. So certainly I would love it if you go and try New Relic, but if you aren't interested in that at least, but you are interested in improving on the existing web application debug toolbars available, then come and talk to me. Tell me what your own thoughts are and ideas and features are that you want to see in this sort of thing. I'll be around for the sprints and we'll be working on this or on Mod Whiskey.

So if you come and talk to me about it then And also finally, if you are looking for an actual paid job and you sort of know a bit about New Relic and you're interested in that sort of stuff, we are actually looking for someone who is strong in Python testing. So also come and talk to me if you're interested in that as well. well. The job for that would be in Portland. Uh no remote, sorry. And that is all