Hidden Gems in WebPageTest with Tim Kadlec

All right. Yeah I know we got to get some pretty [Inaudible 00:08] mute going on here right. We got to figure that out. I think Geno brought that up last time but I warned y'all. I like country music I shouldn't be choosing it. That's going to have to go up to the rest of the crew I think to think of something on side otherwise you're all going to have like Brooks and Dunn playing or something like that at Neil Diamond. Yes. Neil Diamond is the other option right.

Hey everybody thanks for the show. Thanks. Welcome back. Those of you who caught the first one as well. We're going to be doing another one today another one where we kind of dive into a mess inside a web page test. It's going to be a little different last time. We went like an hour and a half. It was a nice long session and the funny thing is it felt like it wasn't. We could have on longer. This time though I do have to [Inaudible 00:53] audio choppy I adjusted the stream settings and everything for this. You know one would think. You know. Yeah. That's fun. Well, let me tweak one thing. Can I? Can. I not do it while it's actually running? I might not be able to do like any sort of tweaking to the actual quality of the audio or audio Funkiness here while it's going.

All right. We'll sort that out at some point. Anyway, hopefully, it's not choppy enough to make it completely useless. If it is obnoxious Let me know we can kind of end and I can try to tweak settings and come back. Twitch is supposed to I optimize the thing before we started streaming. You know to try and bring the bit rate down or play with the audio just to whatever. So fun stuff  

Today we have to keep it a little shorter because I do have to Catchpoint is doing a SRT event. Sure Sergei, I'll just repeat myself twice. Catchpoint is doing an SRT event. See I just did it and I got to moderator and panel. So I do have to keep this one probably to 30 40 minutes or so. I do want to dive in and given that also I felt like it probably wasn't fair to do like an actual live audit in 30 to 40 minutes. Just because I don't think we could do it justice. So instead I want to show some of those sort of a couple specific examples. I was looking at recently that required digging into some of the more hidden or esoteric or advanced however you want to call it features of webpage test. You know some of you vets here in the chat room may know all of these things. Some of you may know some of them. Some of them may be new to one. Hopefully, everybody gets at least one thing out of this that they didn't know before. But this is again in the context of a couple of actual tests that ran. I need to dig in a little bit more beyond what you typically do with some of the web page test stuff.  

So to do that let's take a look at the first one. So first stuff is actually this synthetic monitoring page for catchpoint. Tell catchpoint I was doing this. I'm being kind we're kind we're helping everybody here. That's what might actually do legitimately one of the things I love about the perf community like how helpful everybody is for stuff like you don't see the. Yeah it's just nice. Anyway, the synthetic monitoring page so the page was actually one that Pat and I [Inaudible 03:49] when we were playing around the new core web vitals diagnostics page and we ran a couple of tests on this URL just to kind of as we were experimenting with getting that page in place and some of the features and functionality there and one of the things that jumped out when we did this was its layout shift that's occurring.

So if I do a refresh there you should be able to see right like there's some significant shifting happening here. So this kind of came out in the test itself. So if we go to webpage test I grabbed one this morning again just to make sure nothing. That's great. Nothing changed significantly here. So this was a test that I ran on Chrome again kind of using sort of a 4G network just slightly slower than cable. If we go through and click on I'll bump this up a little bit for you all. There you go. Tim learns computers.

Will click on accumulative load shift which takes us to the cumulative layout shift of that diagnostic page. Oh before we do that one thing that was added just in the last week or so that I think is pretty sweet on this page is if the largest contentful paint is an image. Some of you might have seen this on Twitter. We talked about it a little bit. If it's an image we now it automatically in the waterfall for you so that you can spot right away when that request is made and where it comes into play which is I think super slick. And then we also show the image itself underneath. So you can quickly see like what is this image that is our as largest contentful paint and what are we doing with that. So in this case, if this big bad round circle that goes behind our hero image which is interesting. You can start the question Does it really need to be there. That kind of thing.  Certainly not an example of my opinion of largest contentful paint being the most important thing that's being painted on the screen in this case.

Anyway going down to the CL section that was beside point, I get distracted. On these thumbnails, if you remember well this actually show up this time nice all right. So I was going to say on these thumbnails if you remember you get the actual and this is actually this is a spoiler alert right here is kind of ruining it for me. That's OK. You get this little. This is what you see the image if you hover over it you get the previous frames. You can see what's shifting around. Let me actually pull up a test since this one the timing worked out good. It's actually a decent example that shows the problem we were running into with this though and that you'll run into sometimes when you're doing tests. Let's try this one that’s better.  

OK. So notice how [Inaudible 06:49] time when we hover over these images we're seeing the red blocks but we don't actually see this screenshot changing like we did on the last time. Now you'll have this happens sometimes. You'll be running these tests You'll be trying to dig into SLAs or something like that and the frames don't like you can see that something moved but the frame itself doesn't capture what the element is or at least let you see visually [Inaudible 07:10] on.

So I wanted to explain what's going on there and how you can get around that when you're running into that. So first off the way that this works is if we go to summary and let's look at the JSON result. First up for the JSON result by the way these are huge. Has anybody ever looked at the raw JSON size for these things? Let's just do a refresh. Just to give you an idea. So. Oh, it's coming in. This is the four. I think there's. Like 10 megabytes or so at least. It's the Biggest on file so it's a lot to process. There's a lot of information literally everything we capture gets put in here. This is what you get with the API. This is also where you tend to use this. Sort of as we're rolling things out that we. Figured out how to. Put into the. UI yet. Often it goes in. The JSON first.  

So. This is a matter of file. So what. Are the thing is I do pretty much every time. It. Is if you query strings on I've actually an Alfred. Keyboard Shortcut. Or a snippet I guess yeah they call it a snippet where I type in colon colon and JSON. And it spits out a whole bunch of stuff. So what I'm putting in here is a couple of parameters to turn off different properties. I'm going to hit it and look how with like 10 megabytes to 174K that’s nice, it makes it so much faster. So what I did was like these parameters I put an average equals zero standard equals zero those things say don't include the average run or standard deviation run inside of JSON itself control means don't include all the console stuff.

Yeah, that's one thing people may not even realize I'll see if I show you a later we capture the console elimination whatever is logged out to the consoles reported for the web page does as well lighthouse equals zero don't include lighthouse runs. Repeat [Inaudible 09:11] turn that off and the big one here is weak quests like the request part of the JSON being massive. So turning that to zero as well. So if you were looking at the JSON which you know you might find yourself digging in for some deep-dive stuff or the 3API a lot those parameters can super help get that size down just make it a lot faster. So yeah I found that out when I was building what does my site cost and getting these 2 JSON files on the server and Pat is like you can make getting smaller. And then like it's awesome because I didn't have to pay as much.

Oh. Any link on where to find those parameters. You know what we are improving documentation but we still have a lot of those little hidden things that haven't made their way to the docs yet. And this would be one of them. Yeah we can definitely get those. We got to get them added. So now that we've got the JSON up if we look for the layout shifts object. So this is all information for each of the layout shifts that happen over the course of the page. And so you can see we're collecting a ton of information here. The shift which shift window it occurred what was the score, what's the window score. A bunch of information around the actual nodes that do the shifting. The part that's relevant here to our issue is we're grabbing the coordinates. We can see like the bounding rectangle where it was pretty easily where it gets shifted to we have all those coordinates around the shift itself. And that's what we used to draw those red rectangles here.  

We grab the nearest screenshot from the film strip that we can and we draw those rectangles on them. Now the problem comes in sometimes like this for the nearest screenshot actually doesn't have the visual change or those elements aren't present or at least aren't present in the same state that we were expecting. The reason for this is because when we run tests on desktop agents it default to 10 frames per second. The desktop agents take a slower or lower frame per second rate because if we go higher there can be high observer cost. It can start to impact your performance metrics which we don't want to do. You know mobile devices it's less of an issue because the cameras are kind of built right into the operating system with desktop it's all software-based so it's much more costly to do that.  

However sometimes` you want to have that precision for example when you're doing CLS testing so we can do to get around that whenever you run into something like that that is if we go back to webpagetest and I'm going to add a parameter at the top FPS equals sixty not one of those things that might be documented somewhere. Matt Hobbs is Matt Hobbs in the chat I knew he was talking about coming. He's written about it sure so if we add FPS equals sixty to the end of the page. Now we can drop in our URL I'll just run one test for speed's sake. But what this will do is this will now to force the desk agent to run and capture at 60 frames per second giving us a very granular and precise visual display. So it's great for things like layout shift. When we want to make sure that we are capturing every little visual shift that happens inside of those screenshots.

It’s also helpful for things like start render. So we capture like we report a start render time start render is our synthetic test or point at which we see something first appear on the screen. And so at 10 frames per second, the start render times tend to not be very precise. So for example when I was diving into first contentful paint in Safari and wanted to test you know what is the gap between when pixels actually hit the screen and when you know Safari is reporting first contentful paint I had to bump that up to 60 frames per second so that would make sure that we had very precise standard times yeah.

Hey, Anthony by the way I just I missed seeing everybody in person so it's just so nice watching everybody to see a Josh Berry and Henri I mean it's just fun, Pat I talk to Pat a lot. So now that we've run this again at 60 frames per second if you click through to our accumulative layout shift now we can see a little bit more [Inaudible 13:55 ] we got a little weird. It was weird did weird things or maybe the loading process just looked weird in this particular page load. Don't you love it when things go according to plan? Let's go back to summary and we'll do our film strip view and we're going to go to our 60 frames per second. [Inaudible 14:15] let me open one of these images off.  Did I reload the page or reload the page right. [Inaudible 14:32]. I'm actually not sure why 2.16. All right. Well maybe I didn't actually for some reason it got a little wonky on me here and is capturing screenshots back at the [Inaudible 15:02 ] frame per second rate instead of the 60 that I told it to or wanted to tell it to.

Well that's fun it did some weird stuff in the painting on this capture I guess this is what I get for running one test versus 3 By the way. Let’s highlight the shifts on a film strip here Anthony has brought that up and just if we I mean we see the shift there again but it's not we're not getting that before that I'm hoping to capture there. I'll do the magic television until this moment comes, let's do this. I've done this before so we had something a little funky on that particular test. That's alright, Catchpoint.  This one should be our sixty frames per second one.

So in one of the ways you can tell partly because we've got the precise for CLS but also note this start render time here when it's 60 frames per second. We're able to say it's nine hundred and seventeen milliseconds is our start render time compared to a run where we do 10 frames second. Notice how it falls back to that 800 hundred. It's always going to be a round digit like 800, 900, or something like that when we have start render on a time frame per second captor. So again if you want precision on CLS or you want precision on start render. Bump that frames per second up and you should get a lot more precise measurements there.

All righty. The other one I want to show was not catchpoint but there's another one. So Catchpoint is kind enough to buy us [Inaudible 16:54] around Christmas is like a Christmas gift. I literally joined the day before I got to pick them out. It was perfect timing and I know it's like you know typical like you know I think there's like a whole Silicon Valley hipster thing around [Inaudible 17:05 ] or whatever but they're legitimately ridiculously comfortable. The only problem is like the little ones that they got they're hot. So I'm wearing them in the summer and my feet are getting really sweaty and stuff and is probably TMI. That was TMI. Whatever I wanted to like the cooler version and so the tree runner stuff is supposed to be the cooler light version of it.

So I was looking at these and I noticed on some loads they can take a while for the [Inaudible 17:35] page to just a content take a little bit of time to come in. So I did what any normal person does and I fired webpage test on this. So let's plot test that. One of the test history. That the free accounts do sixty-three frames per second it's built into the testing aides themselves. So yeah it doesn't matter what you sign up whether you're good at let's do this is the one. Yeah. OK. So this is the [Inaudible 18:11] page loaded up on Chrome on an emulated G4 3pipers for summer. I think the tree runner I don't know is there a difference between the two and there is a difference but I think it's the same material. I don't know. I'll let Gina deal with that.

Anyway, for running the test there are a couple of things first off as I've meant I think in the past and probably will mention again I always look at like those gaps and there are big gaps between the first byte, in this case, start render or first contentful paint is significant because there's like a 762 second first byte time and it's not until almost 3 seconds later we get that first contentful paint [Inaudible 18:58 ] we see a big gap dedicates things like blocking resources or maybe client-side AB testing that's hiding that paint. Something like that that and then you see a little bit slow on the largest as well there is another GAP there. But specifically that initial gap on the first contentful paint.

So I click the median run. Go. Do that. Actually here. We're going to switch that. Around. Notice how the default the median runs speed index. Add a query string. Median metric equals first contentful paint and hit enter. that get us to now it's going to base the median run on first contentful paint in the case for first use the same run it did change the Read view though and it clicked through to that run. And sure enough at the start here we see our HTML which has a big chunk of JavaScript execute. And we've got some CSS and JavaScript files here. All of these happening before first contentful paint ever occurs before anything hits the screen some of these are being pulled from the Shopify CDM which is you know how Shopify resources work.

Oh yeah. You'll notice we talked about on the last stream we mentioned this brand video blocking that adding that Chrome is working on. I think this was an oops I didn't mean to actually ship this out so we'll just our secret. Don't tell anybody outside of but what is in the details now. It's there for now nothing more prominent because there are issues with there are a couple of bugs like right now until chrome 92 comes out. It's flagging dynamically inserted scripts as blocking when they're not. And also an issue where if you preload a resource that it is should be blocking like CSS because it's bloated first chrome flags it's at not blocking because there is a preload in the connection to it.

[Inaudible 21:12 ] update this status once they realize oh it's CSS file that we are actually using a page so there's a couple of quirks but yeah that's there going about. So anyway the thing that jumps out here amongst the other stuff here is that they do have a bunch of third-party scripts that are blocking dynamic yield which is a [Inaudible 21:34]

This is how I talk when it's just me by the way [Inaudible 21:43] platform diagnostic.

How does this appearance optimal as a platform personalization stuff like that. OK. And then there's track J.S. there. Yada. Never quite sure how to pronounce that. Anytime there are third parties in that critical path between that initial response and that initial display of the screen. You could probably expect some level of variability at risk for certain risk on delaying that first contentful paint or start render metric and so if we're looking at this one of the things we want to do is kind of see like what is the cost? What are the risks associated?

So there are a few different things we can do here. I'm going to pick on a single third-party provider at the time at now just because I think it will demonstrate things a bit more. It's just what [Inaudible 22:39] I guess. Let's do dynamic yield because I see yadda-yadda coming in I see track JS but if you look at the dynamic yield it's actually here there's a pink block we'll make that a little bit more transparent. Let's go to Sunny. Let's go to run [Inaudible 22:56] again we'll do the film strip view and from here, I'm going go up to the top and add a parameter for N equals 4. There that truncates our waterfall 4 seconds. So yes say that's a segway we're headed there. Either way it just exaggerates a little bit we can see like [Inaudible 23:24] I should say so here we can see some JavaScript execution for both of the dynamic yield scripts. So not only is it the download but there's hefty JavaScript execution before first paint from both.

So I'm going to run a couple of tests around that. Yeah. Henri and Josh sorry the choppy audio is a thing that we are trying to figure out and haven't figured out yet apologies for that. This actually is not this is like a fair price Mic. It just looks fancy. Can you all see me at all? No let me just do this. I can change that at least done. This is actually a fair price mic it’s not real. Hence the audio issues. I just do it.  All right. So if we go back to we are going to run this test again we are going to do a couple of things there. So first we'll get a spot test running in the background this one. I know is one of Pat's favorites. Sergei alluded to it. We are going to simulate what happens if dynamic yield has problems.

So grab dynamicyield.com we'll drop in our URL under advance settings go to the spot tab and then we're going to drop in the seeding.dynamicyield.com host and we'll click that off in the background let that start running. So what this is going to do is going to fire off a test as not normal and then it's going to do a test where it routes any request to this hostname to [Inaudible 25:25] which is a fantastic little thing it just never responds. It just hangs it'll just hang requests forever. This way you can see like what is a risk of having a blocking third-party resource in your critical path and takes a while to load. So we'll have that going in the background.  

The other thing I want to do though. Is I want to run a test and block Dynamic yield. So I'm going to block the full hostname here. Do we want to bump it that high or not? How long is that going to take? Yeah, I run [Inaudible 26:07] we’re going to be both [Inaudible 26:09] or go home. I'll show you why in a second and I'll put a label just so later start the test. So it's going to take seconds for both of us to run show you first the reason why I went with nine test runs because usually I keep it pretty small when I'm doing it like this because I want the results very fast. But for comparison's sake. I think this will for what I would demo this would be better.

So if we go back to our original result. Hi Josh. Yeah. You could [Inaudible 26:55] thousand tests. I know that would be great if somebody did a thing, Pat if you're here. Do you remember somebody like a study about how many test results to run in webpage test [Inaudible 27:10] and I could not find it. I was looking at it the other day but something like above 11 or 9 ish. It was in that range. Like it starts to lose value adding more tests.

So what I'm about to show [Inaudible 27:26] Part of this is part of this comparison point. We often look at the summary table and leave it at that like that median run that's great. But there's actually this plot for result page which again is Matt Hobbs's favorite I think and not a lot of people know about it not a lot of people use it's actually really really nice. Particularly one you want to compare two different runs where you're experimenting with optimization or removing something. What this page does it takes the test or tests that are passed in it. It’s going to plot out on a chart the results for each individual will test run. So even an individual test run like that it's nice as I can see the variability. There's a lot of focus and monitoring and testing to ensure you get consistent and reliable results. The variability is just as interesting though because we can see things where all right we've got some really very largest contentful paint metrics here. That's probably an indication we have something going on that's kind of unstable.  

I mentioned having those third parties in their critical path like that can do risk that could be part of the reason why we have variability on contentful paint largest contentful paint or a really good one for Shopify stuff right. For anybody I guess it's time to first byte I've seen some highly variable server response times. If cache gets busted or something like that. Being able to spot these graphs even an individual run is kind of nice to be able to just see how many outliers you have like how consistent are those metrics on the page itself. And so this is a useful thing. But the other thing you can do it though is compare across different test. So this is the completed result that we just ran. So if we do we'll switch median metric against first contentful paint. Tim learns how to type OK. And we'll go to run 8 and the waterfall and if we look there's no dynamic yield anymore. Those requests removed entirely. We still have other blocky third-party things are track JS and Yadda.

You know this TDM Shopify dot com domain is kind of tough to get away from but remove the dynamic yield thing. So largest contentful paint at the median already looks a little better but what I want to see is again compare the two Tests and see overall did it provide any more stability. What to do across all the runs like consistently faster without dynamic yield in place to do that and to set it up. I mentioned this as Matt Hobbs fit your features he built this handy WT-compare.app and what this lets you do is drop it just makes it easier to generate these compare stages. So if I grab URL from test results for well let's grab the one with dynamic yield.  

so I've got my result URL I'm going to drop back into his I going to add a label DY and then I'm going to the test result with dynamic yield. Full URL drop that and [Inaudible 31:00] generate URL he generates a quick way to generate the URL comparing them on the filmstrip or in this case the graph page comparison view. So if I click through to that now I've got this plot full results and I've got it for both tests. In this case since test optimization [Inaudible 31:23] we're going to compare to our baseline and our baseline test was done with dynamic yield. We're also [Inaudible 31:29] virtual clerks at zero just for consistency across the chart graphs and hit plot results.

So now get some real helpful context so we're not just seeing at the median we're actually seeing a plot of all of the results for both of the tests. We're seeing some statistics to help us determine whether or not it was statistically significant difference between the two. And if so like the degree in terms of that difference. So this is really helpful if you're experimenting with optimizations removing third parties and you want to just move beyond the median this is kind of that next level of how you're looking at it. So here we can see that removing dynamic yield consistently provided much better first contentful paint it also seems to provide a little bit of stability there. And then largest contentful paint again. Substantial actually improvement across the board there. Interestingly some stuff I was not expecting that. Total block time makes it we killed some [Inaudible 32:33] so you can see in a scanning here. Yeah we saw an improvement and we saw a pretty viable improvement across a bunch of the test runs.

So this is one of those really sort of. Again things that maybe you haven't played with a lot of people haven't but it's very very very helpful for you know fine-tuning and making sure that you're getting detailed information about optimizing and not just looking at the median. And even if you are doing it just for the individual test then don't ignore variability like we're very hesitant in webpage test to do anything that's losing out too much the variability, because the variability is important to pay attention to. If you have variability in your time to first bite you have variability in your first contentful paint or something like that. You know that's a hint that there's something else going on you probably need to dive to into. So this is.

And I ran that SPA test didn't I? Look at that too. Where's that. This is the SPA test.

So this is what the SPA ting generates for you as you get this filling up of the original versus the third party failing. And notice how at 6.5 seconds were set but with the starting point of failure issue. You need like good goods effects for this. There's going to be like a song. All right. Maybe they could Jebbie thing we could do. I don't know. There you go 5.5 seconds. So that is a single point of failure and that's the risk of having those third parties in that critical math, Is it can really push that out and I love this SPA  test just to make it like I really ever want to show to anybody in the organization like this film score even better create this video. These are super effective ways of communicating that risk to other people inside the organization particularly a video the videos can be a really powerful way of demonstrating that. So if you're testing those 3rd parties love that combination of this [Inaudible 34:38 ] as we plot graph results with block or block domains except to be able to see the impact of removing individual ones or you know Pat mentioned block except as a great fit for. Quickly finding and pulling out. All third parties. Yeah.

Sarah So I love to show you more stuff that we kind of all of those things that maybe lurk about that maybe you haven't bumped into yet. But I do have to bump in just a few minutes and head off so I guess at this point if any but got questions and comments or just chat and you know I think anybody wants. Nobody wants Tim singing instead of a spinner that's a stub. I feel like it's a terrible idea. Oh no. Yes. Bad Yeah. There's a lot of like secret sauce inside of Webpagetest we're working on documenting a lot of back. And stuff but I've told people for years it's like, I've. Used web page test for over a decade. And as you start digging into it. Even day to day doing that. There's still stuff I come across pretty much every other day. I didn't know exists I didn't know were things. That webpage test does. Just fun. It isn't actually legitimate fun.

Oh yeah. And in the console. Well actually the console really before we hop off too this was the [Inaudible 36:26] page. So if you got a screenshot tab which is probably why you might not have bumped into this before. The screenshot tab gives you a quick link for video and video frames. Single fully loaded thing but it also outs everything that was captured in the console itself. So in this case you can see the networker right because we've blocked dynamic yield so it' refusing those [Inaudible 36:52] get things like issues with particular I guess most of these blocking. Also this like we've got a duplicate Facebook [Inaudible 37:03 ] going on I run into that a lot logs being debugged I wonder if I can find like a deprecating we've got sync [Inaudible 37:12 ] Ajax having here from triggered by yada is boy a bet if I went through my test history. If you've ever used chrome you know that chrome will do things like give you I'm just going to click launch better that's good

No not interesting what I wanted. At least I again [Inaudible 37:42] free folks. Here we go. So like something like this where chrome [Inaudible 37:49] these different violations right. Like a [Inaudible 37:51] document right and a blocking [Inaudible 37:53] or if there was an intervention or something like that. All of that shows up in the console as well and gets recorded on these tests can be a really handy thing to look at. So like violations in particular like Chrome's giving a lot of helpful information or you've ever done like a preload of your routine they give you an error there a preload that ends up not being used by the paid each that shows up in these things. So there's a lot of really help a full context captured and consoles as well. That can debug a little bit there too.

So I got bounce. Sorry for the choppy audio again. The old Kung Fu movie as [Inaudible 38:43] is putting it. Yeah I will work on that. We'll work on fixing that for the next time. I. Will try to. Brush up a little bit. We're learning. Well we did a couple of. Things already. Like on. The stream to make it a little bit better so. Yeah. So if you do. Thanks for joining in again always great to see everybody did all the chatting. So nice you used to see all these people in person anymore. So it's nice to do that. Oh yeah there's a [Inaudible 39:18] things.  

There's a conference online starting tomorrow that Henri curated with the Web directions folks called lazy [Inaudible 39:29]. Which Pat is speaking at the lineup looks awesome. Henri like hats off to you man. You did great job of bringing some familiar faces that you're going to deliver but there are a lot of people that I haven't heard from before and I'm really excited about that. Some really really nice job then yeah.  

In November we're tentatively provisionally trying to performance. Now in Amsterdam in person. Fingers crossed we'll see. So we put out the announce we have it again great lineup we've teased folks already from that maybe. Hopefully, that comes to Gavin and people [Inaudible 40:07] enough people are comfortable and really have a kind of a safe environment for folks to do that in person. It would be amazing to see everybody again. So yeah hopefully that happens as well

But I'm going to jump off thanks everybody for joining. We'll be back on the 24th. Same time 12PM central whatever it is for other things. I am really bad at time zones but at 10 PM, 10 AM. Pacific, 1 PM Eastern. Don't ask me to go beyond that. My skills in time zones are [Inaudible 40:43] We will be back on the 24th with something new. Yeah. If you're interested in kind of joining on we are talking to. I know we've reached out to a few folks about like letting them lead audits or have conversations should be breaking up the format a little bit. If you have ideas for things you think would be interesting. Let us know. All right. Thanks, everybody.

‍