They’re out there. On your production server, QA server, even your dev machine. Crash dumps!
Scary things that they are, once you know how to handle them they are a powerful tool for solving problems in your applications (everything from crashes, excessive memory usage or leaks and performance problems). However, getting to the point where you can handle them can be quite a challenge. I’ve managed to analyze a few crash dumps of production asp.net worker processes in the past, but it’s always been a challenge. Sometimes a challenge that lasted a few days, including lots of googling. Just getting to the point where I could actually open them properly was usually an achievement.
But once you get your hands into them the wealth of information available to you is astounding. Considering how much pain I went through every time I had to analyze one I decided I’d look around and see if there was any way I could write my own front end to use when investigating them. I had a few ideas for this front end:
- Opening a crash dump should be no more difficult than your usual File | Open operation. It shouldn’t take you hours to get to the point where you can see what’s inside.
- You should be able to explore the crash dump with simple point and click interaction. No obscure 2 letter keyboard commands.
- The front end should provide rich interactivity and graphics. A broad statement but pictures are always better than words. I want to see more than just a screen full of plain text in front of me.
- As you explore the crash dump you should easily be able to see the path you’ve followed to get to where you currently are. I’ve found analyzing crash dumps to be a hit and miss experience. This is quite likely due to me not knowing exactly what I was doing, but it was hit and miss none the less. Often I have found myself deep down the rabbit hole, forgetting how I had gotten there, and then having to scroll up through pages of text trying to find out how I got where am I was (only to go back down again!).
- On a technical note, I wanted the app to be 100% c#. Naturally I’d have to use various system libraries, but everything I wrote would be c#.
Fast forward a few years (I’ve been at this for a while) and here we are. After a couple of false starts I’ve managed to get a proof of concept up and running that can load a crash dump and access all information I need: threads, call stacks, modules, memory and even objects on the heap. Before I go on I probably should have already mentioned that I’m only talking about crash dumps of processes running .Net code i.e. where the CLR is loaded and executing code. Crash dumps of processes running purely native code are a whole different kind of problem that I don’t intend trying to solve.
Now that I’ve proved I can get access to all of the information I need, it’s time to start putting together an actual application and getting it into your hands. I’ve got some really big ideas (or rather hopes) around building a front end that will hopefully make and exploring the information in crash dumps much easier than it is today.
It’s been quite a journey getting the proof of concept to where it is now: there have been a lot of questions, a lot of niche APIs and a lot of learning already done (I don’t think the learning will stop any time soon). If I wait until I’ve finished everything it’ll still be a while until anyone gets to see anything and I’m keen to interact with other people who might be interested in this and hear their thoughts and ideas. So I’ve decided to try to “release small, release often”. That way I can start sharing what I’ve learnt and start getting some feedback hopefully. Oh, and some help with testing!
Next time I’ll talk about the “release often, release small” and I’ll actually release something! Stay tuned.