Reading minidump files, part 3 of 4: Reading stream data (ModuleListStream) returned from MiniDumpReadDumpStream

All code is available on my CodePlex project.

This is part 3 of a 4 part series on using MiniDumpReadDumpStream. Part 1 discussed how to access memory mapped files from C#. Part 2 showed how to invoke MiniDumpReadDumpStream. This article follows on from part 2 and discusses how to read the data returned from MiniDumpReadDumpStream.

I ended the previous article with the C# signature we’ll be using to invoke MiniDumpReadDumpStream:

[DllImport("dbghelp.dll", SetLastError = true)]
public static extern bool MiniDumpReadDumpStream(IntPtr BaseOfDump,
        uint StreamNumber,
        ref MINIDUMP_DIRECTORY Dir,
        ref IntPtr StreamPointer,
        ref uint StreamSize);

This article will discuss how to read the data that is returned in StreamPointer.

I’m going to use MINIDUMP_STREAM_TYPE.ModuleListStream as an example as it is fairly simple to read and also covers most of the code we’ll need to use when reading other streams.

Before I go into the details I’ll give a quick summary of what we’re going to be doing:

  • First we’ll invoke MiniDumpReadDumpStream() and ask it to read the MINIDUMP_STREAM_TYPE.ModuleListStream. This minidump stream contains a list of modules.
  • The call will return a pointer (StreamPointer) to a MINIDUMP_MODULE_LIST structure as well as the size of the structure (StreamSize).
  • MINIDUMP_MODULE_LIST has 2 fields: one telling us how many modules there are (NumberOfModules) and the second containing an array of the modules (Modules[]).
  • First we’ll read the MINIDUMP_MODULE_LIST and then we’ll read each of the MINIDUMP_MODULE’s (with a few tricks along the way).

First up we need to call MiniDumpReadDumpStream to read the ModuleListStream.

  • BaseOfDump is the pointer to the memory mapped view that we created previously.
  • StreamNumber is the stream we want to read. We’ll be using MINIDUMP_STREAM_TYPE.ModuleListStream (which happens to be “4”).
  • Dir, StreamPointer and StreamSize are all returned from the call, so we’ll create local variables for them.

The Dir parameter is of type MINIDUMP_DIRECTORY. This is declared in dbghelp.h. We won’t be using this, so I’ll just list the C# translation without going into details:

internal struct MINIDUMP_LOCATION_DESCRIPTOR
{
    public UInt32 DataSize;
    public uint Rva;
}

internal struct MINIDUMP_DIRECTORY
{
    public UInt32 StreamType;
    public MINIDUMP_LOCATION_DESCRIPTOR Location;
}

Now that we know the types for all of the parameters we can invoke the method:

MINIDUMP_DIRECTORY directory = new MINIDUMP_DIRECTORY();
streamPointer = IntPtr.Zero;
streamSize = 0;

// baseOfView is the IntPtr we got when we created the memory mapped file
if (!NativeMethods.MiniDumpReadDumpStream(baseOfView, MINIDUMP_STREAM_TYPE.ModuleListStream, ref directory, ref streamPointer, ref streamSize))
{
    throw new Win32Exception(Marshal.GetLastWin32Error());
}

Using Marshal.PtrToStructure() to marhsal unmanaged to managed memory

So if reading the ModuleListStream is supposed to return a MINIDUMP_MODULE_LIST structure how do we get to it from the IntPtr returned in streamPointer? streamPointer is a pointer to a MINIDUMP_MODULE_LIST structure held in unmanaged memory. In order to make use of it we need to marshal the structure from unmanaged into managed memory. Luckily the .Net Framework does all of the heavy lifting for us, all we need to do is call Marshal.PtrToStructure and it’ll do the marshaling for us, like this:

MINIDUMP_MODULE_LIST moduleList = 
    (MINIDUMP_MODULE_LIST) Marshal.PtrToStructure(streamPointer, typeof(MINIDUMP_MODULE_LIST));

With MINIDUMP_MODULE_LIST defined as:

[StructLayout(LayoutKind.Sequential, Pack = 4)]
internal struct MINIDUMP_MODULE_LIST
{
    public UInt32 NumberOfModules;
    public IntPtr Modules; // Why not "MINIDUMP_MODULE[] Modules"? Keep on reading...
}

Simple hey? Too simple? Unfortunately for us, this time it is too simple.

While the marshaler does an excellent job of letting us call into unmanaged code and transfering data between the two, it’s not always perfect. You’ll remember I said that MINIDUMP_MODULE_LIST contains a Modules field that is an array of MINIDUMP_MODULE’s. So why have I declared Modules as an IntPtr? It’s because the marshaler doesn’t seem to be able to marshal an array of structures (by value) contained within a structure. At least I haven’t been able to find a way to do it. So we’re going to have to do this in 2 parts: we’ll get the MINIDUMP_MODULE_LIST structure like we already have, and then we’ll read the array seperately (we’re actually not even going to use the Modules field directly). Stick with me here, this part needs some concentration.

Lets look at MINIDUMP_MODULE_LIST again: we have NumberOfModules which is 4 bytes (32 bits) followed by an array MINIDUMP_MODULES, each of which is 108 bytes (I’ll show you the C# version soon). When I read my sample minidump I get streamPointer = 0x0f84054c and streamSize = 0x000011bc (4540 bytes). So if NumberOfModules is 4 bytes then I should have 4536 bytes (4540 – 4 = 4536) allocated to the Modules field. If each MINIDUMP_MODULE is 108 bytes then I should have 42 modules (4536 / 108 = 42). The important thing to note here is that all the data we need is contained in one, sequential, block of memory: NumberOfModules is followed by 42 allocations of MINIDUMP_MODULE’s.

After calling Marshal.PtrToStructure() to marshal my streamPointer into a MINIDUMP_MODULE_LIST structure I can see that NumberOfModules is “42”, as expected. So we have all of the data we need, we just need to get it into a format that we can use.

Since Modules is an array of strucutres, and Marshal.PtrToStructure() can marshal pointers into structures, we can use it and some basic pointer arithmetic to do the work for us.

We know the data for the stream beings at 0x0f84054c (that’s my value for the streamPointer). And we know that the first field (NumberOfModules) is 4 bytes. So if we add 4 to the value of streamPointer we’ll get the address of the first MINIDUMP_MODULE structure (the first strucutre in Modules). So we could do something like this:

streamPointer += Marshal.SizeOf(typeof(MINIDUMP_MODULE_LIST.NumberOfModules)); // Advance the pointer past the "NumberOfModules" field

MINIDUMP_MODULE theFirstModule = 
    (MINIDUMP_MODULE) Marshal.PtrToStructure(streamPointer, typeof(MINIDUMP_MODULE)); // read the first MINIDUMP_MODULE

Put that in a for…loop and we’ve read the whole structure successfully.

There is actually an easier way to do this. I’ve been experimenting with using SafeMemoryMappedViewHandle as the return value to MapViewOfFile instead of IntPtr. Either will work, but SafeMemoryMappedViewHandle includes a ReadArray() method, which makes our code easier to read as there is no need to read each MINIDUMP_MODULE individually.

SafeMemoryMappedViewHandle does make other things a bit more tedious though. You have to call AcquirePointer() & ReleasePointer() everytime you want to get at the memory address it references. I’ll probably stick with it though as it does make reading arrays easier. It also handles bounds checking and cleanup for us.

Now that we can successfully read a stream it makes sense to put this all into a method and use some generics:

protected unsafe T ReadStream(IntPtr baseOfView, MINIDUMP_STREAM_TYPE streamToRead)
{
    MINIDUMP_DIRECTORY directory = new MINIDUMP_DIRECTORY();
    IntPtr streamPointer = IntPtr.Zero;
    uint streamSize = 0;

    if (!NativeMethods.MiniDumpReadDumpStream(baseOfView, streamToRead, ref directory, ref streamPointer, ref streamSize))
    {
        throw new Win32Exception(Marshal.GetLastWin32Error());
    }

    return (T)Marshal.PtrToStructure(streamPointer, typeof(T));
}

Now that we can successfully read MINIDUMP_MODULE_LIST it’s time to talk about MINIDUMP_MODULE a bit. The definition for MINIDUMP_MODULE in the MSDN is:

typedef struct _MINIDUMP_MODULE {
  ULONG64                      BaseOfImage;
  ULONG32                      SizeOfImage;
  ULONG32                      CheckSum;
  ULONG32                      TimeDateStamp;
  RVA                          ModuleNameRva;
  VS_FIXEDFILEINFO             VersionInfo;
  MINIDUMP_LOCATION_DESCRIPTOR CvRecord;
  MINIDUMP_LOCATION_DESCRIPTOR MiscRecord;
  ULONG64                      Reserved0;
  ULONG64                      Reserved1;
} MINIDUMP_MODULE, *PMINIDUMP_MODULE;

There are a few new types we need to map here.

ULONG64 is defined as a 64-bit unsigned integer. We’ll use UInt64 in C#.

We’ve discussed ULONG32 before, it’s a UInt32 in C#.

ModuleNameRva is an RVA. This is an interesting one. Physically it’s defined as a DWORD in DbhHelp.h, so we’ll use a uint. That part is easy. What it means is a bit more complicated. An RVA is a “Relative Virtual Address”: it contains the offset from the beginning of the memory mapped view where reading should start. So if the memory mapped view starts at 0x00000010 and the RVA is 4, then you should reading from 0x00000014 (0x00000010 + 4).

Reading MINIDUMP_STRING’s

The documentation for MINIDUMP_MODULE says that ModuleNameRva is “An RVA to a MINIDUMP_STRING structure that specifies the name of the module”. So if we add ModuleNameRva to the memory mapped view pointer we’ll find a MINIDUMP_STRING structure:

typedef struct _MINIDUMP_STRING {
  ULONG32 Length;
  WCHAR   Buffer[];
} MINIDUMP_STRING, *PMINIDUMP_STRING;

I haven’t got the marshaler to read this structure successfully in it’s entirety, so we’ll use the marshaler to read the pieces seperately:

We have the pointer to the memory mapped view and we have the RVA offset to the MINIDUMP_STRING, so if we add them together we’ll get the address of the Length field:

IntPtr positionToReadFrom = new IntPtr(baseOfView + rva);

Length is 32bits, so we’ll ask the marshaler to read that for us:

int numberOfCharacters = Marshal.ReadInt32(positionToReadFrom) / 2;

You’ll see I’ve read the 32 bits for the Length field and then divided it by 2. This is because we’re going to call Marshal.PtrToStringUni() to read Buffer for us. It needs to know the number of characters to read, not the number of bytes to read. Buffer is an array of wide chars (WCHAR), so each character in the string is 2 bytes (this is to cater for unicode characters). Length contains the size of the Buffer in bytes, not the number of characters, so we have to divide it by 2 to get the number of characters. As an example: Length could indicate 20, which means Buffer is 20 bytes big, but contains 10 (20 / 2) characters, each of which is 2 bytes.

To read the string we call Marshal.PtrToStringUni():

// "advance" the pointer 4 bytes (to jump over the "Length" field);
positionToReadFrom += 4; 

// Read and marshal the string
return Marshal.PtrToStringUni(positionToReadFrom, numberOfCharacters);

The full method looks like this (I’ve used SafeMemoryMappedViewHandle in this case instead of IntPtr for the memory mapped view. You can see the extra code this adds):

protected internal unsafe string ReadString(uint rva)
{
    try
    {
        byte* baseOfView = null;

        // _mappedFileView is a SafeMemoryMappedViewHandle 
        _mappedFileView.AcquirePointer(ref baseOfView);

        IntPtr positionToReadFrom = new IntPtr(baseOfView + rva);

        // First 32bits is the length field, which is the number of bytes, not number of chars. Divide it by 2 to get number of chars (WCHAR is 2 bytes - unicode)
        int len = Marshal.ReadInt32(positionToReadFrom) / 2;

        // "advance" the pointer 4 bytes (to jump over the "Length" field);
        positionToReadFrom += 4; 

        // Read and marshal the string
        return Marshal.PtrToStringUni(positionToReadFrom, len);
    }
    finally
    {
        _mappedFileView.ReleasePointer();
    }
}

That takes care of reading ModuleNameRva. Next up is VersionInfo. This is of type VS_FIXEDFILEINFO. I won’t go into the detail of each field; the documentation describes them well enough. The C# version is below.

CvRecord and MiscRecord are both MINIDUMP_LOCATION_DESCRIPTOR which I’ve already mentioned. The final versions of VS_FIXEDFILEINFO and MINIDUMP_MODULE are:

[StructLayout(LayoutKind.Sequential, Pack = 4)]
internal struct VS_FIXEDFILEINFO
{
    public UInt32 dwSignature;
    public UInt32 dwStrucVersion;
    public UInt32 dwFileVersionMS;
    public UInt32 dwFileVersionLS;
    public UInt32 dwProductVersionMS;
    public UInt32 dwProductVersionLS;
    public UInt32 dwFileFlagsMask;
    public UInt32 dwFileFlags;
    public UInt32 dwFileOS;
    public UInt32 dwFileType;
    public UInt32 dwFileSubtype;
    public UInt32 dwFileDateMS;
    public UInt32 dwFileDateLS;
};

[StructLayout(LayoutKind.Sequential, Pack = 4)]
internal struct MINIDUMP_MODULE
{
    public UInt64 BaseOfImage;
    public UInt32 SizeOfImage;
    public UInt32 CheckSum;
    public UInt32 TimeDateStamp;
    public uint ModuleNameRva;
    public VS_FIXEDFILEINFO VersionInfo;
    public MINIDUMP_LOCATION_DESCRIPTOR CvRecord;
    public MINIDUMP_LOCATION_DESCRIPTOR MiscRecord;
    public UInt64 Reserved0;
    public UInt64 Reserved1;
}

And the final version of our read methods are:

public MiniDumpModule[] ReadModuleList()
{
    MINIDUMP_MODULE_LIST moduleList;
    IntPtr streamPointer;
    uint streamSize;

    moduleList = this.ReadStream(MINIDUMP_STREAM_TYPE.ModuleListStream, out streamPointer, out streamSize);
    //4 == skip the NumberOfModules field (4 bytes)
    MINIDUMP_MODULE[] modules = ReadArray(streamPointer + 4, (int) moduleList.NumberOfModules);

    List returnList = new List(modules.Select(x => new MiniDumpModule(x, this)));

    return returnList.ToArray();
}

protected unsafe T ReadStream(IntPtr baseOfView, MINIDUMP_STREAM_TYPE streamToRead)
{
    MINIDUMP_DIRECTORY directory = new MINIDUMP_DIRECTORY();
    IntPtr streamPointer = IntPtr.Zero;
    uint streamSize = 0;

    if (!NativeMethods.MiniDumpReadDumpStream(baseOfView, streamToRead, ref directory, ref streamPointer, ref streamSize))
    {
        throw new Win32Exception(Marshal.GetLastWin32Error());
    }

    return (T)Marshal.PtrToStructure(streamPointer, typeof(T));
}

protected internal unsafe string ReadString(uint rva)
{
    try
    {
        byte* baseOfView = null;

        // _mappedFileView is a SafeMemoryMappedViewHandle 
        _mappedFileView.AcquirePointer(ref baseOfView);

        IntPtr positionToReadFrom = new IntPtr(baseOfView + rva);

        // First 32bits is the length field, which is the number of bytes, not number of chars. Divide it by 2 to get number of chars (WCHAR is 2 bytes - unicode)
        int len = Marshal.ReadInt32(positionToReadFrom) / 2;

        // "advance" the pointer 4 bytes (to jump over the "Length" field);
        positionToReadFrom += 4; 

        // Read and marshal the string
        return Marshal.PtrToStringUni(positionToReadFrom, len);
    }
    finally
    {
        _mappedFileView.ReleasePointer();
    }
}

And that’s it. We can now successfully create a memory dump and read module information from it. Next time I’ll put it all together in an application and publish that code.

Advertisements
This entry was posted in Crash Dumps and tagged , , , . Bookmark the permalink.

3 Responses to Reading minidump files, part 3 of 4: Reading stream data (ModuleListStream) returned from MiniDumpReadDumpStream

  1. Pingback: Reading minidump files, part 1 of 4: Accessing memory mapped files from C# | Greg's Blog

  2. Pingback: Reading minidump files, part 2 of 4: Using MiniDumpReadDumpStream | Greg's Blog

  3. Pingback: Reading minidump files, part 4 of 4: Putting it all together | Greg's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s