Browsing:

Category: Buttonfactory

Analyzing a server 2008 R2 dwp crash dump file

Yesterday the four node file cluster resource crashed and blue screened and was moved to another node. I wanted to analyze the crash dump file (C:\Windows\Minidump\070711-36473-01.dmp) so I copied it to my W7 workstation and tried to open it but Visual Studio could not help me out here.

Reading a crash dump file is far from intuitive and I spent a great deal of the morning learning about debugging. So here is what I did to read the dump file.

First, you need to install the debugging tools from here. Choose the version that corresponds to your architecture. This install will take a long time depending on your network speed. Important is that you include the WinDbg.exe because that is the tool we will be using.

Next, you need to download the symbol files. Note that you can also use the symbol server from Microsoft but it is faster to have a copy of the symbol files on your hard drive. Download them here. Just download them all. And this will also take a long time because the Symbol files are huge.

Next! Open C:\Program Files\Debugging Tools for Windows (x86)\WinDb.exe.
Choose File -> Open -> Symbol File Path

Type: SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols like this:

Now press CTRL+D to open the DWP file! Very exciting.

Now, if you enter !analyze -v like this:

And you'll get more information about the crash. In my case:

8: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

USER_MODE_HEALTH_MONITOR (9e)
One or more critical user mode components failed to satisfy a health check.
Hardware mechanisms such as watchdog timers can detect that basic kernel
services are not executing. However, resource starvation issues, including
memory leaks, lock contention, and scheduling priority misconfiguration,
may block critical user mode components without blocking DPCs or
draining the nonpaged pool.
Kernel components can extend watchdog timer functionality to user mode
by periodically monitoring critical applications. This bugcheck indicates
that a user mode health check failed in a manner such that graceful
shutdown is unlikely to succeed. It restores critical services by
rebooting and/or allowing application failover to other servers.
Arguments:
Arg1: fffffa8038f3ab30, Process that failed to satisfy a health check within the
	configured timeout
Arg2: 00000000000004b0, Health monitoring timeout (seconds)
Arg3: 0000000000000000
Arg4: 0000000000000000

Debugging Details:
------------------

PROCESS_OBJECT: fffffa8038f3ab30

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT_SERVER_MINIDUMP

BUGCHECK_STR:  0x9E

PROCESS_NAME:  System

CURRENT_IRQL:  2

LAST_CONTROL_TRANSFER:  from fffff880030b76a5 to fffff80001a98d00

STACK_TEXT:
fffff880`0253d518 fffff880`030b76a5 : 00000000`0000009e fffffa80`38f3ab30 00000000`000004b0 00000000`00000000 : nt!KeBugCheckEx
fffff880`0253d520 fffff800`01aa4652 : fffff880`0253d600 00000000`00000000 00000000`40800088 00000000`00000001 : netft!NetftWatchdogTimerDpc+0xb9
fffff880`0253d570 fffff800`01aa44f6 : fffff880`030c4100 00000000`03023940 00000000`00000000 00000000`00000000 : nt!KiProcessTimerDpcTable+0x66
fffff880`0253d5e0 fffff800`01aa43de : 00000729`6e09a2ce fffff880`0253dc58 00000000`03023940 fffff880`02517d88 : nt!KiProcessExpiredTimerList+0xc6
fffff880`0253dc30 fffff800`01aa41c7 : 000001c5`99d9f3c1 000001c5`03023940 000001c5`99d9f3fd 00000000`00000040 : nt!KiTimerExpiration+0x1be
fffff880`0253dcd0 fffff800`01a90a2a : fffff880`02515180 fffff880`025202c0 00000000`00000000 fffff880`01368420 : nt!KiRetireDpcList+0x277
fffff880`0253dd80 00000000`00000000 : fffff880`0253e000 fffff880`02538000 fffff880`0253dd40 00000000`00000000 : nt!KiIdleLoop+0x5a

STACK_COMMAND:  kb

FOLLOWUP_IP:
netft!NetftWatchdogTimerDpc+b9
fffff880`030b76a5 cc              int     3

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  netft!NetftWatchdogTimerDpc+b9

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: netft

IMAGE_NAME:  netft.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  4a5bc48a

FAILURE_BUCKET_ID:  X64_0x9E_netft!NetftWatchdogTimerDpc+b9

BUCKET_ID:  X64_0x9E_netft!NetftWatchdogTimerDpc+b9

Followup: MachineOwner
---------

Explanation: USER_MODE_HEALTH_MONITOR (9e) is the bug check code I need to investigate. For a complete list of bugcheck codes look here:
http://msdn.microsoft.com/en-us/library/ff542347%28v=VS.85%29.aspx

And now all that is left for me to say is: 'happy debugging'.

Oh here are some helpful links:
http://blogs.technet.com/b/askcore/archive/2009/06/12/why-is-my-2008-failover-clustering-node-blue-screening-with-a-stop-0x0000009e.aspx

http://blogs.msdn.com/b/ntdebugging/archive/tags/hangs/


Really really back to basics: Intro to OOP: C# classes. (And intro to Linq to XML as well).

Here's an intro to OOP in C# for those who are interested. It's a real world example.
I have a bunch of SQL Servers and I need to execute some SP's on their databases. The configuration information is stored in an XML file. I want to solve this problem in an OOP way. So we need to create a class and instantiate objects from that class.

Obviously, there should be a class named Database server or something like that, and it has properties (or variables, or fields).

Creating the class

First let's create a class:

public class DbServer {

}

Creating its properties

Then, we will define its properties:

public class DbServer {
private string _name;
private string _instance;
private string _database;
private string _username;
private string _password;
}

Why are they private? That is to make sure that these variables can only be manipulated from inside the class.
But what is that good for? These values should be queried, right?
Indeed. That's why public properties will be exposed to them as well:


public class DbServer {

private string _name;
private string _instance;

public string Name {
get { return _name; }
set { _name = value; }
}

public string Instance{
get { return _instance; }
set { _name = value; }
}

Why then, go through all this and not just make the fields public from the start?!
No, this is one of the principles of OOP: Encapsulation. There are some benefits to encapsulation:

  • Preventing unauthorized access to the data
  • Ensuring data integrity through error checking
  • Creating read only properties

Let's alter the code a bit:

public string Name {
   get {
        return _name;
         } 
   set {
   //should not be empty        
       if (value.Length < 1 ) {
        throw new Exception("Servername cannot be empty");
         } else {
         _name = value;                    
          }
   }          
 }           

There, I added a business rule. It would also be nice to add some more regex to the class and throw appropriate exceptions.

here's how to make the property read-only by omitting the 'setter' and for the 'getter' to actually fetch the data from a database, or as in this scenario, from an XML file:

public string Instance
    {
       get
          {
            _instance = getElement().Element("instance").Value;
             return _instance;
           }
      }

Auto-Implemented Properties

In cases when there would be more trivial properties when there is no additional logic required, you can use auto-implemented properties. When you declare a property as shown in the following example, the compiler creates a private, anonymous backing field that can only be accessed through the property's get and set accessors.


public class DbServer {

public string Name { get; set; }
public string Instance { get; set; }

}

In this way, the properties are mutable by other client code.
But what if you would fetch the Instance value from a database or an XML file?

Then the code would be as follows:


public class DbServer {

public string Name { get; private set };

public string Instance
  { 
    get
     {  
          return getElement().Element("instance").Value; 
      }  
 }

//the constructor
public DbServer (string name) 
{
    Name = name;
}

Where did the business logic go? In this particular class, there is hardly need for business logic, as it reads values from an XML file. It might be better to separate the business logic in another (partial) class.
The servername (Name) is obligatory, so we set that when instantiating the DbServer object: in the constructor.

The constructor

To find the right SQL server I need to supply the servername. With that servername, we can query the XML file for the rest of the values of the properties.
So, when instantiating the DBserver object, we should provide the servername.
Therefore we need a constructor.

public DbServer(string name)
   {
        Name = name;
    }

Now we can instantiate a Dbserver instance with a name. Like this:

 DbServer server = new DbServer("SQLSERVER01");

Well.. these are the basics of OOP in C#. Now let's add a little bonus material. Here is how we actually read the values from the XML file:

Linq to XML

This is the XML file that contains all then servers:

To query this with C#, we need to add the following references:

using System.Xml;
using System.Xml.Linq;

Querying an XML document is as simple as this:

var result = from s in XDocument.Load("Servers.xml").Descendants("server")
                where s.Element("name").Value == "SQLSERVER01"
                select s).FirstOrDefault();

we would get the value of the username field like this:

result.Element("username").Value;

All the code

Finally, this is all the code from this article:

Hope this has helped you somehow!