BSODTutorials: Understanding Stop 0x9F

Friday 29 November 2013

Understanding Stop 0x9F - How it Works

I've explained how to debug a Stop 0x9F in previous blog posts, but here I'm going to explain how exactly it detects a timeout, and thus bugchecks with the Stop 0x9F.

We should understand now, that the two highlighted parameters are the most important for this type of bugcheck. The fourth parameter shows the Blocked IRP which caused the timeout, and the second parameter is the PDO of the device the driver is associated with.

Lets look at the blocked IRP, using the !irp extension.

So, we can see that a power related IRP is pending, and has held too long by a device object leading to the crash. tunnel.sys seems to be associated with the IRP, however, this isn't the exact problem here.

We can see that tunnel.sys calls GsDriverEntry, which is a routine called after the driver is loaded and initialized. It take two parameters: a pointer to the DRIVER_OBJECT data structure, and a pointer to the path in the registry where the driver is stored. It's called at IRQL Level 0.

The ndis.sys calls the the driver unload routine, which deallocates any pool and then unloads the driver. The ndis.sys driver then handles all the dispatch routines. ndis.sys is handled by a Microsoft Miniport driver, since IRP dispatching routines are mostly generic, and can be very difficult to code.

I'm not exactly sure what the ndis!ndisDummyIrpHandler does, but a dissembly shows a pointer to the WPP, which is used for software tracing.

It seems to mostly push some data stored in the registers onto the stack.

The device instance seems to point to Teredo, which is used for IPv6 connectivity for IPv4 Internet connected computers. I think we can establish there is something definitely network connected, and it's possibly related to the adapter of some sort.

Now, let's get to the main point of discussion, the call stack which demonstrates how a time out is detected with a driver.

Looking at the stack, as we can see a timer has expired, which is shown with nt!KiProcessExpiredTimerList. We can view all the pending timers on a system with the !timer extension, for demonstration purposes, this was taken from a different dump file. The timer which would have been expired is the nt!PopCheckForIdleness.

So, when the timer expires, it calls the nt!PopCheckIrpWatchdog with nt!PopIrpList routine which searches through the IRP queue. Each time the nt!PopCheckIrpWatchdog is called, it increments a counter, when the counter has threshold has been exceeded, the system bugchecks with Stop 0x9F.

To find the driver which was causing the problem, I had to search the raw thread, it seems to be a very old network adapter driver. I believe it was related to Intel.