ClrMD Part 8 – Spelunking inside the .NET Thread Pool

This post of the series shows how to easily list pending tasks and work items managed by the .NET thread pool using DynaMD proxies.

Part 1: Bootstrap ClrMD to load a dump.

Part 2: Find duplicated strings with ClrMD heap traversing.

Part 3: List timers by following static fields links.

Part 4: Identify timers callback and other properties.

Part 5: Use ClrMD to extend SOS in WinDBG.

Part 6: Manipulate memory structures like real objects.

Part 7: Manipulate nested structs using dynamic.

Introduction

The previous posts introduced the DynaMD nuget that helps navigating among type instances using a C#-like syntax “instance.field”. Let’s see how to use it to enumerate the asynchronous items queued in the .NET thread pool. As a bonus, the running tasks and work items won’t be forgotten.

ThreadPool internals

The .NET ThreadPool is keeping track of the pending work items into two different data structures:

• A global queue: stored as a ThreadPoolWorkQueue instance referenced by the workQueue static field

• several per-thread (TLS) local queues: stored in SparseArray<ThreadPoolWorkQueue+WorkStealingQueue> linked from the allThreadQueues static field

As you can see, the algorithm to list the pending tasks and work items starts from a static field and iterate on a linked list of QueueSegment for global queue and array of WorkStealingQueue for per thread queues. Both are storing arrays of IThreadPoolWorkItem that Task and QueueUserWorkItemCallback are implementing:

Too much theory… Let’s write some code!

Global ThreadPool queue

You have seen in a previous post how to access the value of a static field per application domain:

For an application domain in which the threadpool is not used, we need to check against null for the expected ThreadPoolWorkQueue:

The role of the EnumerateThreadPoolWorkQueue helper method is to iterate on each QueueSegment of the linked list pointed to by the queueTail field of the per appdomain ThreadPoolWorkQueue object.

At the beginning of the following code, note that dynamic allows writing C# code even though the queueTail and nodes fields are not known at compile time. Even more convenient, a foreach statement is possible when the instance behind the DynaMD proxy is an array:

The GetThreadPoolItem helper method will be described soon but first, let’s see how to get the items from the thread local queues.

Local ThreadPool queues

The same static field driven operations are needed to access the sparse array containing… more arrays:

The spare arrays contain either null or a stealing queue that itself contains… an array:

Now that we managed to retrieve the thread pool items, we can try to decipher them.

Deciphering thread pool items

A thread pool item stored in the global or in the local queues could be a Task, a QueueUserWorkItemCallback or a simple method:

The kind of item is computed from the ClrType of the object given by the ClrHeap.GetObjectType method. An ulong address is expected by this ClrMD method and it would be easy to get from the dynamic returned by DynaMD by just casting it to ulong. However, it is easier to simply call the GetClrType method on the dynamic proxy!

Next step…

The next and last episode of the ClrMD series will show you how to decipher tasks and thread pool items to know which of your methods will be called. As a bonus, the running tasks and work items won’t be forgotten.

Post written by: