Age of JIT compiling. Part II. CLR is watching you (EN)

In my previous publication, we have started analyzing JIT compilation. Today we are going to explore method dispatch of interfaces and generics (both for classes and separate methods along with real signatures), as well as how to debug release-mode assemblies with optimization. In addition, we’ll figure out the true purpose of System.__Canon.

Environment configuration

At first, we need to set up Visual Studio for debugging release-mode assemblies.

We are going to use VS 2013. Thus, I recommend doing the following:

enable a compatibility mode to use SOS.dll. Tools -> Options -> Debugging -> General

uncheck Suppress JIT optimization on module load and Enable Just My Code.
Suppress JIT optimization on module load and Enable Just My Code
Enable Native Debugging Project Settings -> Debug -> Enable native code debugging

Now, we can move on to our analysis.

Interface dispatch stubs (Virtual Stub Dispatch)

CLR is constantly monitoring the whole code, especially interfaces. In addition, it has several strategies for updating method’s native code.

This feature appeared in CLR 2.0 in 2006. Since then it remained the same in many ways with new heuristics addition.

Consider the following example:

class Program
{
    static void Main(string[] args)
    {
        ICallable target = new FirstCallableImpl();
        CallInterface(target);

        ICallable target2 = new SecondCallableImpl();
        CallInterface(target2);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CallInterface(ICallable callable)
    {
        for (int i = 0; i < 1000000; i++)
        {
            callable.DoSomething(); // place breakpoint
        }
    }
}

interface ICallable
{
    void DoSomething();
}

class FirstCallableImpl : ICallable
{
    public void DoSomething()
    {

    }
}

class SecondCallableImpl : ICallable
{
    public void DoSomething()
    {

    }
}

Run Debug. Then, open the Disassembly window (Debug -> Windows -> Disassembly).

Take a look at the call call dword ptr ds:[00450010h] instruction.

To find out a value by the 0x00450010 address, open the Memory window (Debug -> Windows-> Memory-> Memory1).

At this stage, JIT has not created a required call node yet. Thus, the environment interprets a call of the interface method. It means that we can see a linear search of the required method at the runtime.

However, let’s run this code twice. After that, we will see that the value of the 0x0450010 address has been changed:

To check the 00457012 value, load SOS.dll:

Immediate window -> .load sos

!u 00457012
Unmanaged code
00457012 813908314400     cmp         dword ptr [ecx],443108h
00457018 0F85F32F0000     jne         0045A011
0045701E E9BD901D00       jmp         006300E0

The jmp 006300E0 instruction is a call to the required interface method. You can check it using this code:

!u 006300E0
Normal JIT generated code
ConsoleApplication1.FirstCallableImpl.DoSomething()
Begin 006300e0, size 1
>>> 006300E0 C3               ret

Well… Let’s figure out what the cmp dword ptr [ecx],443108h instruction compares.

!DumpMT 443108
EEClass: 00441378
Module: 00442c5c
Name: ConsoleApplication1.FirstCallableImpl
mdToken: 02000004  (C:\*path to project*\InterfaceStubsTest.exe)
BaseSize: 0xc
ComponentSize: 0x0
Number of IFaces in IFaceMap: 1
Slots in VTable: 6

Compare this to the FirstCallableImpl type (i.e. MethodTable). If it is true, call the FirstCallableImpl.DoSomething() method.

The jne 0045A011 instruction is fallback to a linear search (as it has been before caching).

When it comes to calling the next type – SecondCallableImpl, it will still be checked in the call-site exactly FirstCallableImpl, rather than SecondCallableImpl.

Still, it is not effective! Thus, after several iterations of code invocations, the environment will exchange the node with cache for a linear search.

Thus, caching is quite effective when we call methods of collections, for example.

Generic types stubs

The release of CLR 2.0 along with generics led to important changes in the runtime environment. Before we needed only the EEClass structure to describe a particular type. Now, we need binding EEClass+MethodTable.

Moreover, List and List will have different EEClass.

Consider an example:

class Program
{
    static void Main(string[] args)
    {
        var refTypeHolder = new HolderOf<object>(null);
        var intTypeHolder = new HolderOf<int>(0);

        // call JIT
        refTypeHolder.GetPointer();
        intTypeHolder.GetPointer();

        Console.Read(); // place breakpoint
    }
}

class HolderOf<T>
{
    private readonly T _pointer;

    public HolderOf(T pointer)
    {
        _pointer = pointer;
    }

    public T GetPointer()
    {
        return _pointer;
    }
}

To check it, we will use the !dumpheap command:

.load sos.dll

!dumpheap -type HolderOf
PDB symbol for mscorwks.dll not loaded
 Address       MT     Size
02d332c8 00f531e0       12     
02d332d4 00f53268       12     
total 2 objects
Statistics:
      MT    Count    TotalSize Class Name
00f53268        1           12 ConsoleApplication1.HolderOf``1[[System.Int32, mscorlib]]
00f531e0        1           12 ConsoleApplication1.HolderOf``1[[System.Object, mscorlib]]
Total 2 objects

As you can see, the environment has created two different specializations of the HolderOf<T> class:

!dumpmt -md 00f53268 (HolderOf<int>)

!dumpmt -md 00f53268
EEClass: 00f514cc
Module: 00f52c5c
Name: ConsoleApplication1.HolderOf``1[[System.Int32, mscorlib]]
mdToken: 02000006  (C:\*path to samples*\InterfaceStubsTest.exe)
BaseSize: 0xc
ComponentSize: 0x0
Number of IFaces in IFaceMap: 0
Slots in VTable: 6
--------------------------------------
MethodDesc Table
   Entry MethodDesc      JIT Name
66ae6a30   66964968   PreJIT System.Object.ToString()
66ae6a50   66964970   PreJIT System.Object.Equals(System.Object)
66ae6ac0   669649a0   PreJIT System.Object.GetHashCode()
66b57940   669649c4   PreJIT System.Object.Finalize()
00f5c088   00f53250      JIT ConsoleApplication1.HolderOf``1[[System.Int32, mscorlib]]..ctor(Int32)
00f5c090   00f5325c     NONE ConsoleApplication1.HolderOf``1[[System.Int32, mscorlib]].GetPointer()

!dumpmt -md 00f531e0 (HolderOf<object>)

!dumpmt -md 00f531e0
EEClass: 00f51438
Module: 00f52c5c
Name: ConsoleApplication1.HolderOf``1[[System.Object, mscorlib]]
mdToken: 02000006  (C:\*path to samples*\InterfaceStubsTest.exe)
BaseSize: 0xc
ComponentSize: 0x0
Number of IFaces in IFaceMap: 0
Slots in VTable: 6
--------------------------------------
MethodDesc Table
   Entry MethodDesc      JIT Name
66ae6a30   66964968   PreJIT System.Object.ToString()
66ae6a50   66964970   PreJIT System.Object.Equals(System.Object)
66ae6ac0   669649a0   PreJIT System.Object.GetHashCode()
66b57940   669649c4   PreJIT System.Object.Finalize()
00f5c068   00f53154      JIT ConsoleApplication1.HolderOf``1[[System.__Canon, mscorlib]]..ctor(System.__Canon)
00f5c070   00f53160     NONE ConsoleApplication1.HolderOf``1[[System.__Canon, mscorlib]].GetPointer()

In the above-mentioned dump, we are interested in HolderOf<T>.GetPointer(). Consider:

!dumpmd 00f5325c (HolderOf<int>.GetPointer())

!dumpmd 00f5325c
Method Name: ConsoleApplication1.HolderOf``1[[System.Int32, mscorlib]].GetPointer()
Class: 00f514cc
MethodTable: 00f53268
mdToken: 0600000b
Module: 00f52c5c
IsJitted: yes
CodeAddr: 01090318

!dumpmd 00f53160 (HolderOf<object>.GetPointer())

!dumpmd 00f53160
Method Name: ConsoleApplication1.HolderOf``1[[System.__Canon, mscorlib]].GetPointer()
Class: 00f51438
MethodTable: 00f53178
mdToken: 0600000b
Module: 00f52c5c
IsJitted: yes
CodeAddr: 010902b8

Please notice that there is no System.Object for Holderof. Instead, we have System.__Canon

[Serializable()]
[ClassInterface(ClassInterfaceType.AutoDual)]
[ComVisible(true)]
internal class __Canon
{
}

In short, it is usually considered that for reference types, the environment uses the System.__Canon type code sharing. But that’s not the point. Really.

The fact is that generic types may contain circular dependencies to other types which may lead to creating unlimited specializations of the code. For example:

Generics cyclomatic dependencies

class GenericClassOne<T>
{
    private T field;
}

class GenericClassTwo<U>
{
    private GenericClassThree<GenericClassOne<U>> field
}

class GenericClassThree<S>
{
    private GenericClassTwo<GenericClassOne<S>> field
}
class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine((new GenericClassTwo<object>()).ToString());
        Console.Read();
    }
}

However, this code won’t fail, but rather output GenericClassTwo`1[System.Object].

So, what about dependencies?

Type loader scans each generic type to find circular dependencies and assigns the order of priority – LoadLevel for a class. Though System.__Canon serves as a type argument for all specializations for reference types, it is a consequence, rather than a cause.

ClassLoadLevels:

enum ClassLoadLevel
{
    CLASS_LOAD_BEGIN,
    CLASS_LOAD_UNRESTOREDTYPEKEY,
    CLASS_LOAD_UNRESTORED,  
    CLASS_LOAD_APPROXPARENTS,
    CLASS_LOAD_EXACTPARENTS,
    CLASS_DEPENDENCIES_LOADED,
    CLASS_LOADED,
    CLASS_LOAD_LEVEL_FINAL = CLASS_LOADED,
};

For SSLCI (Rotor), The code responsible for scanning is in the file sscli20/clr/src/vm/Generics.cpp:

Generics.cpp

BOOL Generics::CheckInstantiationForRecursion(const unsigned int nGenericClassArgs, const TypeHandle pGenericArgs[])
{
    CONTRACTL
    {
        NOTHROW;
        GC_NOTRIGGER;
    }
    CONTRACTL_END;
    
    if (nGenericClassArgs == 0)
        return TRUE;


    _ASSERTE(pGenericArgs);


    struct PerIterationData {
        const TypeHandle * genArgs;
        int index;
        int numGenArgs;
    };
    
    PerIterationData stack[MAX_GENERIC_INSTANTIATION_DEPTH];
    stack[0].genArgs = pGenericArgs;
    stack[0].numGenArgs = nGenericClassArgs;
    stack[0].index = 0;
    int curDepth = 0;


    // Walk over each instantiation, doing a depth-first search looking for any
    // instantiation with a depth of over 100, in an attempt at flagging 
    // recursive type definitions.  We're doing this to help avoid a stack 
    // overflow in the loader.  
    // Avoid recursion here, to avoid a stack overflow.  Also, this code
    // doesn't allocate memory.
    while(curDepth >= 0) {
        PerIterationData * cur = &stack[curDepth];
        if (cur->index == cur->numGenArgs) {
            // Pop
            curDepth--;
            if (curDepth >= 0)
                stack[curDepth].index++;
            continue;
        }
        if (cur->genArgs[cur->index].HasInstantiation()) {
            // Push
            curDepth++;
            if (curDepth >= MAX_GENERIC_INSTANTIATION_DEPTH)
                return FALSE;
            stack[curDepth].genArgs = cur->genArgs[cur->index].GetInstantiation();
            stack[curDepth].numGenArgs = cur->genArgs[cur->index].GetNumGenericArgs();
            stack[curDepth].index = 0;
            continue;
        }
        
        // Continue to the next item
        cur->index++;
    }
    return TRUE;
}

For CoreCLR, the code changed towards ООP.

Thus, reference types have code sharing, while value types don’t. Why? After all, if it comes down to the size of the type (ref is the word size, In32 is 4 bytes, double is 8 bytes, etc.), then we can share DateTime’ and long’ specialization.

First, it is incorrect from the point of semantics. Second, the developers of CLR decided not to do this.

Generic method stubs

We have analyzed the code specialization for generic types. But how to find single methods outside the class.

Consider an example:

Generic methods

class Program
{
    static void Main(string[] args)
    {
        var refTypeHolder = new HolderOf();
        
        Test(refTypeHolder);
        Test2(refTypeHolder);
        Console.Read();
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Test(HolderOf typeHolder)
    {
        for (int i = 0; i < 10; i++)
        {
            typeHolder.GetPointer<Program>();
        }
    } // place breakpoint

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void Test2(HolderOf typeHolder)
    {
        for (int i = 0; i < 10; i++)
        {
            typeHolder.GetPointer<object>();
        }
    } // place breakpoint
}

class HolderOf
{
    [MethodImpl(MethodImplOptions.NoInlining)]
    public void GetPointer<T>()
    {
        Console.WriteLine(typeof(T));
    }
}

At the breakpoint in the Disassembly window, you can see the following assembly code for method Test():

00000045  mov         ecx,dword ptr [ebp-3Ch] 
00000048  mov         edx,10031B8h 
0000004d  cmp         dword ptr [ecx],ecx 
0000004f  call        FFE8BF40

As for Test2(), consider the following:

00000045  mov         ecx,dword ptr [ebp-3Ch] 
00000048  mov         edx,1003574h 
0000004d  cmp         dword ptr [ecx],ecx 
0000004f  call        FFE8BE40

The ECX register contains a pointer to this (calling convention — FastCall). However, GetPointer() has no arguments. In this case, what is written into EDX?

Let’s check:

!dumpmd 10031B8 (from Test())

!dumpmd 10031B8
Method Name: ConsoleApplication1.HolderOf.GetPointer[[ConsoleApplication1.Program, InterfaceStubsTest]]()
Class: 01001444
MethodTable: 01003118
mdToken: 0600000e
Module: 01002c5c
IsJitted: no
CodeAddr: ffffffffffffffff

!dumpmd 1003574 (from Test2())

!dumpmd 1003574
Method Name: ConsoleApplication1.HolderOf.GetPointer[[System.Object, mscorlib]]()
Class: 01001444
MethodTable: 01003118
mdToken: 0600000e
Module: 01002c5c
IsJitted: no
CodeAddr: ffffffffffffffff

Bingo! MethodDesc structure is passed, which contains a pointer to MethodTable (note: both descriptors point to the same MethodTable 0x01003118) and serves as a metadata source.

Thus, when calling generic methods, an additional parameter is passed with MethodDesc.

The addresses FFE8BF40 and FFE8BE40 themselves are the traps that forward the real specialized (for int, object, etc.) native code.

Due to fact that the descriptor itself also stores generic parameters, we gained in the reduction of the number of arguments passed, especially in the case of several generic parameters like Some<T, TU, TResult>(), for example.