Not Impossible : Storing C++ Virtual Method Pointers in C Callbacks. • memdump

A good friend challenged me while we where investigating vtables and various ways to create a reflection system without macros. He stated, “It is impossible to store a virtual method pointer in a C callback”. I decided to prove him wrong. Here, I present the result of this investigation.

Disclaimer : What you are about to see will hurt your eyes, kill your cat and trigger the C++calypse. It is intended as a learning ressource to better understand the underpinnings of the C++ language, not for production use! I am not responsible for the damages caused by this code.

More Disclaimers : Some C++ purists may get triggered by the use of puts(). Worry not, the reason I do not use std::coutis purely for debugging reasons. When I work onthese types of experiments, I spend a lot of time analyzing object dumps and reading the dissasembed executable. Using C++ methods add hundreds of lines or more to the debug information and bloats the resulting vtable layout dumps and IR outputs. That is why I use puts().

A Quick Look At the Vtable

To fully wrap our head around what is coming up, a good understanding of object layouts is beneficial. Lets take 2 simple classes as an example.

class Base {
public:
	virtual void print() {
		puts("Base Class");
	}
	virtual const char* get_base_msg() {
		return base_msg;
	}
	const char* base_msg = "All your base are belong to me.";
};

class Child : public Base {
public:
	virtual void print() override {
		puts("Child Class");
	}

	void child_print() {
		puts(child_str);
	}
	const char* child_str = "Not virtual print.";
};

The Child class is overriding the print() method. The get_base_msg() is virtual but not overridden, and the child has an unrelated method child_print().

The object layout looks like this.

         0 | class Base
         0 |   (Base vtable pointer)
         8 |   const char * base_msg

         0 | class Child
         0 |   class Base (primary base)
         0 |     (Base vtable pointer)
         8 |     const char * base_msg
        16 |   const char * child_str

At the offset 0, an 8 Byte pointer to the vtable is stored. The Child object also stores the parent member variable base_msg at offset 8. After that, you find the childs variable child_str. Note that non-virtual member functions are nowhere to be found in the object layout.

Pointer Patching and Accessing the Vtable

The idea of accessing the vtable directly stems from a technique calledPointer Patching. I read about it on the bitsquid development blog. It is used by “security researchers” and video game cheat engines. The idea is quite simple, you can read or write in the vtable of an object directly. This means you can change an object’s methods at runtime, if you so desire.

((void (*)(void))((long**)child)[0][0])();

This line is where the magic happens. Lets decompose it into digestible chunks.

void (*)(void)

If you are familiar with C callbacks, then you will have noticed this is how we declare them. This type represents a voidmethod pointer *that takes no arguments (void). You may not be aware we can declare a function that take no arguments like this : void do_something(void);. It is an old-school ISO C way of writing your declaration, rarely seen in C++ code.

((long**)child)[0][0]

This is how we access the vtable. It is deceptively simple. Be aware that vtable implementations are not enforced in the C++ standard, so compilers could in theory store the vtable some other way. In practice though, most compilers store a pointer to an array of method pointers at position 0 in your object layout.

We are casting our object pointer into just that, an array of arrays. On a 64bit system, pointers are the size of a long, so we conveniently use that type. The base array, child[] is the object and it’s member variables. We access the first element, always the vtable, and we can then access the methods stored at that location child[][]. In this example, we access the first virtual method.

((void (*)(void))((long**)child)[0][0])();

And so, here we cast our object pointer to a 2D array of type long**, we retrieve the pointer to the first virtual method child[0][0], and we cast that long pointer back into a function pointer and call it!

But there is one problem with our callback…

Finer Details

To understand what is wrong with the our vtable method call, we need to dig a little deeper into how member functions are actually called.

A member function is a glorified C function call. The C++ compiler will assemble methods in the TEXT section of your executable, but will prepend an object pointer (this) to the arguments. In essence, both of the following are similar.

class Child {
public:
	void print(const char* msg) {
		puts(msg);
	}
};

void print(Child* _this, const char* msg) {
	puts(msg);
}

By passing around the object pointer, member functions can access a class’s member variables.

Now, it is easy to see the problem with the pointer patching example. It will work if the method doesn’t use any member variables, but if it does, you will get a segfault, or access forbidden memory, or get error messages you have never seen before! :)

The fix is simple though, we need to prepend the object ourselves in our function callback type and pass the object pointer to the method, just like your C++ compiler would.

((void (*)(void*))((long**)child)[0][0])(child);

This will effectively call the first method in the objects vtable, and if it internally uses variables, no problem will ensue. Neat!

Storing Our Virtual Method in a C Callback

Most of the work is already done, but here is the result.

void (*c_callback_print)(void*) = nullptr;

/* ... */

c_callback_print = (void (*)(void*))((long**)child)[0][0];
c_callback_print(child);

And voilà, it works!

tl;dr; Not impossible.

That is all for today. I hope you enjoyed this ride down the rabbit-hole and learned a thing or two. I have posted a full example below, with some more ways of storing C callbacks. I recommend this good read on a supported callback feature to store non-static non-virtual member functions.

No Animals Were Hurt While Writing This Program

#include <cstdio>;

class Base;
class Child;

void (*c_callback_print)(void*) = nullptr;
const char* (*c_callback_base_msg)(void*) = nullptr;
void (Child::*c_callback_child_print_well_behaved)(void) = nullptr;
void (*c_callback_child_print_naughty)(void*) = { nullptr };

class Base {
public:
	virtual void print() {
		puts("Base Class");
	}
	virtual const char* get_base_msg() {
		return base_msg;
	}
	const char* base_msg = "All your base are belong to me.";
};

class Child : public Base {
public:
	virtual void print() override {
		puts("Child Class");
		puts(child_thor);
		puts(base_msg);
	}

	void child_print() {
		puts(child_str);
	}
	const char* child_str = "Not virtual print.";
	const char* child_thor = "I am Thor.";
};

int main(int argc, char* argv[])
{
	Child* child = new Child();

	/* Call method directly. */
	((void (*)(void*))((long**)child)[0][0])(child);

	/* Store the first vtabe method in a C callback. */
	c_callback_print = (void (*)(void*))((long**)child)[0][0];
	c_callback_print(child);

	/* It works on the base class methods too. */
	c_callback_base_msg = (const char* (*)(void*))((long**)child)[0][1];
	const char* msg = c_callback_base_msg(child);
	puts(msg);

	/* A non-hack way of storing non-virtual method callbacks. */
	c_callback_child_print_well_behaved = &Child::child_print;
	(child->;*c_callback_child_print_well_behaved)();

	/* Casting method pointers isn't allowed. Let's get crazy. */
	union member_pointer_cast {
		void (Child::*member_pointer)(void);
		void* void_pointer[2];
	} naughty;
	naughty.member_pointer = &Child::child_print;

	c_callback_child_print_naughty = (void (*)(void*))naughty.void_pointer[0];
	c_callback_child_print_naughty(child);

	delete child;
	return 0;
}

Use with care.