C++ - The value of ESP was not properly saved across a function call

No comments

C++ - The value of ESP was not properly saved across a function call

Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call.  This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

Another run time error that is almost impossible to quickly identify and debug. It usually occurs when libraries are involved and use different calling conventions e.g. _stdcall instead of _cdecl or other. But sometimes compiler misses a thing or two and compiles something that it shouldn't, after all, compiler only need to know how many bits and bytes to move and what do to with them to translate your program to an executable or a library. Sometimes it lets you do stupid things...

Lets take a look a the following code:

template<typename T>
class Base
        virtual ~Base() {};
        virtual std::vector<T> data() = 0;

template<typename T>
class Derived : public Base<T> 
        Derived() { _data.push_back(1); _data.push_back(2); };
        ~Derived() {};

    std::vector<T> &data() override {
        return _data;
        std::vector<T> _data;

int main()
    std::shared_ptr<Base<float>> a = std::make_shared<Derived<float>>();
    auto b = a->data();
    return 0;

At first look everything looks great. Standard C++ polymorphism with templates, everyday use. Just enough to get Run-Time Check Failure #0. The mistake, one that is easily missed, especially if project is broken down into multiple files, lies at line number 16.

virtual std::vector<T>  data() = 0; // Base
        std::vector<T> &data() override // Derived

The problem is missing & in front of data().

What is an ESP Register?

The ESP register serves as an indirect memory operand pointing to the top of the stack at any time. As program adds data to the stack, the stack grows downward from high memory to low memory. When items removed from the stack, stack shrinks upward from low to high memory.

In a simpler language, when you call a function or a method, address where to return and all of the parameters will be added to the stack. When function or a method returns, those values will be removed. ESP Register always points to the top of the stack therefore changes with removing and adding thing to the stack. If that values gets corrupted, program might override other variable values, jump to incorrect block of code or even data and start executing it.

How does the code above breaks the stack?

Simple. Polymorphism does the trick. With polymorphism, every derived class has it's own virtual table that points to overridden methods. Call to base->data()  really triggers a lookup in that table for a desired method and afterwards jumps there to execute that chunk of code. The return size is expected to be the same as specified in Base class, meaning a pointer to a vector of size 4 bytes of data or 8 bytes, if the program is compiled in 64-bit mode.

When derived->data() is executed (jump in virtual pointer table), to much data is removed from the stack (ESP register increased to much) because sizeof(vector<float>) is significantly larger than just a pointer to it causing Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call.

Leave a Reply

Your email address will not be published.

Time limit is exhausted. Please reload CAPTCHA.