Calling Conventions

The Calling Conventions in C/C++ are the guidelines that determine:

• How the arguments are passed onto the stack.
• Who will clear the stack, caller or callee?
• What registers will be used and how.

Syntax

The following syntax shows how to use the calling convention:

return_type calling_convention function_name {
    // statements
}

The C++ code gets converted to object code at the end of the compilation stage. Then we get the object file. The object files are linked together to create a binary file(exe, lib, dll).

Before the creation of the object file, we can tell the compiler to stop and give us the .asm file. This is the assembly file which will get converted to an object file. Different calling conventions produce different assembly codes. For GCC, the -S can be used for this purpose. Just pass this flag while compiling the code.

gcc -S sourceFileName.c

Different Calling Conventions

There are many calling conventions for different platforms. We are going to look at 32-bit x86 calling conventions.

1. __cdecl
2. __stdcall
3. __fastcall
4. __thiscall (C++ only)

In C/C++ programming, a calling convention is a set of rules that specify how a function will be called. You might have seen keywords like __cdecl or __stdcall when you get linking errors. For example:

error LNK2019: unresolved external symbol "void __cdecl A(void)" (?A@@YAXXZ) referenced in function _main

Here, we see __cdecl being used. It is one of the calling conventions in C/C++. You can also see code like this in 3rd party libraries.

For example:

extern __m128i __cdecl _mm256_mask_cvtepi32_epi16(__m128i, __mmask8, __m256i);

What are caller and callee?

When we call a function, a stack frame is allocated for that function, arguments are passed to the function, and after the function does its work, the allocated stack frame is deallocated and control is passed to the calling function.

The function that calls the subroutine is called the caller. The function that gets called(i.e. subroutine) by the caller is called the callee.

In the above code, main() is the caller and func() is the callee.