Reverse Engineering for Beginners (CH1.6:1.8 Function prologue and epilogue)

Function Prologue and Epilogue

---

The Function Prologue

It consists of a series of instructions at the beginning of the function.

And most of the time, it looks like this part of the code:

Assembly

push        ebp 
mov         ebp, esp
sub         esp, x

  

Let’s talk about this code step by step:

The first thing it does is that it pushes the value of ebp onto the stack.

Then, it makes the value of esp equal to ebp.

After that, it allocates space for the variable on the stack.

Keep in mind that the value of ebp remains the same during the execution of the function, and it’s used to access local variables and arguments.

For the same purpose, it’s also possible to use ESP, but since it changes over time, it’s inconvenient to use.

---

The Function Epilogue

Here, simply, it does exactly the opposite. In short, let me tell you:

It releases the space that was allocated in the stack, restores the EBP to its original state, and returns the control flow to the caller of the function:

Assembly

mov       esp, ebp
pop       ebp
ret          0

  

Usually, the prologues and epilogues of functions are recognized by disassemblers.

Also, prologues and epilogues can negatively affect the performance of recursive functions (those that call themselves).

Then the author started connecting this part with the previous examples we discussed earlier about Empty Function and Returning Values.

---

Let’s first look at the Empty Function example:

Here’s the example:

Assembly

f:
       push      rbp
       mov       rbp, rsp
       nop 
       pop       rbp
       ret

  

Here, the function’s prologue and epilogue were most likely not optimized.

And the NOP here is probably due to the compiler.

The only effective instruction here is RET.

---

Now let’s look at the Returning Values example:

Assembly

f: 
       push         rbp
       mov          rbp , rsp
       mov          eax , 123 
       pop          rbp
       ret

  

Here, the effective instructions are MOV and RET, and the rest are part of the prologue and epilogue.

---