2.2 Canaries (Canary words)

Canary words were initially implemented in StackGuard [9]. StackGuard is a compiler extension that enhances an executable code processed by the compiler (also used in ProPolice and Microsoft VisualStudio .NET). Canaries address space protection mechanism is used against buffer overflows which attempt to overwrite the return address [10]. A canary is placed as a metadata value (4 bytes) between the program and the control data on a stack to monitor and prevent the buffer overflow. The main purpose of Canary words is to protect the return address against overwriting.

Canary in its mechanism uses two functions called "epilogue" and "prologue" [11]. The function "prologue" introduces a canary word value onto the stack before the return address (Figure 2). Commonly, exploitation of stack buffer overflow includes overwriting the return address; if canary words are implemented-the value of the canary will be overwritten instead of the actual return address. The second function "epilogue" monitors the changes in the canary value and if the canary is changed or can not to be matched (before the return from the function which has been executed) - the program or process is forcibly terminated [12].

Figure 2: Canary words approach

There are a few general types of canaries which are used to stop stack overwriting:

1. "Terminator" canaries - can terminate execution of vulnerable functions (e.g.-strcpy(), getc());

2. Random Canaries - randomly generated (by entropy-gathering deamon) to prevent an attacker from obtaining its values. Generated at the program initialization, these values are kept in global variables. An attempt to read these values by using any kind of method prompts a segmentation fault which terminates the program or process. On the other hand, it has a serious vulnerability: an attacker may be able to obtain the current random canary value via exploiting the victim program [9];

3. Random XOR Canary - by expecting that an exploit may be able to gain a random write access to the location of protected data; some or all saved control data can be encrypted (XORed) with a random canary value. The resulting value is kept in the canary's location. This approach has the same vulnerability as Random Canaries have.

Despite the fact that Canary words can prevent the overwriting the return address, this technique has some drawbacks. Some examples of these drawbacks are:

- Local variables and saved frame pointer are not secure;

- Program can be modified until the functions return;

- Heap can be overflowed;

- Format string exploits can be used;

-Generally, when a canary modification is detected, the program is terminated that fact creates a DoS attack opportunity.

2.3 Address Space Layout Randomization (aslr)

It was initially integrated in PaX kernel patch for Linux OS (in 2001) and then formally introduced in Windows Vista, Windows 7 and Windows Server 2008 [13]. ASLR focuses on stack smashing types of buffer overflow attacks. The main function of ASLR is to add a certain amount of randomness into the layout of virtual memory space. It makes an attacking process more difficult as long as memory address keeps altering [14]. By randomizing the location allocated to the stack, heap, loaded *.dll files and executable binaries, ASLR decreases the probability that an exploit (usually relies on hardcoded addresses) will redirect a code execution. Obviously it creates a limitation; all executable binaries must be recompiled and relocated. ASLR can successfully protect stack against the return-to-LIBC and code injection attacks. The following addresses can be randomized:

- Executable images (1 of 255 random positions);

- DLL images (in the first 1of 256 positions of ntdll.dll, the other DLLs loaded after);

- Stack (has more positions than other sections);

- Heap (heap structure is located in 1 of 32 random positions);

- Process Environment Block (PEB) and Thread Environment Block (TEB).

ASLR can be applied to a system (for instance, in Windows OS it protects its processes only) and for those applications which optionally use the DYNAMICBASE linker flag. Linux kernel patch PaX uses more advanced ASLR techniques. It contains four main ASLR's components [12]:

1. RANDKSTACK - to introduce randomness into a task's kernel stack.

2. RANDMMAP - processes the randomization of all files and memory mappings.

3. RANDUSTACK - to randomize stack addresses.

4. RANDEXEC - to randomize the locations of ETC_EXEC ELF binaries.

Nowadays, ASLR is the most advanced technique designed to protect the stack against overflow attacks which are used through worm propagation etc. However, the ASLR approach has some disadvantages, for example:

- Return-to-Code attack is still possible to implement;

- Due to fact that Windows OS have the size memory page limitation (64 Kb), some memory sections can have less entropy after the randomization - this fact can be exploited through the use of a brute force attack;

- Some binaries cannot be recompiled (it is not provided by most distributions);

- ''Forked'' process uses the same layout as the main process (in case of object-oriented programming).