Motive

With a couple of my friends I recently organized nullcon’s HackIM ctf. I authored 0bfusc8 much An RE chall that had 11 solves during the CTF and I got positive reviews about it.

The idea was to keep the challenge simple enough yet tricky so that it can’t be solved directly with angr.

Background

If you’re not into reverse engineering/compilers you might need to look into these terms

Basic Block

A basic block(bb) is a piece of code in the program which only has one path of execution – a straight line. There will be no branches/loops inside a basic block except at the end which means if we execute the first line we will execute all lines in that basic block. This concept is not just in assembly but also compilers in the context of generated code, analyzed code and intermediate code.

Control Flow

A control flow graph defines the paths of execution in a program. The nodes are usually basic blocks in a CFG. To keep a picture in perspective

1int main(int argc, char **argv) {
2    if (argc == 2)
3        puts("two");
4    else
5        puts("not two");
6    return 0;
7}

Here’s how the cfg and bbs look

main was broken into 4 bbs – one entry, one exit and two for if/else. We can see this CFG has only 2 paths.

Opaque Predicate

An opaque predicate is a condition in a code whose value we know at compile time yet it is still calculated during run time. We’ll see how they’re useful in a moment.

LLVM

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. It is a great project to implement your own compilers or play around with static analysis of code. These libraries are built around a well specified code representation known as the LLVM intermediate representation (“LLVM IR”). LLVM IR makes it easier to write automated analysis by exposing APIs to change it at will.

LLVM Pass

When some C code is passed through a compiler it goes through a number of steps – parsing, lexing, semantic analysis, IR code gen, optimizations and native code gen respectively. LLVM as a project offers you to just write a frontend for a language of your choice which parses your specific language and emits LLVM IR. LLVM project then has all the proper backends and analysis/optimizations built in to generate a binary. Passes perform the transformations and optimizations that make up the compiler, they build the analysis results that are used by these transformations, and they are, above all, a structuring technique for compiler code.

If we want to instrument a program at compile time, a pass would be the best way to achieve that.

How make RE harder with an LLVM pass

When we analyze code usually the size and complexity of code determines the time/efforts needed. A function with a very complex cfg with a lot of paths is much harder to analyze and understand. Typically all obfuscators at a function level try to do that.

My idea was to use a lot of Opaque Predicates in the code and create paths that will never be executed. This will throw off the reverse engineer to do something else.

If we have this code

1int a, b, c;
2...
3a = b + c;
4...


This will usually be just a couple of instructions in assembly – normal stuff.

Idea 1

1int a, b, c;
2...
3int x = rand()
4a = b + x;
5a += c;
6a -= x;
7...


This will have multiple instructions in assembly. However an optimizing decompiler/code analyzer like IDA will detect this and show you version 1 when decompiled.

Idea 2

1int a, b, c;
2...
3s:
4...
5switch(b%2){
6    case 1: a = b + c; break;
7    case 0: a = b + c; break;
8    case 2: goto s;
9}
10...


In this case we create an opaque predicate: case 2 for b%2 which loops back to the start of the code. Now if we use these two ideas together we’ll have a very complex piece of code for a simple addition.

1int a, b, c;
2...
3s:
4...
5switch(b%2){
6    case 1: a = b + 0x333435;
7            c -= 0x333435;
8            a += c;
9            break;
10    case 0: a = b - 0x414243;
11            c += 0x414243;
12            a +=c;
13            break;
14    case 2: a = b + c;
15            goto s;
16}
17...


The cfg for this will have multiple branches and a loop which was just a basic block earlier. This increases the complexity of the code. Also if you see we have used addition/subtraction operations to obfuscate addition. If we do this obfuscation recursively this could even go upto a million basic blocks.

To do this I implemented a FunctionPass in llvm. Source was released for this pass here. We can use Clang to generate a bitcode – LLVM IR code for a C program and then run our pass over it. FunctionPass are run over every function in a program so you can efficiently instrument each function individually. To do this we implement a runOnFunction in our pass. Here are some portions of the code.

 1while(obfuscated < level){
 2    errs() << "Round : " << obfuscated << "/" << level << "\n";
 3    toDoAdd.clear();
 4    toDoSub.clear();
 5    toDoOr.clear();
 6    toDoXor.clear();
 7    for (Function::iterator bb = F.begin(); bb != F.end(); ++bb) {
 8        for (BasicBlock::iterator I = bb->begin(); I != bb->end(); ++I) {
 9            if (I->getOpcode() == BinaryOperator::Xor) {
10                auto &ins = *I;
11                toDoXor.push_back(&ins);
12                obfuscated++;
13                break;
14            }
15            if (I->getOpcode() == BinaryOperator::Add) {
16                auto &ins = *I;
17                toDoAdd.push_back(&ins);
18                obfuscated++;
19                break;
20            }
21            if (I->getOpcode() == BinaryOperator::Sub) {
22                auto &ins = *I;
23                toDoSub.push_back(&ins);
24                obfuscated++;
25                break;
26            }
27            if (I->getOpcode() == BinaryOperator::Or) {
28                auto &ins = *I;
29                toDoOr.push_back(&ins);
30                obfuscated++;
31                break;
32            }
33            if (obfuscated > level) {
34                break;
35            }
36        }
37        if (obfuscated > level) {
38            break;
39        }
40    }
41
42    for (auto &I : toDoAdd) {
43        AddSwitch(cast<Instruction>(I), addfnc);
44    }
45
46    for (auto &I : toDoSub) {
47        AddSwitch(cast<Instruction>(I), subfnc);
48    }
49
50    for (auto &I : toDoOr) {
51        AddSwitch(cast<Instruction>(I), orfnc);
52    }
53
54    for (auto &I : toDoXor) {
55        AddSwitch(cast<Instruction>(I), xorfnc);
56    }
57}

It iterates over all instructions in a function and checks if they are one of ^, | , +, -. Such operators are pushed in a set so that we can work on them later. Later we’ll replace each binary operator with a switch-case by calling AddSwitch which is where the actual magic happens.

 1// Adds a switch with a dummy loop, second param is a list of functions
 2// which replace an instruction
 3void AddSwitch(Instruction *I,
 4               Value *(*genfnc[3])(Instruction *, BasicBlock *)) {
 5    Value *op1 = I->getOperand(0);
 6    Value *op2 = I->getOperand(1);
 7
 8    auto F = I->getFunction();
 9    auto BB = I->getParent();
10
11    Type *type = I->getType();
12    IRBuilder<> *builder = new IRBuilder<>(I);
13
14    // A place on the stack to store values from all the paths in the
15    // switch.
16    Value *substitute =
17        builder->CreateAlloca(Type::getInt32Ty(F->getContext()));
18
19    auto two = ConstantInt::get(type, 2);
20    auto zero_c = ConstantInt::get(Type::getInt32Ty(I->getContext()), 0);
21    ;
22    auto one_c = ConstantInt::get(Type::getInt32Ty(I->getContext()), 1);
23    ;
24
25    BasicBlock *swDefault =
26        BasicBlock::Create(F->getContext(), "defaultCase", F);
27    BasicBlock *zeroCase =
28        BasicBlock::Create(F->getContext(), "zeroCase", F);
29    BasicBlock *oneCase = BasicBlock::Create(F->getContext(), "oneCase", F);
30
31    // Create op1%2 check
32    builder->SetInsertPoint(I);
33    auto checkCond = builder->CreateURem(op1, two);
34
35    // create switch for op1%2
36    builder->SetInsertPoint(I);
37    auto switch_main = builder->CreateSwitch(checkCond, swDefault, 2);
38    switch_main->addCase(zero_c, zeroCase);
39    switch_main->addCase(one_c, oneCase);
40
41    // split at the instruction and switch to create loop
42    auto N = BB->splitBasicBlock(I);
43    auto S = BB->splitBasicBlock(switch_main);
44
45    // default case. never hit. dummy loop
46    builder->SetInsertPoint(swDefault);
47    builder->CreateStore(genfnc[0](I, swDefault), substitute);
48    builder->CreateBr(S);
49
50    // use one of the obfuscators to generate a substitiue instruction
51    builder->SetInsertPoint(zeroCase);
52    builder->CreateStore(genfnc[1](I, zeroCase), substitute);
53    builder->CreateBr(N);
54
55    builder->SetInsertPoint(oneCase);
56    builder->CreateStore(genfnc[2](I, oneCase), substitute);
57    builder->CreateBr(N);
58
59    // really?
60    swDefault->moveBefore(N);
61    zeroCase->moveBefore(N);
62    oneCase->moveBefore(N);
63
64    // load the stack variable and replace the occurence of result with it
65    BasicBlock::iterator DI = N->begin();
66    Instruction &Inst = *DI;
67    builder->SetInsertPoint(&Inst);
68    Value *checker = builder->CreateLoad(substitute);
69    I->replaceAllUsesWith(checker);
70
71    // remove dummy jump and original instruction
72    S->getTerminator()->eraseFromParent();
73    I->eraseFromParent();
74}

Here we separate the current basic block at the binary operator and put a switch case instead. To save the calculated values in all paths consistent I had to add a local variable for each operation. This made the size of the stack enormous.

If we run this function over some program, it’ll add some basic blocks with more binary operators. We run this pass again over that code to increase the number exponentially. If we don’t control this it can get huge pretty quick resulting in binaries in 10s MBs of size.

Here’s a simple example on fibonacci function

‍

Normal

After

Tune in next time when we’ll see what kinds of approach can we take to solve such challenges.