Intel Pin Tool

The most difficult part of doing research is prototyping. Especially when it’s about security, its a must one. A researcher has to prove the proposed system is legitimate. It’s true for both attack and defense. Researchers greatly depend on existing technology and software to implement their prototype. It cuts the development time to start from scratch. But on another hand, it is also complicated to work on others work. Most cases the technology is also a prototype from other researcher and presumably unstable, have not enough documentation and very limited feature. Luck sometimes favor for researcher too. Today I am going to discuss how I learned to use Intel Pin tool. It’s developed by Intel research team and well documented for reference. The only limitation is it is only usable for Intel architecture.

Intel pin tool is a great dynamic analysis tool. We can instrument the binary at runtime and collect information about execution. We can collect memory information, register value, which instruction is executing and so on. The main site is here: https://software.intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool. Here I will describe how I get familiar with this tool. I have used Linux distribution (Ubuntu) with Intel x86-64 architecture to install Intel pin 3.4 (https://software.intel.com/en-us/articles/pin-a-binary-instrumentation-tool-downloads). The manual will be available here: https://software.intel.com/sites/landingpage/pintool/docs/97438/Pin/html/.

Download the .tar.gz of Intel pin 3.4 from the above link. Untar it to any location. Inside of it, we will see a pin executive which is our tool. To demonstrate that our system works correctly, let’s start with building few example tools delivered with the source. Follow the following commands:

cd source/tools/ManualExamples/
make all TARGET=intel64
../../../pin -t obj-intel64/inscount0.so -- /bin/ls

So with these commands, we basically go inside of ManualExamples delivered with Intel pin tool and build all tools inside that targeting Intel x86-64 architecture. The last command will execute one tool named inscount0.so for the binary /bin/ls. The inscount0.so tool basically instrument all instruction /bin/ls to count how many instructions being executed. The result will be stored in inscount.out file. If the test is successful, we will have that output file and inside we have how many instructions being executed. Let’s try to do the same last command with a different argument for /bin/ls.

../../../pin -t obj-intel64/inscount0.so -- /bin/ls ..

So now /bin/ls executed with pin tool inscount0.so for path .. from the current position.

This is a good time to look at the tool code to understand how it count number of instructions. Let’s open inscount0.cpp from source/tools/ManualExamples. So, this is our source code and our make command create the .so file inside obj-intel64/ which we execute with Intel pin. Here is the source code:

/*BEGIN_LEGAL 
Intel Open Source License 

Copyright (c) 2002-2017 Intel Corporation. All rights reserved.
 
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.  Redistributions
in binary form must reproduce the above copyright notice, this list of
conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.  Neither the name of
the Intel Corporation nor the names of its contributors may be used to
endorse or promote products derived from this software without
specific prior written permission.
 
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE INTEL OR
ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
END_LEGAL */
#include <iostream>
#include <fstream>
#include "pin.H"

ofstream OutFile;

// The running count of instructions is kept here
// make it static to help the compiler optimize docount
static UINT64 icount = 0;

// This function is called before every instruction is executed
VOID docount() { icount++; }
    
// Pin calls this function every time a new instruction is encountered
VOID Instruction(INS ins, VOID *v)
{
    // Insert a call to docount before every instruction, no arguments are passed
    INS_InsertCall(ins, IPOINT_BEFORE, (AFUNPTR)docount, IARG_END);
}

KNOB<string> KnobOutputFile(KNOB_MODE_WRITEONCE, "pintool",
    "o", "inscount.out", "specify output file name");

// This function is called when the application exits
VOID Fini(INT32 code, VOID *v)
{
    // Write to a file since cout and cerr maybe closed by the application
    OutFile.setf(ios::showbase);
    OutFile << "Count " << icount << endl;
    OutFile.close();
}

/* ===================================================================== */
/* Print Help Message                                                    */
/* ===================================================================== */

INT32 Usage()
{
    cerr << "This tool counts the number of dynamic instructions executed" << endl;
    cerr << endl << KNOB_BASE::StringKnobSummary() << endl;
    return -1;
}

/* ===================================================================== */
/* Main                                                                  */
/* ===================================================================== */
/*   argc, argv are the entire command line: pin -t <toolname> -- ...    */
/* ===================================================================== */

int main(int argc, char * argv[])
{
    // Initialize pin
    if (PIN_Init(argc, argv)) return Usage();

    OutFile.open(KnobOutputFile.Value().c_str());

    // Register Instruction to be called to instrument instructions
    INS_AddInstrumentFunction(Instruction, 0);

    // Register Fini to be called when the application exits
    PIN_AddFiniFunction(Fini, 0);
    
    // Start the program, never returns
    PIN_StartProgram();
    
    return 0;
}

This is a C code written using Intel pin library (pin.H).  It is always the best idea to start exploring code from main method. In the main method, we can see a conditional check using PIN_Init(argc, argv) which validate the command line argument if there are in proper order or not. It also initiates Intel pin system (Pin provides its own locking and thread management API’s, which the Pintool should use).

Next, we have opened a file “inscount.out” to write. If we look back how we declare the filename variable KnobOutputFile, it will look weird at first why we create things complicated for such case. KnobOutputFile is a type Knob defined within Intel pin to automate the parsing and management of command line switches. Just consider, we want to use a different filename in different test execution; the best way is sending the filename with the command line. Because of the knob, we can now send filename in command line like as follows:

../../../pin -t obj-intel64/inscount0.so -o pininscount.out -- /bin/ls

So, the instruction count now will be stored in pininscount.out instead of inscount.out. The only difference we include -o filename here, and even we don’t need to make any change in the source code to overwrite the default filename. So, how pin parse the filename if we want to send such multiple information from the command line. Look closely at the declaration of KnobOutputFile.

KNOB<string> KnobOutputFile(KNOB_MODE_WRITEONCE, "pintool", "o", "inscount.out", "specify output file name");

The first argument is KNOB_MODE_WRITEONCE  which means we want to open the file to write once. Next, we have “pintool” which mentions which family it is supposed to be included; and then we have “o” which works as a flag suggest pin tool that in command line whatever after -o will be the desired string for this variable. Look how we replace the default filename last time (-o pininscount.out). The last argument just gives a suggestion the user what he is missing providing in command line if that field is required. For this case, we provide a default filename “inscount.out” as the fourth argument.

Again, come back to our main method.  Next, we have INS_AddInstrumentFunction() where we send our instrumentation callback method as the first argument and the second argument is a variable length argument where we can send argument for the instrumentation method. In this case, we send Instruction as instrumentation callback method and this method does not require any argument; so we send 0 in the second argument.

Let’s have a look at our instrumentation callback method, Instruction. Typical instrumentation callback method should be looked like following:

typedef VOID(* LEVEL_PINCLIENT::INS_INSTRUMENT_CALLBACK)(INS ins, VOID *v)

So, our Instruction method as the similar signature with (INS ins, VOID *v). The first argument ins is the targeted instruction which is going to be instrumented. Inside our callback method, we have INS_InsertCall() call. The first argument of this call is the target instruction, here ins. Then we have IPOINT_BEFORE, which is a pin flag. This flag means instrument before the target instruction. This is a really important part of instrumentation and wisely need to use for different purposes. Right now, just keep it simple like we can instrument before or after instruction or based on target instruction evaluation (like conditional instruction; it could be taken or not taken). Next, we have a function which should be called before executing the target instruction, here it is named as docount. We have to cast it to (AFUNPTR); it’s a defined return type for such method in pin system to match any return type for the method. Later, we can pass arguments to our docount method if required; but we don’t so we indicate it with IARG_END. The docount method only increments a counter to track how many instructions being executed.

Come back again to the main method, we have another call to PIN_AddFiniFunction() similar to previous one. We send another callback method Fini; which will be called just before application exit. Our Fini method do the file write in the usual way and close it.

At last, we have PIN_StartProgram() which will start executing the target binary with full preparation to instrument all instruction before they executed to count how many instructions are executed. That is a good example to introduce with Intel pin tool.

Let’s try to build our own first tool. Consider we like to know what an instruction read from a global variable. We know for Intel x86-64 architecture, in instruction global variables are represented with memory address based on RIP register. A regular instruction read from a global variable will look like following:

mov r9, qword ptr [rip+0x217cad]
mov r9, qword ptr [rip+0x217cad]
mov rdi, qword ptr [rip+0x217af6]
mov rax, qword ptr [rip+0x217c31]
mov rax, qword ptr [rip+0x217c31]
mov rsi, qword ptr [rip+0x217e23]
mov rax, qword ptr [rip+0x394279]
mov eax, dword ptr [rip+0x217e55]

As a first step, we create a directory under “pin/source/tools/” named “dreamlandcoder”. Now we copied two files from the “pin/source/tools/ManualExamples/”; files are “makefile” and “makefile.rules”. Next, we edit our “makefile.rules” to change only this variable “TEST_TOOL_ROOTS”. Previously it was assigned to source names as comma separated list, but as now we have only one tool and we named our source file as “globalTrace.cpp”; so we do:

TEST_TOOL_ROOTS := globalTrace

So, we open a new .cpp file named “globalTrace.cpp” where we will write our pin tool instructions. The complete code looks like following:

#include <iostream>
#include <fstream>
#include "pin.H"

ofstream outFile;

VOID ReadContent(ADDRINT *addr)
{
    ADDRINT value;
    PIN_SafeCopy(&value, addr, sizeof(ADDRINT));
    outFile << "Address 0x" << hex << (unsigned long long)addr << dec << ":\t" << value << endl;
    return;
}

VOID Instruction(INS ins, VOID *v)
{
    if (INS_IsMov(ins) && INS_OperandIsMemory(ins, 1) && INS_MemoryBaseReg(ins) == REG_RIP)
    {
        INS_InsertCall(ins,
                       IPOINT_BEFORE,
                       AFUNPTR(ReadContent),
                       IARG_MEMORYREAD_EA,
                       IARG_END);
    }
}

KNOB<string> KnobOutputFile(KNOB_MODE_WRITEONCE, "pintool",
                            "o", "global_dump.out", "specify output file name");

// This function is called when the application exits
VOID Fini(INT32 code, VOID *v)
{
    outFile.close();
}

INT32 Usage()
{
    cerr << "This tool dump heap memory information (global variable) ..." << endl;
    cerr << endl
         << KNOB_BASE::StringKnobSummary() << endl;
    return -1;
}

int main(int argc, char *argv[])
{
    // Initialize pin
    if (PIN_Init(argc, argv))
        return Usage();
    outFile.open(KnobOutputFile.Value().c_str());
    // Register Instruction to be called to instrument instructions
    INS_AddInstrumentFunction(Instruction, 0);
    // Register Fini to be called when the application exits
    PIN_AddFiniFunction(Fini, 0);

    // Start the program, never returns
    PIN_StartProgram();

    return 0;
}

We have most of the codes from our previous example except instruction check to instrument and Intel pin mechanism to read from memory. Let’s go through our new codes.

Start with Instruction() method where we can do a check of the instruction first, and then if it passed; do the instrumentation. In our last example, we don’t do any check; we just instrument all instructions. Our intention here is to be able to detect instructions that read from memory and that memory has a base which is basically RIP register. That’s also the basic properties of usage of global variables in the system. First, we do INS_OperandIsMemory checking operand 1 in ins instruction is a memory or not. Next, we do INS_MemoryBaseReg for ins instructor and check if the base register of memory is RIP register or not. We also have an INS_IsMov for ins instruction at the beginning which at first filter only move instructions.

When an instruction passed all these conditions, we should do instrument to that instruction to read the memory content. Now, as like the last example, we mentioned the action method here too which is ReadContent(). Now, this method needs to know the read memory address to perform the retrieval of the content from that memory. So, we send this piece of information with our INS_InsertCall() using IARG_MEMORYREAD_EA argument passing to ReadContent(). Look at the ReadContent() method as a parameter as ADDRINT which accept the memory address of reading memory address passed by INS_InsertCall().

Now, discuss what ReadContent() function does here for us. At first, we declared an ADDRINT to hold the memory content which we are just going to fetch from the memory address. Next, we have called PIN_SafeCopy() with passing the memory address and how many bytes we want to read from there beside where we expect the content will be written. Finally, we do writing down the information as:

memory_address memory_content

So, if we look back to our output file, we will have something like following:

Address 0x7f27a4b630a0:	149632
Address 0x7f27a4b61cf8:	3219913727
Address 0x7f27a4b61d00:	0
Address 0x7f27a4b630a8:	139807916161040
Address 0x7f27a4b61ca4:	10114392040684128785
Address 0x7f27a4b630c8:	139807916875776
Address 0x7f27a4b630d0:	139807916868768
Address 0x7f27a4b61e08:	0
Address 0x7f27a4b61e60:	140726958392976
Address 0x7f27a4b61fe0:	139808243855680
Address 0x7f27a4b61e08:	0
Address 0x7f27a4b630c8:	139807916875776
Address 0x7f27a4b630d0:	139807916868806
Address 0x7f27a4b630c8:	139807916875776
Address 0x7f27a4b630d0:	139807916869992
Address 0x7f27a4b61cb8:	4096
Address 0x7f27a4b61cb8:	4096

We can build the new tool in following steps:

make

make obj-intel64/globalTrace.so

and then we can run this tool with “/bin/ls” as like:

../../../pin -t obj-intel64/globalTrace.so -- /bin/ls

We will enrich this example more in next update.