Binary Ninja Recipes #2

This time I want to explore two problems I saw before while writing plugins for Binary Ninja. First problem, steaming from development of Annotator plugin (and I need to implement what I've learned here) and second is influenced by paper on Static Analysis and theory of lattices (if of course I've understood it correctly).

Problem 1: walk the graph

Let's say we want to track the state of a given variable in a program and all standard methods (get_rag_value_at(), SSA) don't apply. Good example is my plugin where I want to track instructions that influenced given variable (in that case function argument). OK, I'm cheating a bit here in a sense that I haven't yet tried SSA approach - more about that in next time. For now, let's get back to our problem at hand.

In BinaryNinja blocks of a given function are available through an array. Let's take a look at the example bellow

So now we access those blocks programmatically: After running this code we get following result:

1 -> 0x804840bL
2 -> 0x8048450L
3 -> 0x8048435L
4 -> 0x8048455L

I guess you can see the problem right away. When we iterate over blocks we get them sequentially,but not exactly in the order that actual code might execute. Here, we will be processing blocks 2 and 3 after each other, while actually they will never be executed in the same code run (I'm assuming you are reading this in bright future where speculative execution bug was addressed once and for all). Truth be told, all functions parameters should be placed on the stack/in registers in the same block the function is being called, but there is absolutely no guarantee about that. I wasn't sure about that so I've asked Gynvael, to which he responded - "well, for sure it will happen in the same function ...". Thanks buddy.

Fortunately it shouldn't be that difficult to fix that. Well, for certain definition of fix. As you can see there is a simple recursive descent function tracking visited blocks. We are also taking advantage of nifty API feature where every block has incoming and outgoing edges that actually point to other blocks.

So, does this work? Of course it does. Does it solve all problems? No and here is why. This code will work well for all functions with linear flow (with just conditional statements). Things get bit hairy when we introduce blocks with, what BinaryNinja calls back edges. In simple terms - loop statements.

Problem 2: find all paths

So it happened - we hit the loop condition. Like one here

Checking blocks again...

1 -> 0x804840bL
2 -> 0x8048444L
3 -> 0x8048425L
4 -> 0x804844aL

If we use our recursive descent we get two paths: 1->2->3 and 1->2->4. We clearly see this is incorrect and reason for that is condition preventing from revisiting a block that we have already visited. We should be getting 1->2->4 and 1->[2->3->2]*->4 (simplified to 1->2->3->2->4). So, now we know, that blocks can be revisited. What shouldn't be revisited? Of course, edges. Take a look a the code. I think code explains itself pretty well, so there is no point linger too long around it. You might wonder why I'm making local copy of visited edges list. It is fairly simple - in the example bellow you can see that we branch in block 2 and one call stack of walk() is using edge 2->4 and later on, another call stack needs to take this edge again. If I keep single list of visited edges my search does terminate on block 2 missing last step of a path. Fun func: as I was told yesterday, and I blame my poor knowledge of CS I've just reinvented a variant ofrecursive DFS algorithm.

I have tested this code of relatively simple samples so if you have something more complex and it breaks horribly please let me know. Right now I'm just hoping someone will find it useful and I haven't spent my evening doing poor's man implementation of SSA. Well, only one way to find out. See you soon :)

Nobody expected 64 bits

Apparently if you are not mortally embarrassed by the quality of your code you are releasing it too late [(tm) Silicon Valley]. But to use another only-too-often-used-quote - "Release early, release often". I've made mistakes of hoarding my tools and code for too long, not releasing them because they weren't perfect. This was obviously road to nowhere because if I don't release, nobody uses it. And if nobody uses it I have no motivation to develop it anymore. So, to break this circle I present you a new version of function Annotator for Binary Ninja.

First thing worth mentioning is a new database of functions prototypes. To be exact we now have 4728 prototypes. From this place big thanks to Zach Riggle for his functions project - this update would not be here if not for him.

Next thing is virtual stack for x64 platform - from now on you can also annotate 64-bit applications for Intel/AMD processors.

One small thing that I still need to properly implement is full support for functions operating on floating point types (float, double and long double). Right now they are not properly annotated and there are two important reasons for that:
For 32 bit platform floating point arguments are pushed on the stack using instructions like fstp or fst. Sadly, Binary Ninja right now does not have a corresponding Low Level Instructions for those. They are just showing as unimplemented(). The moment Binary Ninja starts supporting them I just add some more parsing code and everything will be fine.
64 bit platform is slightly more complicated. First of all, arguments to functions are passed via registers. Integers, pointers and such are passed through 6 registers - RDI, RSI, RDX, RCX, R8 and R9 and order matters. Floating point arguments are passed via XMM0-7 registers. Now, let's imagine that we have two functions f1(int, float) and f2(float, int). What will compiler do? Well, on Linux, in case of f1() first argument will end up in RDI and second in XMM0, but in f2 first argument will end up in XMM0 and second one in RDI.
"Wait a minute" - you will say - "but this is exactly the same". I'm glad you are seeing the same problem. Just having state of registers won't tell us what the first and the second argument is unless you know types in the first place. Virtual Stack does not know types, so until I refactor my code FP types won't be supported.

New updates are planned so stay tuned! And of course, please let me know what you think about it and report all bugs.

Migrating repository

Because code.google.com will be finally deprecated really soon I've moved all my projects to github. That includes JSONDecoder.

MutProxy

Recently I had very little time to write anything meaningful. New post are coming, slowly but steady. In the meantime I've stumbled upon short code at Gynvael page. It reminded me of a project I wrote some years ago for one assessment.
When I finally found it the code wasn't in state where I'd like to show it to anyone. Past few days I've spent cleaning and expanding it a bit. Today I've pushed code into GitHub. Here, take a look.

So, what MutProxy does? (Yep, I know that name is not very original nor brilliant, but come on, I'm not a Junior Creative Director in D'Arcy, I'm just a plain pentester.) It's just a simple proxy/tunnel with ability to attach functions to alter or log traffic in different ways. ReadMe does not exists at the moment, so you will have to read the code to determine functionality. There is some documentation in code comments :).

A lot of work still to be done - mutators are very basic and act more as an example then real deal, logger is very plain and documentation does not exist. Waiting for more free time. I was also planning to write more how to force applications to go through your proxy.

Small update

This is going to be very short (let's call it a warmup) post.
Just wanted to let you know that I've made small update to JSONDecoder. Changes are mostly cosmetics:

• Content type check is case insensitive now
• Decoder is now removing garbage from JSON payload (like }]);)
• Another Content-type is being checked: text/javascript (twitter uses that)

More stuff soon.

JSON Decoder

Long time no see. Usually people start such notes with oh-so-cliche quote from Mark Twain, but I've already did that on numerous occasions, so no. Anyway, despite the hidden motto of this blog ("no promises, it will be released when it's done") I wrote something. Finally, yesterday I've overcome my pathological laziness and finished version 1 of very small Burp Extensions - JSON Decoder. Code itself is not very impressing, nor is the functionality, but it's a start - now, knowing the basics I can move to more impressive stuff.

The Extension

Since version 1.5.01 Burp Suite Pro comes with new API for writing extensions. No longer you need to write them in Java, bundle into JAR and are forced to do some mojo magic to make them run. New API also gives you access to much more of the Burp internals. I'm not going to give you a tutorial how to write them, but I encourage you to read some of official tutorials on PortSwigger blog. If I see correctly there are eleven tutorials covering quite wide selection of topics.

So, what is my extension doing? Not that much (at least in this version) - it's just an additional tab with pretty printed JSON packet. I have other plans for that but I need to find time (and I've started flying BMS 4.32 again, so no rest for the wicked). I have some others extensions as a work in progress, but they are not in the ready-to-show state.

Debugging

Debugging burp extension is a bit like "Why? Because Fuck You, that's why" experience. You have made a typo, mixed expected type or declared too many parameters in function definition? All you get is JavaRuntimeException. You think that you won't made those mistakes? Let me show you what kind of mistakes I did while coding this extension.

Typos - I've spend 30 minutes failing to spot the difference between CreateTxtEditor() and createTxtEditor(). While writing an extension make sure that every API function follows CamelCase conventions (it can be tricky, because python names are usually flat). For example you can convert byte[] data variable in two ways - burp.helpers.byteToString(data) or data.tostring().

Difference between Java.String and byte[] - some functions accept byte[], some String - always check which type function expects and what it returns. It will save you time spent inserting countless println() lines.

Given the low complexity of my code I was able to use oldest, print everything technique of debugging, but if you are writing something more complex please read this blog entry.

Bit more about Burp stuff

If you are a new to Burp I can recommend a book written by my friend - grab it here. You can read it yourself or give to that new Junior Pentester that just joined.

Small and vulnerable webapp

The problem

You are in the plane, 11000 meters above the sea level, traveling 900kmh. And suddenly (usually after a bottle of wine) you have this brilliant idea about a bug in the browser, new way to filter some data or really anything that just requires writing a webapp. But you are in the plane, having the company machine running the OS_of_not_so_much_your_choice. Not always you have an Apache server wit PHP running on your laptop (well, you really should not have), VMplayer/VMFusion/anyVM is probably even less common. So, maybe you can use Django or Ror (or J2EE+Tomcat+JBoss - and if you say yes to it this is not the blog you are looking for). Anyway, you still want to code something.

So, here comes the Bottle. Repeating after web page - Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library.

Those two sentences neatly summarize all the things I like in it - just one file, no much of a setup needed and development effort is reduced to minimum. Thanks to that you can spent time solving real problems, not struggling with weird Apache vhost config file and wondering why the hell mod_php is not working.

Crash course

I know - talk is cheap, so show me the code. Instead of just pasting some code from the tutorial we can try solving some semi-real problem. Let's get back to my previous post - Using burp in a smart way where we were trying to figure it out how to fuzz for XSS vulnerabilities. To see how Burp behaves in different situation we would need some vulnerable script.

#!/usr/bin/env python

import re
import traceback

try:
  from bottle import run, request, template, route
except:
  print traceback.format_exc().splitlines()[-1]

head = "<html><title>Simple search interface</title><body>"
footer = "</body></html>"
filters = [r'!.*',r'[^a-zA-z0-9]*'] 
 
@route('/show')
def show_patterns():
  t=""
  for p in patterns:
    t+="<p>"+p+"</p>"
 
  return head+t+footer
 
@route('/search/:id')
def index(id):
  q = request.query.q
  t = "<p>You have searched for {{!query}} "
  t+= "and I've applied following filter - {{filter}}</p>"
  t+= "<p>Sadly, nothing was returned</p>"
 
  try:
    f = filters[int(id)]
  except:
    f = filters[0]
   
  return template(t, filter=f, query=re.sub(f,'',q))
 
run(host='localhost', port=8000)

Idea - application echoing back user search query and simple switch that will either show regexp patterns used to filter it or apply one of them.

I'll explain it in more details.

try:
  from bottle import run, request, template, route
except:
  print traceback.format_exc().splitlines()[-1]

First, import couple of functions from bottle framework - for this small program you will need only some functions to route request, get parameters from query, run template and run test webserver. Bear i mind that this is very simple web server and it's not suited to be exposed to the world.
You can safely ignore try..except block construction - I'm just trying to inform you, that you are missing bottle library.

Let's handle our first request:

@route('/show')
def show_patterns():
  t=""
  for p in patterns:
    t+="<p>"+p+"</p>"
 
  return head+t+footer

Most important thing here is a function decorator @route. I hope you are familiar with python function decorators - if not this is simply a function which wraps around other function - (over)simplifying you can treat it as a condition upon which the inside function will run (for all CS degree people - I don't care for formal decorator definition).
So, if we make an request to URL /show the function show_patterns() will run. Inside this function we only enumerate filters inside our script - we glue it together with header and footer then spit it out by return function.

That was really easy. Now time to analyze next function.

 
@route('/search/:id')
def search(id):
  q = request.query.q

Again, important thing is our @route decorator. Take a look at this :id thing - it just an element that will be matched dynamically - whatever you put after /search/ will get translated into argument for your function. Of course we need to grab argument from the query string (be careful, it's bit tricky - GET parameters are in requst.query, but POST requests are in request.forms), hence the q assignment.

Now, let's construct main body of the template file

 
  t = "<p>You have searched for {{!query}} "
  t+= "and I've applied following filter - {{filter}}</p>"
  t+= "<p>Sadly, nothing was returned</p>"
  [..]
  return template(t, filter=f, query=re.sub(f,'',q))

Bottle can use multiple template engines, but by default it uses Simple Template Engine. Two important things here - first, take a look at {{filter}} - it tells you that this is a place where you are going to put data while rendering template (by template() function). Second thing - template engine by default escapes all dangerous HTML characters - probably we don't want that, so precede it with ! character to disable that feature.

Being practical

I was using this program (well, actually I was not - I wrote it during 15 minutes break on the conference) writing previous article about burp fuzzing - you can use it as a testbed for both, testing some filters (still not perfect, but I can make better version later) and learn how to fuzz looking things with burp. Currently I'm using Bottle for both - writing small vulnerable things if I need to test some concept/attacks but also for some more serious projects - but let's save it for the next entry.

Summary

I know that this topic is not groundbreaking. Why writing then? Well, maybe because I want to show people that there is alternative to LAMP - you don't have to set up whole Apache + MySQL to create script with 5 lines of code just to test some simple case of anti-xss mechanism in a browser.