Tuesday, November 24, 2009

A Use Case for Reverse Engineering

OK, so I'm going to try and motivate some of what I do with an example application. Here is the problem that I'm looking at. A few weeks ago, I had to put a little bit of code in an application to open up a web browser in Eclipse and display a web page. I had remembered doing exactly that many years ago, back during the transition from Eclipse 2.x to Eclipse 3.x when the browser became a standard component. I remembered it being really simple, but I couldn't remember for the life of me how to actually do it. I feel a little ashamed of that because it's something I should have known, but I didn't. So, I had to figure it out.

At this point, the entry might go long. So read if you are interested. Just quit otherwise.

Anyway, I started in the documentation. I did a search for "Browser". My problem is that I have so many plug-ins installed in Eclipse that help search is starting to get a little bit useless. I ended up with over 100 pages of documentation that had something to do with a browser, but I couldn't find what I was looking for. So, I decided to go to the Eclipse code.

So, I knew that if I browser was going to be opened, one had to be created. I was interested in using the internal browser, so I just went to the Browser widget, and set a breakpoint in the constructor to see who creates it. I started up a run-time workbench which had a project in the workspace containing an HTML file, and I opened it up. My breakpoint was hit, and I got this stack trace:




So, I started looking through the classes that were in there. What should I use? WebBrowserEditor? No, it's internal. WebBrowserEditorInput? Don't tell me that I have to construct an editor input, and pass it to the workbench just to get a web browser to open. No, I can't anyway since it is internal. I know that there is an easier way.

Anyway: long story short, I went down a rabbit trail of following inheritance hierarchies, and searching for references of different classes to see where they are created and accessed. I didn't have any real success, so I gave up on that train. I decided to give my Diver tool a try and see where it would take me. Note that this is a recreation of the event, and a number of the details are likely different.

So, to use Diver, I could just launch Eclipse in debug mode like I did before, but I did it using an Eclipse Trace Launch, supplied by Diver, as illustrated here:


There is a little more set up to do though. By default, Diver is set up to watch for code that I've written (i.e. the files in my source folders). But, I'm interested in Eclipse code, so I've got to change that. It can be changed in the Java Trace tab. I just guessed that the code that I was interested in was probably in a ui package, so I added org.eclipse.ui.* so that it would be included. Everything else will be ignored unless a class in an org.eclipse.ui package is accessed somewhere along the call chain:

Then, I just launched Eclipse like I normally would. I followed the same process as I did for my debug session, but instead of just stopping at the breakpoint, Diver is going to log all of the interactions that Eclipse makes around the time that I open the browser. I did that by using a little "play" button found in the Debug View: . Again, I opened the browser, and as soon as it started to show up, I paused the trace (), and quit Eclipse.

Now, it took a little while for Diver to analyze the data that it just captured. This is necessary to make Diver more zippy later on, so it's worth the wait. There were over 4 000 000 events that were logged. It didn't take too terribly long, though, and I did have other stuff to do.

Anyway, once that was done, I could see my trace in the Program Traces view:


This view shows all of the threads that were running while I was performing the trace. Double-clicking on one of them will open up a sequence diagram view that I could explore to figure out what the thread is doing. But, there were too many to try. So, I tried a different route. Notice the green dots? They mean that the trace associated with those threads is the "Active Trace" in Eclipse. It's kind of like Mylyn's tasks. If you activate a trace, then Diver will enable different features that will filter your package explorer to show you only the classes and methods that were used during the trace. I activated the trace that I just made, and went to the Package Explorer.

Again, I figured that what I was looking for probably happened around the same time that a Browser instance was created. The problem with using the debugger before was that I couldn't see everything that happened around that time: I could only see the things that occurred within the call chain to the construction of the instance. So, this time I used Diver to show me the context in which the browser was instantiated. I found it in the Package Explorer, right-clicked and selected "reveal in > main". What this did was it opened up a sequence diagram of the Main thread, and located the first method that was called on the Browser class:


Here lies another problem, though: the sequence diagram is really big. Here's a zoomed-out view of it:




I don't really want to look at it all. There are a couple of things that helped me out. First, there is a timeline on the sequence diagram:



The blue vertical bars are all the invocations of methods on the Browser class. The first one is the constructor call, so I adjusted the range on the timeline (the darker area around the first invocation) to just surround the constructor call a little bit. This filtered the sequence diagram to show only invocations that occur within that range of time. The other thing that helped me is the fact that the sequence diagram can collapse lifelines based on the packages that classes are contained in. For example, we can see here that the org.eclipse.swt.widgets package contains five classes that I'm not really interested in, so I can collapse it:



I collapsed a bunch of other uninteresting packages like java.* and sun.* and org.eclipse.core.*, etc., and got a much smaller diagram that I could start browsing. In a little while, I found another class that looked to be of interest: WorkbenchBrowserSupport. A method called createBrowser is called on it after the browser is actually instantiated. That's why I couldn't find it in the debugger:



Now, what I want to know is how to get a WorkbenchBrowserSupport, so I scanned up the lifeline, and I found this (cleaned up a little to make it look nicer):



At this point, I thought to myself, "Of course! You get the browser support by calling IWorkbench.getBrowserSupport()! I should have known that!" And I'm a little embarrassed to say that I didn't. But, my problem was solved at any rate. I just had to get an instance of the workbench (from PlatformUI), get the browser support, and create a browser. Done and done.

Now, this isn't to say that I solved a problem that couldn't be solved using other Eclipse tools. I'm sure that if I were willing to spend some time with the Java Search dialog, I could have found the reference to createBrowser eventually. But, I always get those search queries wrong and I come up empty. I'm just inept at using Java Search. So, here I had an alternative using some of the reverse engineering support of Diver.

I hope that this post motivated some of the neat things that can be done with a little bit of extra tool support. You can go ahead and try it if you like. It's all free.

That's it for now. I'll see you next time.

13 comments:

  1. Interesting way of doing reverse engineering. I generally weed my way through the Java Type Search dialog and hunt for references.

    Can you set your blogger settings so the full feed appears on planet.eclipse.org instead of the summary?

    ReplyDelete
  2. pretty interesting! I couldn't install diver though. Looks like the package is broken:

    Cannot complete the install because one or more required items could not be found.
    Missing requirement: 1259138571846 0.0.0.1259138571846 requires 'ca.uvic.chisel.diver.feature.feature.group [0.0.3.200911201202]' but it could not be found

    Is it?

    ReplyDelete
  3. Hi Chris,

    Sorry, I set it to "short" because I knew that this one was going to be long. I didn't want to fill up the planet page.

    Andre,

    The problem is that Diver only works on Windows currently. I'm working on fixing the problem as we speak. Are you using Linux? Building and testing for Linux is on the schedule for today.

    Del

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Chris, I've set the feed to long now. Will this post end up on the planet?

    ReplyDelete
  6. Hi Del

    Yes I run linux. I guessed that after reading your requirements twice (what is bound to windows? looks like the widgets dont). Looking forward to run it :-) Let me know if I can be of any help.

    Cheers
    André

    ReplyDelete
  7. Hi Andre, there is some native C++ code for interacting with the JVMTI. It's only in one fragment. It's been written so that it should hopefully be portable, but....

    ReplyDelete
  8. I would like to try on Mac OSX Snow Leopard. How can I check out code?

    Maarten

    ReplyDelete
  9. HI Maarten, you can go to the sourceforge site (sourceforge.net/projects/diver) and check everything out of SVN. The repository URL is https://diver.svn.sourceforge.net/svnroot/diver

    The native code is in ca.uvic.chisel.tracing.jvmti. If you can get that to compile on Snow Leopard, we are more than half way there. Note that the project is currently unstable because I'm trying to get it to build for Linux as well. I don't have a "build machine" here, so the build system is pretty ad-hoc right now, as you'll probably notice when you check it out.

    You can check out the rest of the projects, too. You probably want to ignore the "Development" and "Release" projects, though. They are used for P2, and just contain jar files of the builds.

    ReplyDelete
  10. OK got the code and installed C/C+ tools but maybe not al source is there:
    utils/data/DataFormat.h: No such file or directory

    ReplyDelete
  11. Hi Maarten,

    Sorry, I was away for the weekend and just got your comment. Are you sure that the check-out didn't fail somehow? I just checked it out on a linux machine that I just got set up for testing, and everything was there for me.

    ReplyDelete
  12. problem is now down to the boost library used, there is no easy version of that for Mac OSX :-( I'll continue next week...

    ReplyDelete
  13. Hi Maarten, why don't you open up a bug for this? It would be easier to track that way.

    ReplyDelete