Reflections on EclipseCon

Thanks to everyone at EclipseCon this past week for making it a great experience. JaxEnter asked me to write a short reflection about my experiences. I don't want to steal traffic from them, so you can see the article by following the link above :-).

The article is technology-centric. It was difficult for me to choose from among the many great things that are going on with Eclipse since it is such a vibrant and innovative community. The technologies in the article have heavy support from the top-level projects, so they will be interesting to follow. Also, in reality, they only reflect my own interests and experience and I could not have possibly seen everything going on at EclipseCon. That isn't to diminish any of the other great technologies that are built on Eclipse. There are a tonne of them that have become so integrated into my daily work, that I barely even notice them anymore. That's a real compliment, I think.

The best thing, really, though is the community. I have to admit that it was a little difficult to get to talk to people before I gave my talk on Wednesday. After that, though, I had a few great conversations. I'd like to make a shout-out to the guys over at Ericsson. For the past while, we have been working on the same set of problems, with support by the same funders. We had to go all the way to California for EclipseCon before we had heard about each other, though. It's a funny world.

Oh well, time to head back home. Good-bye cold, rainy California. I'll be going back to the rain and palm trees of Victoria :-).

Tracing Web Apps Using Diver

Diver is a tool designed to help Java Developers understand how unfamiliar, or long-forgotten, software works. It lets developers easily trace the actions of Java Software and analyze it using an advanced Sequence Diagram View and some workbench filters that help focus on the software artifacts associated with particular features of the software.

With more and more applications moving to the Web, I've been asked a number of times, "Can Diver trace web apps?" I've always told people that it is possible to do, but I haven't given detailed instructions on how to do it. This post will hopefully remedy that.

First, a few notes. The version of Diver used to illustrate the points in this post is 0.3.1. It isn't an official release yet because there is still some testing to be done. I've had to do some tweaking to Diver, though, to make tracing web apps a little easier. So, you can get your copy of Diver 0.3.1 by pointing p2 to http://diver.svn.sourceforge.net/svnroot/diver/Development. Another thing: Diver only traces Java, not JavaScript. So, Diver can't be used to help get at any JavaScript errors that are running in the browser. It can, however, trace Java apps that have been deployed on a Java-enabled web server such as Jetty, Tomcat, or JBoss. That includes code invoked from JSPs. In this post, I'll be using JBoss 5.0.1GA. There is no particular reason for that. I've used Diver to trace applications deployed in Jetty as well.

Install And Deploy Your Web App

The focus of this post is on tracing web apps, not on setting up web servers. So, I assume that you have your server ready to go. It's probably best, though, that you run on a small test server on your localhost rather than on a deployment machine. That just keeps things compact and easier to manage.

Diver works best if you have the source code for that app. It isn't strictly necessary, but it makes the tools that Diver offers a lot more powerful. At the very least, you have to be able to deploy your app into your own web server. You can just build and deploy your web app the way you normally would. For example, in JBoss, this would typically involve copying a WAR file into the jboss_root/server/server_name/deploy directory. When you are tracing your app, Diver will actually be tracing the copy that has been deployed on your server, not the version that is in your workspace. So, after you make any changes to your code, be sure to rebuild and redeploy before running your trace. Then, Diver will be able to match the calls to various methods/classes in the deployed WAR to those classes in your workspace, making it a lot easier to find things.

Preparing for Launch

When tracing a Java Web App, what we are really doing is capturing specific events that happen inside a Java web server instance. So, the application we will be tracing is the Java Web Server. Ideally, Diver would be able to interact with the Eclipse Web Tools Platform so that you could just fire up a trace of a web server launch. Unfortunately, I haven't gotten that far yet. Fortunately, though, programs like JBoss are really just Java applications, so we can use the Java Application Trace functionality of Diver to grab a trace, but it will take a little set up first.

The Diver Java Application Trace is just an extension of the JDT Java Application Launch. The JDT requires that the applications it launches are visible in the workspace, so Diver requires the same. That doesn't mean, though, that you have to have all of the JBoss source to get started. You can launch directly from your installed JBoss instance. To get JBoss to run, you just need to make the right jar files visible. This is how you do that.

I suggest that you make the appropriate jars visible in their own project. You could, possibly, create dependencies to JBoss or Jetty, or what-have-you, in your web app's project, but that could cause build problems or runtime errors if your web app needs its jars to be isolated from the server. I just create a small Java Project called JBoss:

Once the project is created, you need to make the JBoss launch jar visible. To do that, just select the JBoss project in the Package Explorer or the Navigator and choose File>Properties <Alt+Enter> from the menu. Jump over to the Libraries tab of the Java Build Path settings page, and select Add External Jars:

The jar that you are looking for is run.jar. It's just the JBoss launcher. On my computer, it is found in C:\jboss-5.0.1.GA\bin:

That's all that you need for preparation. Now, we can get onto the launch.

Setting Up The Trace

Now that the JBoss launcher is visible to Eclipse, we can get onto running it to gather a trace of our web app.  Open up your Run or Debug Configurations dialog, and create a new Java Application Trace. In the Main  tab, set the Project to the one we created earlier. The Main class that we will use is just org.jboss.Main. Simple enough:

Your web server will likely require a number of arguments to be passed both into the program and into the VM. Go to the Arguments tab. JBoss requires the following arguments:

Program Arguments:

-c <server> where <server> is the name of the configured server that you deployed your web app into. I typically just use default one supplied by JBoss, so my program arguments are -c default.

VM Arguments:

-Xshare:off -Dprogram.name=run.bat -server 

-Dorg.jboss.resolver.warning=true 

-Dsun.rmi.dgc.client.gcInterval=3600000 

-Dsun.rmi.dgc.server.gcInterval=3600000 -Xms128m -Xmx512m  

-XX:MaxPermSize=256m

-Xshare:off is a Sun/Oracle specific VM argument that is there for Diver's sake. It tells the VM that this launch will not share its classes with anyone (the Java VM allows different processes to share the same loaded classes unless this flag is set). If you aren't using the Sun/Oracle VM, get rid of this argument. The other ones are there for JBoss. The really important ones (I think) are -Dprogram.name=run.bat (or run.sh if you are running on Linux) and -server. You can set the heap and Perm Gen sizes according to your liking.

Working Directory:

/jboss_home/bin. On my computer, this is just C:\jboss-5.0.1.GA\bin. This has to be set so that JBoss can know where it is running from and resolve all of its own resources.

If you are wondering how I discovered what all of these arguments should be, I did it by inspecting the run.bat script that is used to launch JBoss. The script just sets a bunch of default variables, and I used those. Another thing that you should be aware of is that you need to have your JAVA_HOME environment variable set to a valid JDK. A normal JRE might work, but (depending on the version of your server) it may not be able to compile JSPs unless you have the JDK. I'm sure you already knew that, but I'm just throwing it out there for good measure.

Tweaking The Trace

That is all you technically need to do to get started. Now, you can just select the Run button and start gathering information like you normally would with Diver (see the Diver documentation). But there are a few things that you can do to tweak the trace and make sure it is efficient. Web apps can be slow as it is. Tracing them introduces i/o overhead that we would like to minimize. Go over to the Java Trace tab of your configuration, and set it up to look something like this:

First of all, you are likely only interested in the things that your web-app is doing, so I suggest selecting the Set Filter Manually option, and adding only the packages that are a part of your web app to the Inclusion Filters area. By Default, Diver will analyze the trace and only watch for methods defined in the project that is named in the Main tab of the launch configuration. In this launch, there is nothing but the JBoss launching jar, so the analysis would end up being mostly useless unless you set the filters manually.

I also suggest making sure Pause On Start and Apply Filters at Runtime are checked. Pause On Start is the default, and it just means that none of the start-up process of the program will be traced. You can define what actually gets recorded in real-time using the Eclipse Debug View as described in the Diver documentation.

Older versions of Diver (0.2 and earlier) recorded absolutely every method call, and "rejected" data based on the defined filters during a separate analysis step. Since version 3.0, Diver allows you to reject that data before it gets stored, which can speed up your traces and your analysis by many, many times. Make sure Apply Filters at Runtime is checked to enable this speed up.

Get Going

There you go, that is all you need to get a trace of a web app using Diver. Now, you can record and compare traces to focus in on specific bugs or features in your app and use Diver's sequence diagram to analyze what is going on. The Diver documenation, and the video tutorials tell you how to do all that fun stuff:

There is just one more small bit of housekeeping that you might want to be aware of. Launching a web server in this way doesn't give you a clean, interactive way to shut it down. You can always just use the "Stop" button in the Debug View to force it to quit, but that could leave your server in an undefined state. I suggest opening up a console, and using your web server's shutdown command. For JBoss, all you have to do is type shutdown -S.

That's all for now. If you are out at EclipseCon, make sure to come to my talk where I'll be presenting Diver and how it helps users of Eclipse understand their software. Happy coding!

Diver is a finalist for an Eclipse Community Award

I've been swamped with work lately, so I'm afraid that I haven't been able to devote much time to blogging. But I got news today that my Diver tool has been selected as a finalist for the Best Developer Tool category along side the excellent Pydev environment for the Python language. Diver has really been my baby for the past couple of years. I've been working hard on trying to solve the problems associated with understanding software. I hope that Diver can help developers in that respect (I know it helps me :-) ). Even so, it is a great honour to be selected as a finalist from among more than 40 other excellent solutions. Eclipse has such a vibrant developer community. There is a solution out there for almost every problem.

If you are going to EclipseCon, be sure to check out my talk to find out more about Diver and how to incorporate reverse engineering into your everyday work to improve your understanding of software.