Eclipse, the multi-language open source integrated development environment (IDE), has just received a considerable gift from Google’s Open Source Programs Office: a $20,000 donation to help refine the performance of the latest 4.2 release.
A Friend in Need
This donation wasn’t exactly out of the blue. It was made in direct response to a lengthy discussion on the official Eclipse mailing list, where a number of users reported significant performance issues when using the 4.2 release, compared to identical setups running the older 3.8. Some users even went as far as to claim that the 4.2 release was rushed out, and that it offered no significant improvements over the previous stable releases.
A number of issues were discussed between users and the developers, but one of the core concerns was that the Eclipse developers were no longer running the extensive performance tests that they had done on previous releases. Without these tests, it was difficult to ascertain what impact new features and changes had on the end user experience.
In response to this, Eclipse founding member and UI team leader Mike Wilson posted that the team wanted to run the performance tests, but simply couldn’t at this point in time:
We definitely *want* to re-enable the performance tests. The two things that have been holding it up currently are:
1) Resources to get them working on the foundation hardware
2) Working through the issues caused by running the performance tests on shared devices.
When we had a lab to hold dedicated performance machines we put a lot of effort into configuring and managing them so they behaved as close to exactly the same as possible. For example, there was one time that the RAM in a machine failed and we were able to see the impact of replacing that RAM (even though it was rated the same) as a difference in the test results.
In a world with potentially other tasks running on the same machines, wildly variable network traffic, etc. I don’t think our current performance testing story will work. If that’s true, it means it will be a *lot* of work to get them running again. Btw, if anyone has good insights on this and/or wants to help us get the tests running again, we’d love to get your help.
Basically, the Eclipse developers simply didn’t have the machines available to perform the extensive performance testing that they could before. Some debate followed in the discussion about the pros and cons of attempting performance testing with virtual machines, but in the end it seems the team felt “bare metal” testing was in order.
Within 48 hours of Mike’s post, a representative of Google’s Open Source Programs Office let the developers know they would be taking up his challenge of helping getting the performance tests running again:
Given the recently raised news concerning 4.2’s performance, and the loss of testing hardware previously provided by some member companies, Google’s Open Source Programs Office is sending the Eclipse Foundation a donation of $20,000 to purchase hardware and begin building a common testing lab.
Some of this donation may also be used to support the common build infrastructure, subject to Denis Roy’s and Mike Milinkovich’s discretion.
Of course, given Google’s reliance on Eclipse, their support shouldn’t come as much of a surprise. Eclipse is the de facto IDE for Android development, and Google has gone so far as to bundle Eclipse with their official Android SDK.
With nearly every Android developer on the planet sitting down in front of a copy of Eclipse, it’s certainly in Google’s best interest to make sure that performance is as optimized as is possible. A better performing Android development kit means happier Android developers, and when Google is locked in a battle with some of the biggest names in technology over the future of mobile operating systems, happy developers are an extremely important resource.