Friday, August 21, 2009

Using Eclipse With Large Code Bases, Part II

In Part I, linked source with excludes failed to solve the problem of how to break up a large, single-rooted source tree into multiple projects in Eclipse.

After a night off, I thought of using file-system links to create the illusion of multiple directory roots. But we needed directory-to-directory links, not file-to-file links, and Windows doesn't directly support those.

A coworker found the Windows-specific "junction" command, which can be downloaded from Microsoft. It's an add-on to Windows, not part of the standard set of shell commands. With the junction command, I was able to create a multi-rooted source tree that points to the source it needs from the single-rooted source tree.

For example, starting with:
C:\src\
com\
parent\
Parent.java
child1\
Child1.java
child2\
Child.java
Create a parallel structure:
C:\projects\
Child1\
.classpath
.project
src\
Child2\
.classpath
.project
src\
The .project files don't have linked-source directives in them, and the .classpath files just have the standard <classpathentry including="**/*.java" kind="src" path="src"/> entries.

From a command prompt, execute commands to create links from the project src directories into the real source tree:


mkdir C:\projects\Child1\src\com\parent\child1
junction C:\projects\Child1\src\com\parent\child1 C:\src\com\parent\child1

mkdir C:\projects\Child2\src\com\parent\child2

junction C:\projects\Child2\src\com\parent\child2 C:\src\com\parent\child2


(The commands for creating directory and file links in linux/Mac are of course different, but the concepts are the same.)

The resulting directory structure looks like this:
C:\projects\
Child1\
.classpath
.project
src\
com\
parent\
child1 → linked to
C:\src\com\parent\child1
Child2\
.classpath
.project
src\
com\
parent\
child2 → linked to
C:\src\com\parent\child2
After refreshing the projects in Eclipse, the unwanted packages and source files are gone:


Unfortunately, the code doesn't compile:


Remember the third item in the list of problems?:

3. Some source files used throughout the code are located in the top of the source tree

It has come back to haunt us. We have to have visibility to Parent.java in both projects, but we can't link to the root of the source tree, because that's the problem we're trying to solve with links.

To fix this, create another project, Parent, but use a file link instead of a directory link:

mkdir C:\projects\Parent\src\com\parent
fsutil hardlink create C:\projects\Parent\src\com\parent\Parent.java C:\src\com\parent\Parent.java


Then add the Parent project to the dependencies of Child1 and Child2, and now it does compile:


This approach works very well--the entire million-plus lines of code is broken up into 40+ projects in Eclipse, and the code compiles quickly after the initial import.

You can envision an Eclipse plugin that would semi-automate this process. At a minimum it would be nice to generate the projects and link scripts from some kind of description, instead of editing the files by hand. Unfortunately, by the time I had worked out the pattern, the projects and links were mostly already finished.

Note: Although it's not shown in the examples above, this approach can also be used to link to multiple child nodes in a source tree to produce a combined tree for a project. For example:

mkdir C:\projects\Child1\src\com\parent\child1
junction C:\projects\Child1\src\com\parent\child1 C:\src\com\parent\child1


mkdir C:\projects\Child1\src\com\parent\otherChild
junction C:\projects\Child1\src\com\parent\otherChild C:\src\com\parent\otherChild


In Part III, I'll describe how we dealt with this issue:

4. Packages and layers have mutual dependencies (for example, business logic in the UI layer), but Eclipse treats cycles among projects as compile errors

No comments:

Post a Comment