[vos-d] build system reviews (long)

[Peter Amstutz]

As brought up earlier, I'm unhappy with the current automake build 
system used in VOS due to maintainance overhead (even trivial modules 
have grown to require a dozen lines of boilerplate) and the considerable 
difficulty it poses to properly supporting Windows development.  We 
resolved the last discussion with "needs more research".  Well, I've 
done some research.

I've decided on the following criteria to look for in a build system:

- Free software (obviously)
- Cross platform (must support Linux/OS X/Windows, gcc and MSVC)
- Build files arn't too verbose
- Build files can be located in the same directory as the source code 
for each subproject (library or app)
- Can add new rules that apply across the whole project without having 
to edit every build file individually
- "configure" ability, or easy to integrate with autoconf
- Supports robust parallel builds
- Out-of-tree builds
- Something along the lines of "make install", "make dist" and "make 
distcheck"

Here's what I'm currently looking at, grouped into two caterogies:

Makefile generators:

* Bakefile

We talked about this one before.  This tool was developed for WxWidgets 
and supports a long list of compilers and backend build systems.  It 
describes the build using a XML file.  A build rule inherits from a 
"template" to supply environment parameters (such as debug or not) and 
templates can extend other templates, which seems fairly powerful.  By 
having a few common "root templates", it it easy to apply changes across 
the entire project.  The user can also declare certain kinds of 
conditional expressions based on parameter values.

Bakefile creates Makefile.in files and integrates nicely with autoconf, 
which means the user does not need to have bakefile installed if they 
only need to compile.  Because it uses other tools for the actual build, 
parallel build support is only as good as the underlying tool.  I'm not 
sure how bakefile handles subprojects.  I suspect you need to set up a 
toplevel build file and use the <include> feature to bring in each 
separate project file, which results in a monolithic Makefile (this does 
suggest parallel builds are more likely to work).  There does not appear 
to be any "make dist" support, but there is "make install" support.

* CMake

Another project file generator.  This one was developed for VTK (the 
Visualization Toolkit) and is also notable for being the build system 
used by KDE.  It uses its own simplistic syntax for describing build 
rules.  It has parameter substitution and conditionals.  It has commands 
like FIND_LIBRARY() that take care of some of the responsibilities of 
autoconf, but I believe these checks are done at the point that you run 
"cmake", so you can't distribute the resulting Makefile if they have 
system-specific configuration.  This means the user needs to have cmake 
installed.  I'm not sure how you would extend a build rule across an 
entire project, possibly using ADD_CUSTOM_COMMAND() to set up a 
customized rule.

Subprojects are easy, each file specifies what subdirectories to search 
for further build rules.  Makefiles are output in each directory 
separately, and it uses recursive make (which mean parallel builds are 
out).  There is no "make dist" support, but there seems to be "make 
install" support.


Make replacements:

* Perforce Jam

Jam is a make replacement used by several projects, most notably for us 
Crystal Space (as such I've had more direct experience with it than the 
others).  It provides a way of specifying a rule and a target (generally 
an output library or application) and specifying attributes that goes 
with that target (such as other libraries to link to).  It has parameter 
substitution and conditionals.  Customization seems to be done mainly 
through customizing build rules.  A build rule is defined using Jam's 
own scripting language which seems to be an interesting mix of 
declarative referencing of other rules and imperative statements 
(conditionals and variable assignments).

Jam uses a monolithic build, but allows the user to specify individual 
jamfiles recursively (each jamfile specifies a list of subdirectories to 
look in for more jamfiles).  Somewhat obnoxiously, each jamfile must 
explicitly specify the path from the root directory to the directory the 
jamfile is in (I don't understand why it can't figure this out for 
itself.)  Because it computes a gloal dependency tree, it does a good 
job with parallel builds.

Jam seems to be more reliant on user customization than the others, with 
a very minimal default rule set.  However, if we were to use Jam, we 
would probably want to adopt an existing build system with a robust set 
of rules, such as the Crystal Space build system.  Jam can used with 
autoconf, by using a files like "Jamconfig.in" that lets autoconf fill 
in all its relevant findings.

Because Jam rules abstract out compiler-specific settings, it is 
possible to use Jam (with sufficient application of custom rules) as a 
makefile generator as well.  Crystal Space produces both makefiles and 
Visual Studio projects from its jam-based build system.

* Boost.Build (bjam)

Boost Jam is a fork of Perforce Jam adopted by the Boost project.  There 
are two flavors, known as Boost.Build v1 and v2.  Boost jam is designed 
to be used specifically with the Boost ruleset, so it is most comparible 
to the combination of Perforce Jam + Crystal Space rules.  The Boost 
build system provides support for many different compilers, supports 
out-of-tree builds and has rules for both "staging" (copying binaries to 
an intermediate directory, similar to what the VOS "inplace/" directory 
tree) and installing.

Although derived from Jam, bjam seems to have a different syntax for 
target attributes.  Notably different from Perforce Jam (so far as I can 
tell) it supports rule templates so the user can specify a set of 
attributes to be used across different targets, and templates can derive 
from other templates. 

Boost itself uses autoconf for features tests, but you wouldn't know it 
because the boost distribution by default is preconfigured for a variety 
of platforms so it is rarely required to do the full autoconf check.  
So, the discussion about autoconf and jam above presumably holds for 
Boost Jam as well.  The discussion above about the monolithic build and 
good support for parallel builds also holds.

Unlike Crystal Space, which supports MSVC through generating Visual 
Studio project files, Boost supports MSVC by requiring the user go to 
the cmd shell and running bjam.exe which runs the compiler itself.  This 
has the advantage of ensuring everyone is always using the same build 
system, but it can be very annoying to Windows users used to using the 
IDE for everything.

* Scons

Scons is, supposedly, the "next big thing".  Scons is, at its heart, a 
robust dependency engine written in Python.  Scons files are actually 
Python scripts that call functions declaring targets and dependencies; 
these scripts can access the entire Scons API as well as perform any 
Python action.  This grants the developer an unusual level of 
flexibility in scripting builds, since arbitrarily complex logic is 
available.

The Scons developers focused strongly on providing a robust solution to 
the most basic task of the build system, which is to figure out what 
needs to be built, and to build it, and never do any extra work or 
(worse) fail to rebuild when something has changed.  Unlike most other 
build systems, dependencies and rebuild requirements in Scons are based 
on file hashes rather than timestamps, which gives much stronger 
guarantees about whether a file has changed on not and needs to be 
rebuilt.  Scons is said to have terrific parallel build support, as well 
as distcc support (enabling multi-host parallel compiles).

Scons provides feature test functions to fill the role of autoconf.  It 
supports hierarchical builds, allowing the toplevel file to either list 
all subprojects, or recursively.  In either case, the result is still a 
monolithic build where all dependencies are scanned and computed before 
compiling begins.  Scons has install support, but no dist support.  I'm 
not sure if it supports out-of-tree builds (probably, but not obvious 
from the documentation.)

Scons has a notion of "construction environments" which provide 
parameters (such as CFLAGS) to build rules.  This appears to be similar 
to the templating features described earlier for Bakefile and Bjam.  
"Deriving" an environment seems to be a copy-and-modify operation done 
at the procedural Python level.

One feature unique to Scons is support for checking out code from source 
control systems.  This is also interesting since both Scons and Bzr are 
written in Python, we might be able to integrate them in interesting 
ways.

Scons is probably the most interesting of these packages in terms of 
approach and capabilities.  Unfortunately, I also found a number of 
comments by people in charge of large projects that had tried Scons and 
found some problems:
 - Scons is slow.  Very, very slow.  It has to parse and evaluate all 
its Python scripts every time, and then compute hashes for every file 
before it can do anything.  It also runs the "configure" feature tests 
every time.
 - Failure modes are apparently quite cryptic, and indicated with a 
Python exception and a stack trace rather than a nice error message.
 - Being that input files are actually Python scripts, they require some 
extra quoting which can be non-obvious for people who don't know Python.

* Good old make

Finally, to round out the discussion, it bears mentioning that some 
projects (Mozilla, for example) use autoconf and handwritten makefiles 
(well technically Makefile.in).  These makefiles are written to deal 
with platform issues (microsoft compiler vs. gcc and such) but such 
solutions essentially are reinvented for each project.  This also 
requires people do command line builds, even on Windows (? unless some 
way of generating visual studio projects is also written).  Files 
written this way tend to require a lot of boilerplate code, as well as 
possibly having to maintain the dependency graph by hand.  This is the 
default position that all the other build tools and makefile generators 
are trying to improve upon.

All of the packages reviewed here definitely have both advantages and 
disadvantages, and as such no tool clearly wins out.  I have not decided 
which way to go yet, but could see us using any of these packages.  
Thoughts?

-- 
[   Peter Amstutz  ][ tetron at interreality.org ][ peter.amstutz at gdit.com ]
[Lead Programmer][Interreality Project][Virtual Reality for the Internet]
[ VOS: Next Generation Internet Communication][ http://interreality.org ]
[ http://interreality.org/~tetron ][ pgpkey:  pgpkeys.mit.edu  18C21DF7 ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://www.interreality.org/pipermail/vos-d/attachments/20070423/219373f6/attachment-0001.pgp