Overview of parallel building

HCL VersionVault can perform builds in which multiple processes execute in parallel the build scripts associated with the makefile targets.

The processes executing the build scripts can run on a single host or across a collection of machines around the local area network (parallel distributed build). By using more concurrent processes, parallel builds can reduce the overall build time significantly. Instead of one process running one build script at a time, you can have multiple processors working in parallel. For large software systems, this performance improvement can make a critical difference. For example, you can use parallel builds to enable a build of your entire software system to run overnight and finish before developers and testers arrive for work in the morning.

You start a parallel build the same way as a single-host build: by entering a clearmake command. A command-line option or environment variable setting causes the build to run in a parallel mode.

A parallel build is controlled by specifications on all the hosts involved. The host on which you enter the clearmake command is the build client or build controller. On this host, you specify a limit to the number of build scripts to be executed concurrently. You can also specify a build hosts file, which lists the server hosts to be used for building.

Each build server host used in a parallel build can have an access-control file, named bldserver.control. To use the host as a build server, a build client must meet the access-control requirements.

When building in parallel, clearmake starts one or more audited build executor (abe) processes. An abe is a server process invoked by clearmake to control and audit execution of a build script during a parallel build. The first time it dispatches a build script to a host, clearmake starts an abe process there. Subsequent build scripts dispatched to the same host might be executed by the same abe process or by a different one.

An abe process starts by setting the same view as the calling clearmake. It executes a build script dispatched to it in the same way as clearmake--each command in a separate shell process.

All make macros are expanded by the build script calling clearmake, but environment variables are expanded by the shell process in which a build command runs. This environment combines the abe startup environment and the entire environment of the calling clearmake. Where there are conflicts (for example, SHELL and PATH), the abe setting prevails. To this environment, other macros are added:

• Special make macros, such as MAKEFLAGS, MAKEARGS, and (in the compatibility modes, gnu) MFLAGS. These macros are needed if the build script invokes clearmake recursively.
• Macros are assigned in a BOS file or on the clearmake command line. These settings are always placed in the build script's environment; they override, if necessary, settings in the environment of the calling clearmake or settings in the abe startup environment.

The stdout and stderr output produced by the build script is sent back to clearmake, which stores it in a temporary file. When the build script stops, clearmake prints its accumulated terminal output.

Each abe process sets to the same view and working directory as the clearmake process; each process then executes build scripts dispatched to it from the controlling clearmake process. When the abe process is being started on a remote machine, clearmake uses a standard UNIX or Linux remote shell facility, referenced through the versionvault-home-dir/etc/rsh symbolic link. A build script runs under abe control as if it were executed by clearmake, except that abe collects terminal output produced by the build script and sends it back to the build controller, where it appears in your clearmake window. The abe process terminates after waiting three minutes for an initial connection or after waiting three hours for a subsequent response.