Load balancing

The HCL VersionVault load-balancing algorithm controls the way in which build scripts are dispatched to hosts.

During a parallel build, your clearmake process creates and updates a list of qualified hosts, a subset of the hosts listed in the build hosts file. A host is qualified if all the following criteria are met:

  • The host is at least 50% idle.
  • Your clearmake process meets the requirements of the host, as specified in its bldserver.control file.
  • An abe process can be started on the host.

Whenever it needs to dispatch a build script, clearmake updates its qualified hosts list and selects one of these hosts. If it cannot find any qualified host, it pauses and updates the list again. On any pass, if all hosts are eliminated because of errors, clearmake exits. If the hosts do not meet the first two requirements, clearmake waits and tries again. clearmake keeps trying in this manner until it finds at least one qualified host with which to build.

The selected host is not necessarily the best one, for example, the one that is most idle at that particular moment.

Randomizing host selection

The default load-balancing algorithm tends to select hosts near the top of the list more often than those near the bottom, subject to availability. For more even-handed selection when the list of hosts exceeds 20 or so, include this line:

-random
Note: This also changes the effective location of any #include directives.

A -random line can appear anywhere within a build hosts file. It applies to all host names in the build hosts file.

Idleness threshold

By default, your clearmake process does not dispatch a build script to a host unless it is at least 50% idle.

You can adjust this idleness threshold with a line in the build hosts file:

-idle percentage [ % ]

percentage can be any integer from 0 to 100. Idleness is negatively correlated with the load factor of the host, as shown by uptime(1); the approximate correspondence is this:

Load
Idle Percentage
0.0
100
0.5
68
1.0
47
2.0
22
4.0
almost 0

-idle directives can appear anywhere within a build hosts file. Until a -idle directive appears, the default value of 50% is applied.

-idle can appear multiple times within a build hosts file. Each -idle directive applies to the host names that follow, until another -idle directive appears or the end of the build hosts file is reached.

Because a host can appear multiple times in a build hosts file, it can be associated with different idleness thresholds. Each inclusion of the host is recorded with the associated idleness threshold. (In practice, if a host appears twice, once with a low idleness threshold and once with a high threshold, clearmake might select it once but reject it another time.)

Note: The idleness threshold can be specified with -idle directives on both the client and server. If there is a conflict, the overall principle is that the build server host controls its fate. For example:
  • A clearmake process is searching for hosts that are at least 50% idle (the default). A build server that appears to qualify because it is 70% idle is not used if its bldserver.control file includes the line -idle 75.
  • A bldserver.control file on a build server host permits access, because it contains the line -idle 60 and the host is currently 75% idle. However, clearmake does not dispatch a build script to this host, because the build hosts file specifies a higher threshold: -idle 80.

clearmake uses the idle specification in the bldserver.control file of your host to determine whether it can perform the build on your host. If your host does not have a bldserver.control file, clearmake assumes an idle threshold of zero and performs the build regardless of the load on your host. If a specified host appears in your build hosts file, clearmake ignores any -idle specifications for the host in the build hosts file and uses -idle 0.