Managing Downloads

BigFix uses several methods to ensure that downloads are efficient and make the best use of available bandwidth.

Among other techniques, caching is used extensively by all the BigFix elements, including servers, relays, and clients.

When an action on a client runs a download file command, the existence of the file is checked first in the client local cache. If the client cannot find it locally, it requests the file from its parent, typically a relay. In turn, the relay checks its own cache. If it finds the file, it immediately sends it down to the requesting client. Otherwise, it passes the request up to its parent, which might be another relay and the process continues. Ultimately, a server retrieves the file from an internal server or the Internet, caches it, and then passes it back down the chain. After receiving the file, each relay in the chain caches it, and continues to forward it down to the original client, which also caches it.

If the agent runs the download now command while performing the action, the file is requested and collected from the URL specified in the action script.

Each cache retains the file until it runs out of space. At that point, the cache is purged of the least-recently used (LRU) files to provide more space. You can view the relay cache size and other relay information by activating the Analysis ID# 227 BES Relay Cache Information available from the BES Support Site. The default cache size is 1 GB, but you can change it by using the Task ID# 148 BES Relay/Server Setting: Download Cache Size, also from the BES Support site.

There might be situations that require files to be manually downloaded and cached, typically because such files are not publicly available, in which case you must download the files directly from the source. Review the Fixlet Description tab for more information about specific manual cache requirements. You can pre-populate the download cache by copying files to the download cache location __Download. You can also delete these files manually.

The caches are stored as sub-folders of the program folder, which is created by default at %PROGRAM FILES%\BigFix Enterprise on Windows systems, and /var/opt/BES Server on Linux systems. The server download cache is BES Server\wwwrootbes\bfmirror\downloads\sha1, and the client download cache is found at BES Client\__BESData\__Global\__Cache\Downloads.

As well as the download cache, relays maintain an action cache (also 1 GB) holding all the files needed for each Action, and clients maintain a Utility cache.

For information about troubleshooting relays, including bandwidth and downloading, see Relay Health.

The client collects the file by requesting it from the url listed in the action script in one of the following ways:
  • When the complete set of downloads can be computed by parsing the action script, the complete set of downloads is computed by the server. The agent can ask the relay with a single request if the prefetch downloads are available for a specific action. In this request, the agent sends up the action ID, and the server response indicates all the files are available, or they are not. If these are all available, the agent starts requesting the files by their ordinal number (1 indicates the first file in the script, 2 indicates the second file in the script, etc.). If the files are not available, the relay informs the agent they are not and begins the process of fetching them, and the agent notifies that it is waiting for downloads to be available and put itself into a pending downloads state for that action for 10 minutes, at which time it asks the relay again, if the downloads are available for the specific action.

    When the downloads for an action become available on a relay, a notification is sent to the children of the relay, which uses the notification to accelerate requesting the downloads again. If notification messages are blocked for any reason, the agents 10 minute 'ask the relay again' behavior will eventually result in the agent detecting that the downloads are available, and begin to collect them. Child relays are also notified by their parent when the downloads based on the action ID and the ordinal numbers become available. They use this notification to accelerate their own request for the downloads again.

  • For downloads where any of the download url, size, and hash value are listed in the action script such that only the agents can compute them, the agents query their parent relay using an itemized downloads available request. The request contains a list of download items the particular agent needs. The relay and client behave the same way as described above, delaying subsequent requests, waiting for notifications

Resuming a download

If the download fails for connection problems, the download process is resumed as follow:
  • If the client is downloading from a BigFix Relay or Server, the download can be restarted at 10,000 byte chunks. This means that, when the client process is restarted, it verifies the 10,000 byte blocks already received, and then it resumes the download after the last verified block.
  • If the client is running a direct download from another server's URL, when the client process restarts, the download starts again from the beginning.