Using –sha1_download_dir with air-gapped and internet-enabled BES Server

Previously, using the “buildRepo –key server-7-x86_64" with RHSMDownloadCacher mirrors the Red Hat Repo “server-7-x86_64" structure offline. This might result to duplication of packages if they are found in multiple repositories. Using –sha1_download_dir will download all packages from all repositories (keys) as files with a sha1 filename into a single, flat directory. The use of –sha1_download_dir varies, depending on whether the BES Server is air-gapped or internet-enabled.

The use cases indicated in this section require version v1.0.1.0 and later.

Storage space savings benchmark

Space-saving benchmarks have been established with the use of the --sha1_download_dir. Using the --sha1_download_dir have shown significant decrease in storage size, download size, and time when caching multiple repositories of the same RHEL version. This is because many packages are duplicated among repositories with the same RHEL version (server-6-x86_64, workstation-6-x86_64, client-6-x86_64). Space is not saved if you only cache a single repository for each RHEL version, such as caching server-6-x86_64, server-7-x86_64.

Caching server-6-x86_64, workstation-6-x86_64 (with --sha1_download_dir)
Total Repo Metadata and Packages will take up 84.2 GB of space instead of 158.3 GB (45% space saved)
Caching server-6-x86_64, workstation-6-x86_64, client-6-x86_64 (with --sha1_download_dir):
Total Repo Metadata and Packages will take up 85 GB of space instead of 235 GB (63% space saved).

Air-gapped BES Server

  1. Increase the BES Server’s sha1 folder size limit.

    From the BES Console, right click the computer and select Edit Computer Setting. Increase the _BESGather_Download_CacheLimitMB size. The suggested size is the current BES Server’s sha1 folder size plus the size of sha1_download_dir.

    If the size of the sha1_download_dir cannot be determined beforehand the suggested size of the sha1_download_dir is a minimum of 60GB per repository. Note that the minimum size of 60GB might increase over time.

  2. Use the RHSM download cacher to cache packages to "sha1_download_dir" and the Repo metadata to "download_dir".

    RHSMDownloadCacher --rootCertDir "certs" --download_dir "download_dir" --sha1_download_dir " sha1_download_dir" buildRepo --key server-7-x86_64

  3. Transfer the "download_dir" to the air-gapped BES Server and the sha1 files in "sha_download_dir" to the sha1 folder of the BES Server.
  4. Configure the values in the plugin.ini file.

    The plugin.ini file is located in C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\plugin.ini.

    Configure the values to the following settings:

    localCache = "location of the transferred download_dir"
     localCacheOnly = yes

    localCacheOnly is set to 'yes' so that the air-gapped BES Server will not get the repository metadata online but only from the localCache.

    Repository metadata:

    The BES Server will only get the Repo Metadata from the localCache.

    Packages:

    The BES Server will only get the packages from the localCache.

Note: Ensure that the path for the localCache parameter is different from the RHSM Download Plugin cache folder path. By default, the RHSM Download Plugin cache is located at C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\cache.

Internet-enabled BES Server

  1. Increase the BES Server’s sha1 folder size limit.

    From the BES Console, right click the computer and select Edit Computer Setting. Increase the _BESGather_Download_CacheLimitMB size. The suggested size is the current BES Server’s sha1 folder size plus the size of sha1_download_dir. If the size of the sha1_download_dir cannot be determined beforehand, the suggested size of the sha1_download_dir is a minimum of 60GB per repository. Note that the minimum size of 60GB might increase over time.

  2. Use the RHSM download cacher to cache the package to BES Server’s sha1 folder.

    RHSMDownloadCacher --rootCertDir "C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\certs" --download_dir "C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\download_dir" --sha1_download_dir "C:\Program Files (x86)\BigFix Enterprise\BES Server\wwwrootbes\bfmirror\downloads\sha1" buildRepo --key server-7-x86_64

  3. Configure the values in the plugin.ini file.

    The plugin.ini file is located in C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\plugin.iniConfigure the values to the following settings:

    localCache = local

    CacheOnly = nolocal

    Cache is not specified as the packages are saved directly in the BES Server’s sha1. localCacheOnly is set to 'no' as the BES Server is internet-enabled and can get the repository metadata and packages from the Internet.

    Repository metadata: The BES Server will get the repository metadata from the RHSMPlugin's cache folder first, and then the internet if it is expired.

    Packages: The BES Server will check its sha1 folder for the package first, and then get it from the internet if the package is not found.

Internet enabled BES Server (no sha1_download_dir)

Since v1.0.1.0, using the --sha1_download_dir to cache the packages is considered best practice. Users that have their BES Server in an internet-enabled environment might want to store the packages in a separate folder instead of the BES Server' sha1 folder. This is useful if the BES Server’s sha1 folder size inflates. Since the BES Server only stores the latest download, the packages stored there might be replaced by newer files if the BES Server sha1 folder size limit that is set is too small.

Storing the packages in the localCache will allow the RHSMPlugin to use it instead of getting it from the internet.

Using the localCache to cache packages instead of the sha1_download_dir might take up additional space as the package will also be cached in the BES Server's sha1 folder when the BES Server requests the package from the localCache.

  1. Use the RHSM download cacher to cache packages to "download_dir".
    RHSMDownloadCacher --rootCertDir "certs" --download_dir "download_dir" buildRepo --key
          server-7-x86_64
  2. Configure the values in the plugin.ini file.

    The plugin.ini file is located in C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol\plugin.ini.

    Configure the values to the following settings:

    localCache = "location of the transferred download_dir"

    localCacheOnly = no local

    CacheOnly is set to 'no' since the BES Server is internet-enabled and can get repository metadata and packages from the internet.

    Repository metadata: The BES Server will get the repository metadata from the RHSMPlugin cache folder first, and then the internet if the repository metadata is expired.

    Packages: The BES Server checks for the package in the following sequence: its sha1 folder for the package first, the localCache, and then gets it from the internet.