Scanning and removing orphaned Codestation blob files

The csorphanscan tool in the /bin directory scans for orphaned Codestation blob files and relocates them to an orphan’s directory.

About this task

An orphaned Codestation blob file is a file that contains data for a component version that is no longer referred to by any component version currently recorded in the database. Such files use disk space but can never be accessed by a deployment process.

By default, the tool runs in a dry-run mode such that the file system is not modified and the tool only reports statistics. The tool requires some time to execute.

The csorphanscan tool supports only disk-based blob storage. S3 storage is unsupported.

The csorphanscan tool can run while the server is online. In an HA cluster, it must be run on only one cluster member.

To scan and remove orphaned Codestation blob files:

Procedure

  1. Run the csorphanscan command without the --move-orphans argument to report statistics.

    This run takes some time. In an HA cluster, all members share the same database and Codestation storage, so run the csorphanscan tool on only one cluster member.

  2. Run the csorphanscan command with the --move-orphans argument to relocate orphans.
    This process takes some time. The orphan files are relocated to the <appdata>/var/repository/blob/orphans folder. In an HA cluster, all members share the same database and Codestation storage, so run the csorphanscan tool on only one cluster member.

    You can also use the following flags with the csorphanscan tool:

    Option Description
    --orphan-list-file <path> Write a list of orphaned files to a file in the specified path.
    --workers N

    Set the number of concurrent workers. The default number is 4, and the minimum number is 1.

    Values greater than 4 might not reduce tool run time.
    --memory N[k|m|g] Set the amount of heap memory. The default size is 2 GB. Instances with very large numbers of component versions stored in Codestation might require additional memory. The default value is sufficient for at least 400K component versions.
  3. Back up the <appdata>/var/repository/blob/orphans file and delete it to free the disk space. You must use the file deletion tools of your operating system. In an HA cluster, all members share the same Codestation storage, so perform this procedure on only one cluster member.
    Note: You must never manually delete any other <appdata> content outside the orphan's directory.