vob_scrubber

VersionVault creates a metadata item called an event record in a VOB database almost every time it modifies the database. Event records are created to record such events as the checkin of a new version, the attaching of an attribute to an element, or the creation of a new branch type. Each event record consumes 300–400 bytes. Some event records, like those for element and version creation, are valuable indefinitely, but many minor event records are not. For example, the removal of a version label from a collection of versions creates a minor event record for each affected object. Over time, such minor event records occupy more space as they become less useful. (After a month or a year, no one is likely to care who removed the version labels, especially if the label type itself has also been deleted.)

In each replicated VOB, cleartool creates oplog (operation log) entries, which store all the information required to repeat the changes in some other replica of the VOB.

Each oplog entry logically includes this information:

• The identity of the replica where the change originally took place.
• The VOB-database-level change—for example, creation of a new element, checkin of a new version, attaching of an attribute, and so on.
• The storage-pool-level change, if any—for example, the contents of a new version.
• The event record generated for the change.
• An integer sequence number—1 for the first change originating at a particular replica, 2 for the next change, and so on. This is the epoch number of the oplog entry.

Like event records, oplog entries are stored in the VOB database and can be scrubbed.

When you export an update packet from a replicated VOB, MultiSite creates one export_sync record for each target replica. These records are stored in the VOB database and are used by the recoverpacket command to reset a replica's epoch matrix. You can scrub these records, but scrubbing old records limits the date range over which recoverpacket can operate.

By default, the scheduler runs vob_scrubber periodically. See the schedule reference page for information on describing and changing scheduled jobs. A configuration file, vob_scrubber_params, provides control over which event records and oplog entries are deleted.

You can run vob_scrubber manually as needed.

Default

Event statistics are listed briefly, categorized by kind of object. For example, all event records for branch objects are grouped.

–long

Produces a detailed report of the event statistics, categorized by kind of object, kind of event, and kind of operation.

Default

The report is sent to the log file, vob_scrubber_log.

–nlog

Sends the report to stdout instead of the log file.

Default

Scrub event records and then oplog entries. Report statistics on the number of objects, the number of records before deletion, the number of records deleted, and the number of records after deletion.

–stats_only

Suppresses deletion of records; the report includes statistics on the number of objects, event records, and oplog entries in the VOB.

–noe/vent

Suppresses deletion of event records.

–noo/plog

Suppresses deletion of oplog entries.

If both –noevent and –nooplog are specified, nothing is scrubbed.

Default

None.

–lvobs

Scrubs event records and oplog entries from all VOBs that reside on the local host.

vob-storage-dir-pname

Scrubs the VOB whose storage directory is at the specified pathname.

Default

The default behavior is specified by the default values for –wait and –stop.

–wait wait-minutes

When it encounters a locked VOB or when a VOB gets locked while scrubbing is in progress, the vob_scrubber waits wait-minutes (default 5). During this time, the vob_scrubber will attempt to scrub the VOB once per minute until the VOB is unlocked or wait-minutes expires. If wait-minutes expires and the VOB is still locked, and if the VOB has been locked for more than a week, the vob_scrubber skips it. If the VOB has been locked for less than a week, the VOB is moved to the end of the list (as specified by the –lvobs or vob-storage-dir-pname options) of VOBs that are to be scrubbed. The vob_scrubber then tries to scrub the next VOB on the list. The vob_scrubber attempts to scrub any locked VOBs that remain on the list once a minute for 12 hours, unless you use –stop to specify a different length of time, after which the vob_scrubber exits.

–stop stop-minutes

Stop moving locked VOBs to the end of the VOB list after stop-minutes (default 720 minutes, or 12 hours).

Default

Each event record or oplog entry is deleted in a single transaction.

–trs/ize transaction-size

Delete transaction-size items (event records, oplog entries, or both) in a single transaction. The default transaction-size is 20. Greater values of transaction-size can reduce the length of time required scrub a VOB but also increase the duration of the locks held by the vob_scrubber while these transactions are in progress.

If the environment variable CCASE_MAX_OPLOG_TRIM_ENTRIES is set to an integer value between 1 and 100, it controls the maximum number of oplog entries that the vob_scrubber removes in a single transaction, overriding any setting of –trsize when oplog entries are being processed.

A host-wide configuration file controls the operation of vob_scrubber; each VOB can have its own configuration file, which overrides the systemwide settings (UNIX® or Linux® pathnames are shown, but the Windows® paths are the same (except for slash direction):

The event-scrubbing configuration file is a text file. A line that begins with a number sign (#) is a comment. All other lines control how one kind of event is to be scrubbed—how long to keep the most recent one, and how long to keep other events of that kind:

These are the components of an event-scrubbing control line:

event

A keyword that indicates that the remaining components of the control line apply to the event records created by a particular CM operation. For a list of operations and the associated object to which event records are attached, see the events_ccase reference page.)

operation

Kind of event, specified by the operation that creates the event record. (For a list of operations and the associated objects to which event records are attached, see the events_ccase reference page.)

–keep_all { n | forever }

For each object: keep event records created by the specified operation for at least n days, or forever. If –keep_last is also specified, this period applies to all but the most recent such event; otherwise, the period applies to all such events, including the most recent one.

–keep_last { n | forever }

(Optional) For each object: keep the most recent event record created by the operation for at least n days, or forever. The keep_last period must be at least as long as the keep_all period. The meaning of "most recent event" depends on the operation. For a list of operations and the associated objects to which event records are attached, see the events_ccase reference page.

The vob_scrubber_params files also control scrubbing of oplog entries and export records. The syntax for these lines follows. (Do not begin these lines with the keyword event.)

oplog –keep { n | forever }

Specifies the number of days an oplog entry is kept in the VOB database. You must preserve oplog entries long enough to guarantee delivery of synchronization updates based on them. The default is forever.

export_sync –keep { n | forever }

Specifies number of days an export synchronization record is kept in the VOB database. By default, this line is not included in the vob_scrubber_params file, and the records are scrubbed with the same frequency as the oplog entries.

If the configuration file includes no control line for a particular operation, all event records created by the operation are kept forever. Therefore, an empty configuration file preserves all event records (except obsolete ones, which are always discarded); see the section, Which event records are deleted. The calculated times are always compared to the logical event creation time (as shown by lshistory), rather than the physical event creation time. These can differ if the event records were created by an exporter, such as clearexport_pvcs.

If the configuration file includes no –oplog control line, the oplog entries are kept forever.