Parsing OCLI files in streaming

HCL Universal Orchestrator enables you to add or replace files containing multiple item types through OCLI in a single command.

The parsing of the file is done in streaming, meaning that the parser reads the file one line at a time. As a result, the file is not stored during the parsing process, and each item is processed while the file is read.

Detecting item type

The parser detects the item type in two phases:

1. If a dollar sign $ is present at the beginning of an item definition, the parser assigns the item type following the dollar sign. Valid values are:
   • acl
   • securityrole
   • jobstream
   • job
   • credentials
   • variables
   • vartable
   • workstation
   • folder
   • calendar
   • apikey
2. If there is no dollar sign, the parser uses "job stream" as the default item type. When a different item type is detected, the default item type changes to the detected type. In this scenario, if no dollar sign is detected, the item type is taken from the section the item is in.

Reference handling

Items in the file can reference each other. A simple reference example is a folder and an item defined inside that folder. HCL Universal Orchestrator can parse items and handle references even if referenced items are not listed in order. Listing items in order guarantees better performances.

If the parser encounters an item with an unknown reference, HCL Universal Orchestrator goes through the following steps:

1. The parser tries the add or replace action on the item
2. If step one fails and the item that is being read has an item type that can reference other items, the item is saved in memory and the parser continues to read the remaining items in the file
3. When the whole file is read, the parser tries the add or replace action on every item saved in memory. The parser finds and resolves previously unknown references, decreasing the number of items saved in memory.
4. Step three is repeated until either there are no more items saved in memory, or there is at least one item saved in memory with an unknown reference that cannot be resolved.
5. The parsing is complete when the selected action is applied to every item in the file. If errors are encountered that cannot be resolved by step four, the errors are presented to the user.

Loop dependencies handling

HCL Universal Orchestrator can perform the add or replace action on job streams with loop dependencies.

Consider the following example:

• JS1 follows JS2
• JS2 follows JS3
• JS3 follows JS1

To manage this situation, the job streams in error are added in draft without their dependencies. If an error occurs during this process, the item is discarded and the error is reported to the user.

When all the job streams are added in draft and the rest of the file is parsed, dependencies are added and the job streams are saved. If a job stream has a reference to a non-existing predecessor, an error occurs and it is reported to the user. The user also receives an additional error message warning them that the job remains saved as a draft without any dependency.

Cache fine-tuning

It is recommended to have an ordered list of items to maximize performances. When listing items, put the items that refer to other items after the items they are referring to.

Items with references that are not found are put in error and saved in a cache. The property uno.ocli.file.max.errors.cache.size can be used to set the maximum cache size to prevent it from growing indefinitely. The default maximum cache size is set to 1,000 items.

Items in error parsed after the cache is full are processed as regular items with no reference handling.