Logical or adapter-specified Fetch Units

All adapters have a Fetch Unit. Some adapters have the capability to aggregate transactional data units. A transactional data unit refers to some unit of data that can be acted upon without affecting the state of another such data unit - for example, removing a message from a queue. Some adapters have the capability to divide one transactional unit into smaller discrete units of data. Some adapters cannot do either of the above. How the Fetch Unit is interpreted depends on what the adapter's capabilities are.

When the data retrieved from a source adapter represents one transactional unit that cannot be subdivided into smaller discrete units of data without the adapter knowing the content of the data, the Fetch Unit is interpreted as a logical unit. For example, if an input is a file, it is not a good idea to delete that file until a map completes its work - a stream file is one transactional data unit that cannot be divided into physical units by the adapter. In this case, the Fetch Unit is interpreted as logical by looking at the type definition of the input data to decide when a burst is complete.

When an adapter can aggregate transactional data units as one source, the Fetch Unit is interpreted in terms of the transactional data unit. For example, if the data source is MQSeries, multiple messages can be aggregated, and the Fetch Unit is the maximum number of messages to retrieve for a given burst. In this case, it is up to you to make sure the input type definition matches the amount of data retrieved for a single burst.

For database adapters, an input query represents one transactional unit that can be divided into rows by the adapter. In this case, the Fetch Unit is the maximum number of rows to retrieve from the database for a given burst.

When the source adapter can only retrieve one transactional unit of data, the source Scope is limited to Map - this prevents you from trying to delete half a file, or half a buffer, for example. If the source adapter can retrieve multiple transactional units of data, the source Scope can be Map, Burst or Card.