Data Load utility architectural overview

In order to work with the Data Load utility, you must first have an understanding of how it works.

The following diagram describes the components of the Data Load utility:

A high-level architectural diagram of the Data Load utility.

Business object builder layer
The business object builder layer contains the data reader and the business object builder. The data reader is responsible for reading the raw data and passing it to the business object builder for processing and building business objects. The business object builder takes the data as input from the data reader, then populates and instantiates the business objects. Each business object is defined as a common entity throughout the WebSphere Commerce data model. In other words, you only have to understand a single representation of the data through the store front, authoring tools, and the data load infrastructure.

The business object builder layer supports different types of data sources, such as data stored in comma-separated values (CSV) format, XML files, external databases, and enterprise resource planning (ERP) systems. A CSV data reader is provided with the Data Load utility. It is necessary to create custom data readers in order to support data sources of types other than CSV files.

Feature Pack 6 or laterAn XML data reader is now provided with the Data Load utility. This data reader supports a specific XML format. If your input files use a different format of XML data, you must create a custom data reader to support you XML file format.

The following business object builders are provided:
  • com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder
  • com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder
Business object mediator layer
The business object mediator layer contains the business object mediator. The business object mediator converts the business objects into objects that represent the physical database schema, also referred to as physical objects. Several mediators are available for catalog, inventory, and price components. A TableObjectMediator is also provided to support loading data to any table.

The ID resolver is also a part of the business object mediator layer. It is used to figure out if a physical object, which represents a row in a table, exists in a database. If the object is in the database, it returns the existing primary key of the object. Otherwise, it returns a new primary key for this new physical object.

Persistence layer
The persistence layer persists the physical objects into the WebSphere Commerce database. Two data writers are provided:
  • com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter
  • com.ibm.commerce.foundation.dataload.datawriter.NativeDBDataWriter
Business context service
The business context service manages contextual business information. The information is encapsulated within the different layers of the Data Load utility, and can be reused when needed to improve efficiency. Business objects uses a business context to represent common configuration information necessary to load data such as for a particular store, member IDs, or language. The context can also store component-specific configuration information such as master catalog identifiers, or invocation-specific configuration information such as the specific persistence mechanism and parameters.