Configuring the Data Load utility

In this lesson, you configure the Data Load utility environment, business object, and load order configuration files.

About this task

Environment configuration file: Use to specify your business context information, database connection property, ID Resolver, and data writer class. For more information about this file, see wc-dataload-env.xsd.
Business object configuration file: Use to specify your data reader, business object builder, and business object mediator. For more information about this file, see wc-dataload-businessobject.xsd.
Load order configuration file: Use to specify a pointer to your CSV input file, environment variable settings file, and business object configuration file. You can also specify which data load mode you want to perform. For more information about this file, see wc-dataload.xsd.

Procedure

Configure the environment variable settings that are used by the Data Load utility within an environment configuration file:

Create an XML file named wc-dataload-env.xml in the WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

Open the wc-dataload-env.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadEnvConfiguration
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../xml/config/xsd/wc-dataload-env.xsd" 
	xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">
 	<!-- Madisons Store in Toolkit -->
	1 <_config:BusinessContext storeIdentifier="Madisons" 
		languageId="-1" currency="USD">

	</_config:BusinessContext>

	<!-- database setting for derby in Toolkit -->
2 	<_config:Database type="derby" name="..\db\mall" schema="APP"/>
	
<!— IDResolver class -->
 <_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="10000000"/>

<!— DataWriter class -->

	3 <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

</_config:DataLoadEnvConfiguration>

Update the attribute values in the following elements within the file:

Element Required update Example

Element	Required update	Example
1 `<_config:BusinessContext>`	Specify your store settings.	`<_config:BusinessContext storeIdentifier="Madisons" catalogIdentifier="Madisons" />`
2 `<_config:Database>`	Specify your database environment properties.	`<_config:Database name="..\db\mall" type="derby" />`
3 `<_config:DataWriter>`	Specify your data writer class.	`<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />`

1 <_config:BusinessContext>

Specify your store settings.

<_config:BusinessContext storeIdentifier="Madisons" catalogIdentifier="Madisons"  />

2 <_config:Database>

Specify your database environment properties.

<_config:Database name="..\db\mall" type="derby" />

3 <_config:DataWriter>

Specify your data writer class.

<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

For more information about data load environment settings, see Configuring the data load environment settings.

Specify your data reader, business object builder, and business object mediator that is used by the Data Load utility:

Create a business object configuration file named wc-Passwordobject-loader.xml in the WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

Open the wc-Passwordobject-loader.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataloadBusinessObjectConfiguration 
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload-businessobject.xsd" 
	xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

  <_config:DataLoader className="com.ibm.commerce.foundation.dataload.BusinessObjectLoader">
<!-- Define data reader class to read the input from CSV file for loading -->
    1 <_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" >
    </_config:DataReader>

    2 <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >
    <!-- exclude the columns whose values are loaded previously from  USERREG table except LOGONPASSWORD & SALT -->
      <_config:Table name="USERREG" excludeUnListedColumns="true">
      	<_config:Column name="USERS_ID" value="USERS_ID" valueFrom="IDResolve">
		  <_config:IDResolve tableName="USERREG" generateNewKey="false" >
            <_config:UniqueIndexColumn name="LOGONID" value="logonId" />
		  </_config:IDResolve>
      	</_config:Column>
	    <_config:Column name="LOGONID" value="logonId" />
		<!-- The value of the SALT will be generated and added back to the userreg table later -->
		<_config:Column name="SALT" value="" />
		<!-- Plain password read from CSV file will be encrypted by the customized class PasswordEncryptionhandler  -->
	    <_config:Column name="LOGONPASSWORD" value="LOGONPASSWORD" valueFrom="ColumnHandler" >
	        <_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler">
			<!-- The following parameters will be passed to the  class PasswordEncryptionhandler  -->
	            <_config:Parameter name="logonPassword" value="logonPassword" />
				<!-- WC-server.xml is the file to store the merchant key  -->
	            <_config:Parameter name="instanceConfigLocation" value="c:\IBM\WCDE_ENT70\workspace\WC\xml\config\wc-server.xml" valueFrom="Fixed" />
				<!--  Customer may choose the merchant key in customKeyConfigLocation 
	            <_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" /> -->
	        </_config:ColumnHandler>
	     </_config:Column>
      </_config:Table>

      3 <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" 
     componentId="com.ibm.commerce.member" >
	  </_config:BusinessObjectMediator>
    </_config:BusinessObjectBuilder>
  </_config:DataLoader>

</_config:DataloadBusinessObjectConfiguration>

Update the attribute values in the following elements within the file:


Element	Required update	Example
1 `<_config:DataReader>`	Specify the data reader class that you implement.	`<_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" > </_config:DataReader>`
2 `<_config:BusinessObjectBuilder>`	Specify the business object builder class that you implement. Note: Specify the TableObjectBuilder class in this tutorial since you extend the generic table loader to load the customized data into the database. The table object builder is an extension of a business object builder for data load. It accepts data that is read by the data reader and constructs value objects to represent physical tables in the database.	`<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >`
3 `<_config:BusinessObjectMediator>`	Specify the business object mediator class. Note: Specify the TableObjectMediator class in this tutorial since you extend the generic table loader to load the customized data into the database instead of extending the business object logic.	`<_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" componentId="com.ibm.commerce.member" > </_config:BusinessObjectMediator>`

Set the excludeUnListedColumns parameter to true to exclude database columns that were previously loaded to the USERREG database table during the initial load:
```
<_config:Table name="USERREG" excludeUnListedColumns="true">
```
When a column is excluded, the value in that column is not modified during the data load.
The following code snippet shows the parameter values for the columns from the USERREG database table. Any columns that are defined in the database table (such as STATUS and PLCYACCT_ID) that are not included in this snippet were already loaded into the database during the initial load.
```
<_config:Column name="USERS_ID" 1 value="USERS_ID" valueFrom="IDResolve">
		  <_config:IDResolve tableName="USERREG" generateNewKey="false" >
            <_config:UniqueIndexColumn name="LOGONID" value="logonId" />
		  </_config:IDResolve>
      	</_config:Column>
	    <_config:Column name="LOGONID" 2 value="logonId" />
		
		<_config:Column name="SALT" 3 value="" />
		
	    <_config:Column name="LOGONPASSWORD" 4 value="LOGONPASSWORD" valueFrom="ColumnHandler" >
	        <_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler">
			
	            <_config:Parameter name="logonPassword" value="logonPassword" />
				
	            <_config:Parameter name="instanceConfigLocation" value="c:\IBM\WCDE_ENT70\workspace\WC\xml\config\wc-server.xml" valueFrom="Fixed" />
</_config:ColumnHandler>
	     </_config:Column>
Note: Customer also can choose to store their merchant key in their own customKeyConfigLocation, the usage example is provided below as well.

				<!--  Customer may choose the merchant key in customKeyConfigLocation:
	            <_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" /> 
```
1 USERS_ID

The value for this parameter is retrieved from the IDRESOLVE column of the USERREG database table, according to the LOGONID input.

2 LOGONID

The value for this parameter is read from the LOGONID column that is defined in the CSV input file.

3 SALT

The value for this parameter is generated by the generateSalt method and then added to the USERREG database table later. Specify the column for SALT here so that the corresponding salt value is written into the database later. The excludeUnlistedColumns parameter is set to true to specify that all of the columns that are not listed in the preceding code snippet are excluded and are not written to the database.

4 LOGONPASSWORD

The value for this parameter is read from the ColumnHandler interface. A new class that is called PasswordEncryptionhandler will be created as an extension class of AbstractColumnHandler in the next step of this tutorial. This class is used to generate the value for the SALT parameter and encrypt the user password. This value is achieved by taking the user password from the CSV input file and the merchant key from the wc-server.xml file. Parameters such as logonPassword and instanceConfigLocation are passed to the new PasswordEncryptionhandler class.

Specify the data load order and mode that the Data Load utility uses to load data:

Create an XML file named wc-dataload.xml in the WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

Open the wc-dataload.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadConfiguration
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload.xsd" 
	xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

	1 <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

  2 <_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace">
  	   
		<!-- Dataload password -->
        <_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml">
            <!--Indicates the location of input csv files, can be overrided at runtime.-->          
  	        <_config:DataSourceLocation location="MemberDataload_Password.csv" /> 	         
        </_config:LoadItem>

    </_config:LoadOrder>

</_config:DataLoadConfiguration>

Update the attribute values in the following elements within the file:

Element Required update Example

Element	Required update	Example
1 `<_config:DataLoadEnvironment>`	Specify your environment variable file.	`<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />`
2 `<_config:LoadOrder>`	Specify your CSV input file, business object configuration file, and data load mode.	`<_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace"> <!-- Dataload password --> <_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml"> <!--Indicates the location of input csv files, can be overrided at runtime.--> <_config:DataSourceLocation location="MemberDataload_Password.csv" /> </_config:LoadItem> </_config:LoadOrder>`

1 <_config:DataLoadEnvironment>

Specify your environment variable file.

<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

2 <_config:LoadOrder>

Specify your CSV input file, business object configuration file, and data load mode.

<_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace"> 

<!-- Dataload password --> 
<_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml"> 
<!--Indicates the location of input csv files, can be overrided at runtime.--> 
<_config:DataSourceLocation location="MemberDataload_Password.csv" /> 
</_config:LoadItem> 

</_config:LoadOrder>

For more information about configuring your data load order, see Configuring the data load order.