Data Load utility configuration properties

You can configure the load properties to change the behavior of the Data Load utility when you are loading data into HCL Commerce. You can edit the data load configuration files to add, update, or remove configuration property settings.

Data Load utility configuration properties can be added in the data load order or business object configuration files. The scope of a configuration property depends on where the property is specified within this file. Within a load order configuration file, properties can be set to affect all load items, or specific load items.

In the following load order configuration file example, the property1 value "a" applies to all business object configurations except for CatalogDescription to which the property1 value "b" explicitly overrides.

<_config:LoadOrder commitCount="1" batchSize="1" maxError="1" dataLoadMode="Replace">
<_config:property name="property1" value="a" />

<!-- Catalog business objects --> 
<_config:LoadItem name="Catalog" loadSequence="1.0" businessObjectConfigFile="wc-loader-catalog.xml"> 
  <_config:DataSourceLocation location="${Catalog}" /> 
</_config:LoadItem> 
<_config:LoadItem name="CatalogDescription" loadSequence="1.5" businessObjectConfigFile="wc-loader-catalog-description.xml">
  <_config:property name="property1" value="b" /> 
  <_config:DataSourceLocation location="${CatalogDescription}" /> 
</_config:LoadItem>

For information about where you can configure properties within a file, see

The following table identifies the optional configuration properties that are provided by default for the configuring the Data Load utility. This table also identifies sample configuration files that are available with HCL Commerce that you can configure to include each property. These sample configuration files are in the component and object directories:

WCDE_installdir\samples\DataLoad
utilities_root/samples/DataLoad


Property	Description	Configuration file available for property usage
`actionOnError`	Controls the action on error. You can set the following values for this property: 0 Upon encountering an error, throw an exception and exit (default value). 1 On soft delete error, go to next. A soft delete error is an attempt to delete a non-existent attribute or value.	All load order or business object configuration files
`allowChangeAttributeType`	Controls behavior of ATTRUSAGE column of ATTR table during a Data Load when the column is set to null. You can set the following values for this property: true Allowed values can be added to attributes even if the ATTRUSAGE is null. In this case, the ATTRUSAGE is set to 1 (default value). false An exception would be thrown in this case. Note: When you use Catalog Upload, the default value for the `allowChangeAttributeType` flag is false.	Business object configuration files for attributes, such as: wc-loader-AD-attribute-allowed-values.xml wc-loader-AD-attribute-and-allowed-values.xml
`allowDuplicateAdministrativeName`	This property configures the Data Load utility to create a promotion instead of skipping the promotion when the promotion has the same administrative name as an existing promotion. You can set the following values for this property: true The Data Load utility creates a promotion in the store for each promotion in the input file regardless of whether a duplicate administrative name exists. By setting this value, the Data Load utility creates all promotions that are in the input file. By enabling this property, the load process can create duplicate promotions in a store. false If a duplicate administrative name is encountered during the load operation, the Data Load utility does not load the promotion with the duplicate name. The Data Load summary report does report that a promotion did not load. This value is the default value.	Business object configuration files for loading promotions with the PromotionDOMMediator business object mediator. For instance: wc-loader-xml-promotion.xml
`attributeValueValidation`	Indicates whether to validate the attribute value. The attribute value is required except within a product and defining attribute relationship. You can set the following values for this property: true Enables validation (default value). false Disables validation. Note: The `attributeValueValidation` flag is configured to be `false` when you use Catalog Upload.	Business object configuration files for attribute values, such as: wc-loader-catalog-entry-AD-attribute-relationship.xml
`buildSearchIndex`	Requests that the Solr search index is to be built for category and catalog entry information. You can set the following values for this property: delta Indicates that a Delta build of the search index is required (default value). full Indicates that a full build is required. none That no search index build is to be requested.	All configuration files.
`createParentProductWhenMissing`	Indicates whether to create the missing parent product of the catalog entry if no parent product is provided. You can set the following values for this property: true The mediator creates the missing parent product. false The missing parent product is not created (default value).	Business object configuration files for catalog entries, such as: wc-loader-catalog-entry.xml wc-loader-catalog-entry-update.xml
`createReprocessCSVFile`	An optional property to indicate that a CSV file is created containing all failed data entries that are attempted to be loaded. Note: This property works for CSV source files only. You can set the following values for this property: true The Data Load utility will create a separate CSV file containing all data entries that failed to load. Note: For this feature be useful, you must also set the `maxError` property to `0`. This ensures that processing is not halted when an error is encountered. By default, this value is set to `1`, indicating that processing should be halted once an error is encountered. You can set the `reprocessCSVFileName` property to specify the output CSV filename. false The Data Load utility will not create a CSV file containing all data entries that failed to load (default value).	All load order configuration files.
`customerSegmentNameIsFileName`	Indicates whether the file name of the input file, excluding the file extension, is also the name of an existing customer segment. This property is supported only for use with the MemberGroupMemberMediator business object mediator. You can set the following values for this property: true The Data Load utility uses the file name as the name for the customer segment. If the customer segment name exists in the database, the utility can add or replace members for the customer segment. If the customer segment does not exist, the utility creates the customer segment member group. The created segment includes the file name as the segment name and description. If you are loading a file that includes only customer email addresses, you must set the value to be true. false The file name must be an existing customer segment name. If the customer segment does not exist, the utility does not create the customer segment member group. This value is the default value. If you set the value to be false and the customer segment doesn't exist, the Data Load utility throws an error during the load process that the customer segment member group is not specified.	Business object configuration files for loading member group members into a customer segment with the `MemberGroupMemberMediator` business object mediator. For instance: wc-loader-customer-segment-email.xml
`DataCacheInvalidationEnabled`	Indicates whether the load operation inserts records into the CACHEIVL database table for the object that is being loaded. These records are picked up when the DynaCacheInvalidation URL command runs to clear the data cache in the HCL Commerce server. Use this property to ensure that the data cache is invalidated so the Management Center can pick up the new object changes. You can set the following values for this property: true The data cache is invalidated false The data cache is not invalidated as part of the load process. This value is the default value.	Data Load utility order and business object configuration files. If you want the property to apply to all objects that are being loaded, include the property within the load order configuration file. To have the property apply to only a specific object, include the property within the appropriate business object configuration file. By default, all catalog upload load order configuration files include this property with a value of true.
`deleteSKUFromExistingParentProductWhenNewParentIsDifferent`	Indicates whether to delete existing parent product of a SKU if the new parent product is different. You can set the following values for this property: true The data load logic deletes the existing parent product of a SKU when the Data Load adds a SKU to a new parent product (default value). false The data load logic does not delete the existing parent product of a SKU when you use data load to add a SKU to a new parent product. You must manually remove the old product to avoid a SKU that has more than one parent product.	Business object configuration files for catalog entries that have child SKUs, such as: wc-loader-catalog-entry-parent-product.xml
`DeltaIndexingThresholdUponWorkspaceApproval`	Specifies the delta indexing threshold for workspace indexes. If the threshold value is reached when a workspace approval occurs, a full index runs instead of a delta index.	Data Load utility order configuration files that are used to load objects that can be indexed with HCL Commerce search.
`fixSEO`	Configures the load operation to replace invalid characters in the Keyword column for an SEO URL with the underscore `"_"` character. You can set the following values for this property: true When an input file includes invalid characters in the Keyword column for an SEO URL, the load operation replaces any invalid characters with an underscore character. false The load operation does not replace the invalid characters and an error occurs.	Business object configuration files for loading catalog entry objects with the `CatalogEntrySEOMediator` business object mediator. For instance: wc-loader-catalog-entry.xml
`ignoreNonExistUser`	Indicates how the Data Load utility is to handle loading email addresses when a member unique ID cannot be resolved or created for the address. This property is supported only for use with the MemberGroupMemberMediator business object mediator. You can set the following values for this property: true When the email address cannot be resolved to a user ID, the Data Load utility ignores the error and does not load the email address. The utility then continues with the load operation. When the load completes, the utility includes the number of email addresses that could not be resolved. false When the email address cannot be resolved to a user ID, the Data Load utility throws an exception. This value is the default value.	Business object configuration files for loading member group members with the `MemberGroupMemberMediator` business object mediator. For instance: wc-loader-member-group-member.xml
`initAttribute`	Indicates whether to implicitly initialize the attribute allowed value if it is not defined. You can set the following values for this property: true The attribute allowed value is implicitly initialized if it not defined. false The attribute allowed value is not implicitly initialized.	Business object configuration files for attribute dictionary attribute and catalog entry relationships, such as: wc-loader-catalog.xml wc-loader-catalog-entry-AD-attribute-relationship.xml
`initAttributeDictionary`	Indicates whether to initialize the attribute dictionary for the store. You can set the following values for this property: true The attribute dictionary is initialized. When you load attribute dictionary attributes into a store, if the store does not have an attribute dictionary, an attribute dictionary is created. false The attribute dictionary is not initialized (default value).	Business object configuration files for loading catalogs and catalog entries that use the CatalogEntryMediator or CatalogMediator, such as: wc-loader-catalog.xml wc-loader-catalog-entry.xml
`inputDataListSize`	Used when parallelization is enabled via the `multipleThreadsEnabled` property. The maximum number of CSV line entries that is included in a batch of data to be added to the queue. Each writer thread handles a single batch of data from the queue. Once it is loaded, the thread is freed to process another batch from the queue. By default, the `inputDataListSize` parameter is set to `20`.	All load order configuration files, within the respective load item.
`loadSEO`	Defines whether the Data Load utility can load SEO URLs. You can set the following values for this property: true The Data Load utility can load SEO URLs for catalog entries false SEO URLs are not loaded.	Data Load utility order configuration files for loading catalog entry or category information or files for loading page description information.
`markForDelete`	Indicates whether to perform a physical delete or flag an object as marked for delete. You can set the following values for this property: true The mediator marks for delete. This value is the default value for the CatalogEntryMediator business object mediator. false The mediator performs a physical delete. This value is the default value for the CatalogGroupMediator business object mediator.	Business object configuration files for catalog entries that use the CatalogEntryMediator or CatalogGroupMediator, such as: wc-loader-catalog-entry.xml wc-loader-catalog-group.xml
`multipleThreadsEnabled`	Defines whether parallelization is enabled for the specific load item. By setting this parameter to `false` for a specific LoadItem, you override the set parallelization parameters and force the data load utility into single threaded operation. Manually set per LoadItem, if this parameter is not specified its default value of `true` is assumed.	All load order configuration files, within the respective load item.
`numberOfThreads`	Used when parallelization is enabled via the `multipleThreadsEnabled` property. The maximum number of individual writer threads that take batches of data from the queue, process them in order, and write the processed data into the database. By default, the `numberOfThreads` parameter is set to `1`, meaning the data load utility should run in single threaded (legacy) mode. The maximum number of threads is `8`. If a number greater than `8` is provided, then the maximum number of threads is used. Note: From internal performance testing, HCL recommends that the number of threads used be `4`. Use of more than four threads has shown to reduce overall load performance, and can result in errors such as the following: `com.ibm.commerce.foundation.dataload.exception.DataLoadApplicationException: A problem occurred while initializing the property information during the business object builder initialization.`	All load order configuration files, within the respective load item.
`replaceAllElements`	Indicates whether the Data Load utility replaces the existing elements for an activity with the elements within the input file. This property is supported only for use with the MarketingElementMediator business object mediator. You can set the following values for this property: true The Data Load utility deletes all existing elements for an activity before the utility loads new elements. This value is the default value. false The Data Load utility does not replace all existing elements. Only the existing elements that match elements in the input file are replaced.	Business object configuration files for loading activity elements with the `MarketingElementMediator` business object mediator. For instance: wc-loader-marketing-element.xml
`replaceAllExistingMembers`	Indicates whether the Data Load utility replaces the existing users in a member group with the members in the input file. This property is supported only for use with the MemberGroupMemberMediator business object mediator. You can set the following values for this property: true The Data Load utility deletes all existing members from the member group before the utility loads new users. false The Data Load utility does not replace existing members. This value is the default value.	Business object configuration files for loading member group members with the `MemberGroupMemberMediator` business object mediator. For instance: wc-loader-member-group-member.xml wc-loader-customer-segment-email.xml
`replaceParentMasterCategory`	Indicates whether the Data Load utility overwrites an existing parent master category of a catalog entry if the input includes a new parent master category. You can set the following values for this property: true The Data Load utility overwrites the existing parent master category with the new parent master category. false The Data Load utility does not overwrite the existing parent master category and throws a validation exception.	Business object configuration files for catalog entries, such as: wc-loader-catalog-entry-parent-catalog-group.xml wc-loader-catalog-entry.xml
`reprocessCSVFileName`	An optional property used to specify the filename of the CSV file that is created when the `createReprocessCSVFile` property is enabled. Each reprocess CSV file name must be defined within the respective load item element. If no filename is defined, the generated CSV file is named `sourcefile`Reprocess.csv. Where `sourcefile` is the name of the source CSV file used in the load item. You can set the following values for this property: `filename`.csv Where `filename` is the custom file name for the CSV file that is created upon running the Data Load utility. The `.csv` file extension must be included in the filename value.	All load order configuration files, within the respective load item.
`rollbackForAnyException`	Indicates whether the Data Load utility rolls back all of the data that is processed after the last committed record when any type of exception occurs. This property is supported for use only with the data load order configuration file. You can set this property within the configuration for each load item or within the parent `<_config:LoadOrder>` element to have the property setting apply to all load items. You can set the following values for this property: true The Data Load utility rolls back all of the data that is processed after the last committed record when any type of exception occurs. For instance, when incorrect input data causes a NumberFormatException or similar exception. The number of records that are processed before the records are committed is determined by the value for the commit count attribute in the load order configuration file. false The Data Load utility does not always roll back all of the data that is processed after the last committed record when any type of exception occurs. If an SQL exception occurs, the utility does roll back all of the processed data after the last committed record. If a different type of exception occurs, the utility rolls back only the data that is not successfully processed. The load operation commits the successfully processed records. This value is the default value.	Data load order configuration files. For instance, wc-dataload-`catalog-entry`.xml
`shareAssignedAttributeValues`	Indicates whether the Data Load utility supports reusing assigned values when duplicate values are included for different catalog entries in the input file. You can set the following values for this property: true Assigned values are shared when the input file includes duplicate values (default value). false Assigned values are not shared and the utility creates separate records for each duplicate assigned value.	Business object configuration files for loading catalog entry and attribute relationships. wc-loader-catalog-entry-AD-attribute-relationship.xml
`startingSourceLocation`	An optional property to indicate the line in a source CSV file that the Data Load utility begins from. This feature can be used when the file has been loaded up to a certain row, so that the file does not need to be fully loaded, or modified in any way. Note: This property works for CSV source files only. You can set the following values for this property: `integer` Where `integer` is an integer that indicates the row number from which data is loaded from.	All load order configuration files.
`summaryReportLevel`	Indicates whether to include extra information within the summary report that is generated for the load process. If this property is not specified the default summary report generates. You can set the following values for this property: 1 ID resolver information is included within the summary report. This information is appended to the default summary report information. 0 The ID resolver information is not included within the summary report. This value is the default value.	All load order configuration files.
`supportMultipleCalculationCodes`	Indicates whether to multiple calculation codes for each calculation usage. You can set the following values for this property: true Enables support for assigning multiple codes for each calculation usage. false Disables support. When you load a new calculation code and a relationship with a calculation usage exists, the existing relationship is deleted and the new code is assigned to the calculation usage.	Business object configuration files for catalog entries and categories that use one of the following mediators CatalogGroupMediator CatalogGroupCalculationCodeMediator CatalogEntryMediator CatalogEntryCalculationCodeMediator Such as: wc-loader-catalog-group.xml wc-dataload-`catalog-entry`.xml
`supportMultipleValuesForADAttributes`	Indicates whether to support multiple values for attribute dictionary attributes. You can set the following values for this property: true Enables support (default value). false Disables support.	Business object configuration files for attribute dictionary attribute and catalog entry relationships, such as: wc-loader-catalog-entry-AD-attribute-relationship.xml
`syncSalesCatalogs`	Indicates whether to synchronize the product relationships in linked sales catalogs when the load process creates a product under a category. If this flag is set to true, the load process searches all of the sales catalogs that this category is linked to. If the category-product relationship does not exist in a catalog, the load process creates the relationship in the catalog. You can set the following values for this property: true Enables synchronization. Enabling this property can affect performance. false Disables synchronization (default value). Note: This flag must be set for each business object configuration file.	Business object configuration files for catalog entries, such as: wc-loader-catalog-entry.xml wc-loader-catalog-entry-update.xml wc-loader-catalog-group.xml wc-loader-catalog-entry-parent-catalog-group.xml
`validateAttribute`	Indicates whether to run the validation logic to validate that the attribute dictionary attribute or its allowed values belong to the current store. This flag also checks whether the SKU and the product have compatible defining attributes when it moves a SKU under this product. You can set the following values for this property: true Validation logic determines whether the attributes or allowed values to be create/update/delete belong to the current store. false Validation logic does not check attributes. Setting this property to false can improve performance. Ensure that the attributes or allowed values belong to the current store (default value).	Business object configuration files for attribute dictionary attributes, such as: wc-loader-AD-attribute-and-allowed-values.xml wc-loader-AD-attribute-allowed-values.xml
`validateCatalog`	Indicates whether to validate the catalog for the store. Validation checks if more than one master catalog is created for a store. If the store support sales catalog, checking whether a catalog entry already belongs to more than one master category. The validation checks if an attribute allowed value can be set to default in the current store. You can set the following values for this property: true Validation occurs. false Validation does not occur (default value).	Business object configuration files for catalogs, categories, and catalog entries, such as: wc-loader-catalog.xml wc-loader-catalog-group.xml wc-loader-catalog-entry.xml wc-loader-catalog-entry-parent-product.xml wc-loader-catalog-entry-parent-catalog-group.xml
`validateCatalogEntry`	Indicates whether to run the logic to validate that the load operations are valid for the catalog entries. For example, if you set the value for the property to true, a validation exception can occur when you load data to make a product the parent for another product. You can set the following values for this property: true Validation logic checks the types of the SKU and product when the Data Load adds a SKU under a product. This check is to make sure that the SKU is really a SKU and product is really a product. false Validation logic does not check the types of the SKU and product when the Data Load adds a SKU under a product (default value).	Business object configuration files for catalog entries that use the CatalogEntryMediator and CatalogEntryParentProductMediator, such as: wc-loader-AD-attribute-and-allowed-values.xml wc-loader-catalog-entry-parent-product.xml wc-loader-catalog-entry-update.xml wc-loader-catalog-entry.xml
`validateCatalogGroup`	Validates whether the category belongs to the specified catalog. You can set the following values for this property: true Validation occurs. false Validation does not occur (default value). Note: If you set the value to true, you must set the batchSize to 1 because the validation depends on the committed data in the database. For more information, see Configuring the data load order.	Business object configuration files for categories, such as: wc-loader-catalog-group.xml
`validateDelete`	Validates whether a category that is being deleted includes child categories. You can set the following values for this property: true Validation occurs to check whether a child category exists. If a child category exists, the load operation does not delete the specified parent category that is included in the input file. The load operation completes this check only when categories are being explicitly deleted during a load operation. The load operation does not perform this validation when the `markForDelete` configurable property is set to `true`. false Validation does not occur. The load operation deletes the specified category and can delete child categories (default value).	Business object configuration files for categories that use the CatalogGroupMediator business object mediator, such as: wc-loader-catalog-group.xml
`validateScope`	Validation occurs to check whether an object belongs to the store that is specified within business context of the environment configuration file. You can set the following values for this property: true Validation occurs to check whether the object belongs to the context store. If the object does not belong to the store, the utility does not load data for the object. false Validation does not occur (default value).	All business object configuration files for loading catalog data. This parameter is not specified in any sample configuration files.
`queueSize`	Used when parallelization is enabled via the `multipleThreadsEnabled` property. The maximum number of batches that can exist in the queue. Once the queue is filled with the maximum number of batches, the reader waits for batches from the queue to be consumed before continuing to produce and queue further batches. By default, this is set to the `numberOfThreads` property value.	All load order configuration files, within the respective load item.