com.ibm.commerce.foundation.dataload.util

Class DataLoadHelper

  • java.lang.Object
    • com.ibm.commerce.foundation.dataload.util.DataLoadHelper


  • public class DataLoadHelper
    extends java.lang.Object

    This class contains static helper methods for data load.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method and Description
      static void addParameter(java.lang.String nvp)
      Parse the name value pair and add it to the parameters.
      static java.lang.String[] addParameter(java.lang.String nvp, java.util.Map parameters)
      Parse the name value pair and add it to the parameters.
      static java.lang.String addQuote(java.lang.String valueString)
      It adds single quote for the string value in a where clause.
      static java.lang.String appendToFilePath(java.lang.String filePath, java.lang.String suffix)
      Appends a suffix to a file name
      static java.lang.String buildWhereClause(com.ibm.commerce.foundation.dataload.database.ColumnMetaData columnMetaData)
      Builds a where clause based on the column is nullable or not.
      static boolean checkId(java.lang.String tableName, java.util.Map primaryKeyMap)
      Check the primary key to see if the row exists in the database or not.
      static java.lang.String convertMapToPropertiesString(java.util.Map parameterMap)
      Convert a map to a string which contains all name value pairs and each separated by &.
      static java.lang.Float convertMillisecondsToSeconds(long time)
      This method converts milliseconds to seconds
      static java.lang.String convertStackTraceToString(java.lang.Throwable throwable)
      This method converts the stack trace of the Throwable object to a String for tracing purposes.
      static java.lang.String encrypt(java.lang.String value, java.lang.String customKeyConfigFilename, java.lang.String instanceXMLFilename)
      Encrypts the value using WC encryption algorithm.
      static TableDataObject findByPrimaryKey(java.lang.String tableName, java.util.Map primaryKeyData)
      Find a row from the table by primary key and return the populated TableDataObject.
      static TableDataObject findByUniqueIndex(java.lang.String tableName, java.util.Map uniqueIndexDataMap)
      Finds the row of a table by unique index data and returns it as a table data object.
      static java.lang.String formatDecimalString(java.lang.String decimalValue, int decimalDigits)
      Pads the "0" at the end of the decimalValue if the decimal digits in the decimalValue is less than the decimalDigits.
      static java.lang.String formatTimestampString(java.lang.Object timestampObject)
      Format the timestamp object to an ISO standard format with default 6 digits of nanoseconds.
      static java.lang.String formatTimestampString(java.lang.Object timestampObject, int nanoSecondDigits)
      Format the timestamp object to an ISO standard format with nanoSecondDigits.
      static java.lang.String generateDataLoaderUniqueName(java.lang.String dataLoaderName)
      Generate a unique data loader name by appending a date time string at the end of the data loader name.
      static java.lang.String generateDateTimeString()
      Generate a date time string in the format of yyyy.MM.dd_HH.mm.ss.nnn
      static java.lang.String getBaseSchema()
      Get the base schema name in the workspace environment.
      static int getBatchSize(ConfigProperties configProperties)
      Gets the batch size from the configuration properties.
      static java.lang.String getCatalogIdFromCtx()
      Gets the catalog ID from business context.
      static java.lang.Integer getColumnSize(java.lang.String tableName, java.lang.String columnName)
      Gets the column size
      static int[] getColumnTypes(java.lang.String aTableName, java.util.List columnNames)
      Get the column types for a list of column names in the table.
      static java.lang.String getCSVInputDataLine(java.lang.Object inputData, java.lang.String uniqueId, java.lang.String tokenDelimiter)
      Gets the CSV line from the input Data.
      static java.lang.String getCurrencyFromCtx()
      Gets the currency from business context.
      static java.io.File getInputFile(java.lang.String loadItemName)
      Gets the input file from the configuration file.
      static java.lang.String getLangIdFromCtx()
      Gets the language ID from business context.
      static java.util.List<java.lang.String> getListOfRegistryNameToRefresh(ConfigProperties configProperties)
      Gets a list of registry name to be refreshed from the configure properties.
      static java.lang.String getLocalizedMessage(java.lang.String messageKey, java.lang.Object[] params)
      This method returns the localized message from resource bundle associated with the Logger.
      static java.lang.String getLocalizedMessage(java.lang.String messageKey, java.lang.Object[] params, java.util.logging.Logger logger)
      This method returns the localized message from resource bundle associated with the Logger.
      static java.lang.Long getNextKey(java.lang.String aTableName)
      Get the next key from key manager.
      static java.lang.Long getNextKey(java.lang.String aTableName, long startKey, long endKey)
      Get the next key with the fixed key range.
      static java.lang.String getParameter(java.lang.String parameterName)
      Get the parameter value from the Environment.
      static com.ibm.commerce.foundation.dataload.config.QueryProperties getQuery(java.util.List<com.ibm.commerce.foundation.dataload.config.QueryProperties> queryList, java.lang.String queryName)
      Returns the query from the list which has the specified name
      static java.lang.String getReadSchema()
      Get the read schema name in the workspace environment.
      static java.lang.String getStoreIdFromCtx()
      Gets the store ID from business context.
      static long getTableRowCount(java.lang.String aTableName)
      Get the number of rows in the table.
      static java.lang.String getTask()
      Get the task identifier in the workspace environment.
      static java.lang.String getTaskGroup()
      Get the task group identifier in the workspace environment.
      static java.lang.Long getTaskGroupId(java.lang.String taskGroupIdentifier)
      This method will return the task group ID given the external identifier.
      static java.lang.Long getTaskId(java.lang.String taskIdentifier)
      This method will return the task ID given the external identifier.
      static java.util.List getUniqueIndexCoulumnsNames(java.lang.String tableName)
      Get the unique-Index columns name for a table.
      static java.lang.String getWorkspace()
      Get the workspace identifier in the workspace environment.
      static java.lang.Long getWorkspaceId(java.lang.String workspaceIdentifier)
      This method will return the workspace ID given the external identifier.
      static java.lang.String getWriteSchema()
      Get the write schema name in the workspace environment.
      static void initailizeAssociateQuery(java.util.List<com.ibm.commerce.foundation.dataload.config.QueryProperties> associatedQueryList, java.util.List<com.ibm.commerce.foundation.dataload.database.DBPreparedStatement> psArray, java.util.Map idMap)
      Initializes the associated query for the list type parameters.
      static java.io.BufferedWriter initCSVFileWriter(java.lang.String loadItemName, java.lang.String appendName, java.lang.String filePath, java.lang.Object inputData, java.lang.String uniqueIdCSVColumnName, java.lang.String lineSeparator, java.lang.String encoding, java.lang.String tokenDelimiter)
      Initializes the CSV file writer.
      static boolean isBatchEnabled(ConfigProperties configProperties)
      Check from the configuration properties to see if the jdbc batch update is enabled.
      static boolean isCloudscape()
      The method will check the database set in the command line argument is cloudscape or derby.
      static boolean isDataExtract()
      Checks if its the DataExtract utility being executed.
      static boolean isDBConnectionOptional()
      Checks if the dbConnectionOptional = true parameter is specified on the command line.
      static boolean isJ2SE()
      Checks if the dataload is running in J2SE environment and returns a flag accordingly
      static boolean isNumericType(int columnType)
      Check if the columnType is a numeric type based on the constants defined in the java.sql.Types.
      static boolean isOracle()
      The method will check the database set in the command line argument is oracle.
      static boolean isStorePublish()
      Checks if it is running inside the store publish.
      static boolean isValidXmlFile(java.lang.String filePath)
      Checks to see if the file is an XML file.
      static java.lang.String printDataGraph(commonj.sdo.DataGraph dataGraph)
      Prints the contents of a data graph.
      static java.lang.String printDataObject(commonj.sdo.DataObject dataObject)
      This method prints a data object.
      static java.lang.String printDataObject(java.lang.Object dataObject)
      This method prints a data object.
      static void printSQLException(java.sql.SQLException ex)
      Print the detailed exception information including the sql state and error code.
      static java.lang.String prunedString(java.lang.String string, int maxUtf8Length)
      Gets a string pruned to length bytes.
      static java.lang.String rebuildSQLWithMultipleQuestionMarks(java.lang.String sql, int position, int numberOfQuestionMarks)
      Rebuilds the SQL and change a single question mark to multiple question marks.
      static java.lang.String resolveId(java.lang.String tableName, java.util.Map uniqueIndexValueMap, boolean bGenerateNewKey, java.lang.Long startKey, java.lang.Long endKey)
      This method will call resolveIds() to get the primary key map and then get the primary key value from the map.
      static java.util.Map resolveIds(java.lang.String tableName, java.util.Map uniqueIndexValueMap, boolean generateNewKey, java.lang.Long startKey, java.lang.Long endKey)
      It just calls: resolveIds(tableName, uniqueIndexValueMap, generateNewKey, startKey, endKey, null).
      static java.util.Map resolveIds(java.lang.String tableName, java.util.Map uniqueIndexValueMap, boolean generateNewKey, java.lang.Long startKey, java.lang.Long endKey, TableDataObject tableDataObject)
      This method will resolve identifiers for a physical object with multiple primary key columns.
      static void setCurrentSchema(java.lang.String schemaName)
      Change the current schema to the schemaName
      static void setParameter(java.lang.String parameterName, java.lang.String parameterValue)
      Set the parameter name and value to the Environment.
      static void setParameterValue(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement statement, java.lang.String tableName, java.util.List whereClauseColumnNames, java.util.Map whereClauseDataMap)
      Sets the parameter value in the prepared statement.
      static int setWhereClauseColumnValue(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement statement, com.ibm.commerce.foundation.dataload.database.ColumnMetaData columnMetaData, java.lang.Object columnValue, int parameterIndex)
      Sets the where clause column value in the prepared statement.
      static java.util.List<java.lang.String> tokenizeToList(java.lang.String value, java.lang.String delimiter)
      Tokenizes the value string with the delimiter and convert it into a list.
      static java.util.List<java.lang.String> tokenizeToList(java.lang.String value, java.lang.String delimiter, boolean trimWhiteSpace)
      Tokenizes the value string with the delimiter and convert it into a list.
      static void updateLoggerLevel()
      Update the logger level
      static void updateParamValueOfListType(int index, com.ibm.commerce.foundation.dataload.config.QueryProperties query)
      Updates the parameter value for the list type
      static void validateTaskGroupAndTaskStatus(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement taskGroupStatusPS, com.ibm.commerce.foundation.dataload.database.DBPreparedStatement taskStatusPS, java.lang.String taskGroupIdentifier, java.lang.String taskIdentifier)
      This method will validate that the task group and task in a correct status.
      static void validateTaskGroupTaskRelationship(java.lang.String taskGroupIdentifier, java.lang.String taskIdentifier)
      This method will validate that the task group and task identifier provided in the configuration are associated with each other.
      static java.lang.String validateUniqueTaskGroupName(java.lang.String taskGroupName)
      This method will validate that the task group name provided in the configuration is unique and in the correct state.
      static java.lang.String validateUniqueTaskName(java.lang.String taskName)
      This method will validate that the task name provided in the configuration is unique and in the correct state.
      static void validateWorkspace(java.lang.String workspaceIdentifier, java.lang.String taskGroupIdentifier, java.lang.String taskIdentifier)
      This method will validate the workspace information provided.
      static void validateWorkspaceTaskGroupRelationship(java.lang.String workspaceIdentifier, java.lang.String taskGroupIdentifier)
      This method will validate that the workspace and task group identifier provided in the configuration are associated with each other.
      static void warn(java.lang.String messageKey, java.lang.Object[] params, java.util.ResourceBundle resourceBundle)
      Warns the user and prompts the user whether to continue or not.
      static void warn(java.lang.String messageKey, java.util.ResourceBundle resourceBundle)
      Warns the user and prompts the user whether to continue or not.
      static boolean workspaceEnabled()
      Check if the workspace is enabled by check if the workspace is in the context.
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Method Detail

      • getTableRowCount

        public static long getTableRowCount(java.lang.String aTableName)
                                     throws DataLoadException
        Get the number of rows in the table.
        Parameters:
        aTableName - a table name
        Returns:
        the number of rows in the table
        Throws:
        DataLoadException - this exception is thrown if there is a problem to find out the number of row for the table.
      • addQuote

        public static java.lang.String addQuote(java.lang.String valueString)
        It adds single quote for the string value in a where clause. It also escapes the single quotes in the value string by replace a single quote to two single quotes.
        Parameters:
        valueString - the input value string
        Returns:
        the single quote added value string.
      • isNumericType

        public static boolean isNumericType(int columnType)
        Check if the columnType is a numeric type based on the constants defined in the java.sql.Types.
        Parameters:
        columnType - the column type integer.
        Returns:
        true if it is a numeric type, false otherwise.
      • getColumnTypes

        public static int[] getColumnTypes(java.lang.String aTableName,
                                           java.util.List columnNames)
                                    throws DataLoadException
        Get the column types for a list of column names in the table.
        Parameters:
        aTableName - a table name
        columnNames - a list of column names
        Returns:
        the array of column types
        Throws:
        DataLoadException - if the jdbc throws exception.
      • getColumnSize

        public static java.lang.Integer getColumnSize(java.lang.String tableName,
                                                      java.lang.String columnName)
        Gets the column size
        Parameters:
        tableName - a table name
        columnName - a column name
        Returns:
        the column size
      • getNextKey

        public static java.lang.Long getNextKey(java.lang.String aTableName)
                                         throws DataLoadException
        Get the next key from key manager.
        Parameters:
        aTableName - table name
        Returns:
        the next key
        Throws:
        DataLoadException - this exception is thrown if there is some error to get the next key.
      • getNextKey

        public static java.lang.Long getNextKey(java.lang.String aTableName,
                                                long startKey,
                                                long endKey)
                                         throws DataLoadException
        Get the next key with the fixed key range. It still goes through the key manager.
        Parameters:
        aTableName - a table name
        startKey - start key
        endKey - end key
        Returns:
        the next key
        Throws:
        DataLoadException - this exception is thrown if there is some error to get the next key.
      • printDataObject

        public static java.lang.String printDataObject(java.lang.Object dataObject)

        This method prints a data object.

        Parameters:
        dataObject - The data object to print
        Returns:
        A String representation of the data object
      • printDataObject

        public static java.lang.String printDataObject(commonj.sdo.DataObject dataObject)

        This method prints a data object.

        Parameters:
        dataObject - The data object to print
        Returns:
        A String representation of the data object
      • printDataGraph

        public static java.lang.String printDataGraph(commonj.sdo.DataGraph dataGraph)

        Prints the contents of a data graph.

        Parameters:
        dataGraph - The data graph to print.
        Returns:
        A String representation of the data graph.
      • printSQLException

        public static void printSQLException(java.sql.SQLException ex)
        Print the detailed exception information including the sql state and error code.
        Parameters:
        ex - an sql exception
      • isJ2SE

        public static boolean isJ2SE()
        Checks if the dataload is running in J2SE environment and returns a flag accordingly
        Returns:
        true if it is in the J2SE environment, false otherwise.
      • isDataExtract

        public static boolean isDataExtract()
        Checks if its the DataExtract utility being executed.
        Returns:
        true if the DataExtract utility is being executed, false otherwise.
      • getParameter

        public static java.lang.String getParameter(java.lang.String parameterName)
        Get the parameter value from the Environment.
        Parameters:
        parameterName - the parameter name
        Returns:
        the parameter value if it is defined in the Environment. It will return null if it is not defined in the Environment.
      • setParameter

        public static void setParameter(java.lang.String parameterName,
                                        java.lang.String parameterValue)
        Set the parameter name and value to the Environment.
        Parameters:
        parameterName - parameter name
        parameterValue - parameter value
      • addParameter

        public static void addParameter(java.lang.String nvp)
        Parse the name value pair and add it to the parameters.
        Parameters:
        nvp - a string in format of -Dname=value
      • addParameter

        public static java.lang.String[] addParameter(java.lang.String nvp,
                                                      java.util.Map parameters)
        Parse the name value pair and add it to the parameters.
        Parameters:
        nvp - a string in format of -Dname=value
        parameters - a map of (parameterName, parameterValue) pairs; default to Environment.properties if unspecified
        Returns:
        (parameterName, parameterValue) pair
      • appendToFilePath

        public static java.lang.String appendToFilePath(java.lang.String filePath,
                                                        java.lang.String suffix)
        Appends a suffix to a file name
        Parameters:
        filePath - the old file path
        suffix - a suffix to be appended
        Returns:
        the file name appended with the suffix.
      • updateLoggerLevel

        public static void updateLoggerLevel()
        Update the logger level
      • formatTimestampString

        public static java.lang.String formatTimestampString(java.lang.Object timestampObject,
                                                             int nanoSecondDigits)
        Format the timestamp object to an ISO standard format with nanoSecondDigits.
        Parameters:
        timestampObject - the timestamp object
        nanoSecondDigits - the number of nanosecond digits.
        Returns:
        the formatted timestamp string.
      • formatTimestampString

        public static java.lang.String formatTimestampString(java.lang.Object timestampObject)
        Format the timestamp object to an ISO standard format with default 6 digits of nanoseconds.
        Parameters:
        timestampObject - the timestamp object
        Returns:
        the formatted timestamp string.
      • formatDecimalString

        public static java.lang.String formatDecimalString(java.lang.String decimalValue,
                                                           int decimalDigits)
        Pads the "0" at the end of the decimalValue if the decimal digits in the decimalValue is less than the decimalDigits. if the decimal digits in the decimalValue is more than the decimalDigits, it will truncate the decimalValue. For example, if the decimalValue = "0.1", and decimalDigits = 5, it will return "0.10000". If the decimalValue = "0", and decimalDigits = 5, it will return "0.00000".
        Parameters:
        decimalValue - a decimal value string
        decimalDigits - number of decimal digits
        Returns:
        the formated decimal value string.
      • generateDataLoaderUniqueName

        public static java.lang.String generateDataLoaderUniqueName(java.lang.String dataLoaderName)

        Generate a unique data loader name by appending a date time string at the end of the data loader name.

        Parameters:
        dataLoaderName - The configured name of the data loader.
        Returns:
        String The configured name of the data loader with the current date and time appended to it.
      • generateDateTimeString

        public static java.lang.String generateDateTimeString()
        Generate a date time string in the format of yyyy.MM.dd_HH.mm.ss.nnn
        Returns:
        the generated date time string based on the current time.
      • getLocalizedMessage

        public static java.lang.String getLocalizedMessage(java.lang.String messageKey,
                                                           java.lang.Object[] params,
                                                           java.util.logging.Logger logger)

        This method returns the localized message from resource bundle associated with the Logger.

        Parameters:
        messageKey - The message key to lookup the localized message.
        params - The message parameters to be substituted in into the localized message.
        logger - a Logger object.
        Returns:
        String The localized message.
      • getLocalizedMessage

        public static java.lang.String getLocalizedMessage(java.lang.String messageKey,
                                                           java.lang.Object[] params)

        This method returns the localized message from resource bundle associated with the Logger.

        Parameters:
        messageKey - The message key to lookup the localized message.
        params - The message parameters to be substituted in into the localized message.
        Returns:
        String The localized message.
      • convertMillisecondsToSeconds

        public static java.lang.Float convertMillisecondsToSeconds(long time)

        This method converts milliseconds to seconds

        Parameters:
        time - The time, in milliseconds.
        Returns:
        float The time, in seconds.
      • getBatchSize

        public static int getBatchSize(ConfigProperties configProperties)
        Gets the batch size from the configuration properties. The default is 1 if not configured.
        Parameters:
        configProperties - configuration properties.
        Returns:
        the batch size.
      • isBatchEnabled

        public static boolean isBatchEnabled(ConfigProperties configProperties)
        Check from the configuration properties to see if the jdbc batch update is enabled. The default is not enabled. If the batchSize is not 1, the batch is enabled.
        Parameters:
        configProperties - configuration properties.
        Returns:
        true if the batch is enabled, false otherwise.
      • validateWorkspace

        public static void validateWorkspace(java.lang.String workspaceIdentifier,
                                             java.lang.String taskGroupIdentifier,
                                             java.lang.String taskIdentifier)
                                      throws DataLoadException

        This method will validate the workspace information provided.

        The following information will be validated:

        • The workspace exists.
        • The task group exists.
        • The task exists.
        • The workspace is in the state 'Active'. (1=active, 2=complete, 3=cancelled, 4=cancel in progress)
        • The task group is in the state 'Working'. (0=inactive. 1=working. 2=ready for approval. 3=approved. 4=ready to commit. 5=complete. 6=cancelled. 7=commit failed. 8=commit in progress. 9=committed. 10=ready to publish. 11=publish. 12=publish in progress. 13=publish failed.)
        • The task is in the state 'Working'. (0=Inactive 1=Working 2=Complete)

        Parameters:
        workspaceIdentifier - The workspace identifier.
        taskGroupIdentifier - The task group identifier.
        taskIdentifier - The task identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the workspace information is invalid.
      • validateUniqueTaskGroupName

        public static java.lang.String validateUniqueTaskGroupName(java.lang.String taskGroupName)
                                                            throws DataLoadException

        This method will validate that the task group name provided in the configuration is unique and in the correct state. If it is unique the task group identifier will be returned otherwise an exception will be thrown.

        Parameters:
        taskGroupName - The task group name.
        Returns:
        taskGroupIdentifier The task group identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the task group name is not unique.
      • validateUniqueTaskName

        public static java.lang.String validateUniqueTaskName(java.lang.String taskName)
                                                       throws DataLoadException

        This method will validate that the task name provided in the configuration is unique and in the correct state. If it is unique the task identifier will be returned otherwise an exception will be thrown.

        Parameters:
        taskName - The task name.
        Returns:
        taskIdentifier The task identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the task name is not unique.
      • validateTaskGroupAndTaskStatus

        public static void validateTaskGroupAndTaskStatus(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement taskGroupStatusPS,
                                                          com.ibm.commerce.foundation.dataload.database.DBPreparedStatement taskStatusPS,
                                                          java.lang.String taskGroupIdentifier,
                                                          java.lang.String taskIdentifier)
                                                   throws DataLoadException
        This method will validate that the task group and task in a correct status. The valid status for task group and task is (1, 1) or (2, 0). That is, the task group and task can be in both active status or task group in ready for approval status and the task in inactive status.
        Parameters:
        taskGroupStatusPS - a prepared statement to get task group status.
        taskStatusPS - a prepared statement to get task status.
        taskGroupIdentifier - a task group identifier.
        taskIdentifier - a task identifier
        Throws:
        DataLoadException - if getting the status of task group or task is failed, or the validation of the status is failed.
      • getWorkspaceId

        public static java.lang.Long getWorkspaceId(java.lang.String workspaceIdentifier)
                                             throws DataLoadException

        This method will return the workspace ID given the external identifier.

        Parameters:
        workspaceIdentifier - The workspace identifier.
        Returns:
        Long The workspace internal identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the was a problem determining the workspace id.
      • getTaskGroupId

        public static java.lang.Long getTaskGroupId(java.lang.String taskGroupIdentifier)
                                             throws DataLoadException

        This method will return the task group ID given the external identifier.

        Parameters:
        taskGroupIdentifier - The task group identifier.
        Returns:
        Long The task group internal identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the was a problem determining the task group id.
      • getTaskId

        public static java.lang.Long getTaskId(java.lang.String taskIdentifier)
                                        throws DataLoadException

        This method will return the task ID given the external identifier.

        Parameters:
        taskIdentifier - The task identifier.
        Returns:
        Long The task internal identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the was a problem determining the task id.
      • validateWorkspaceTaskGroupRelationship

        public static void validateWorkspaceTaskGroupRelationship(java.lang.String workspaceIdentifier,
                                                                  java.lang.String taskGroupIdentifier)
                                                           throws DataLoadException

        This method will validate that the workspace and task group identifier provided in the configuration are associated with each other.

        Parameters:
        workspaceIdentifier - The workspace identifier.
        taskGroupIdentifier - The task group identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the workspace and task group are not related.
      • validateTaskGroupTaskRelationship

        public static void validateTaskGroupTaskRelationship(java.lang.String taskGroupIdentifier,
                                                             java.lang.String taskIdentifier)
                                                      throws DataLoadException

        This method will validate that the task group and task identifier provided in the configuration are associated with each other.

        Parameters:
        taskGroupIdentifier - The task group identifier.
        taskIdentifier - The task identifier.
        Throws:
        DataLoadException - A configuration exception will be thrown if the task group and task are not related.
      • getBaseSchema

        public static java.lang.String getBaseSchema()

        Get the base schema name in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The base schema name.
      • getWriteSchema

        public static java.lang.String getWriteSchema()

        Get the write schema name in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The write schema name.
      • getReadSchema

        public static java.lang.String getReadSchema()

        Get the read schema name in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The read schema name.
      • workspaceEnabled

        public static boolean workspaceEnabled()

        Check if the workspace is enabled by check if the workspace is in the context.

        Returns:
        boolean Returns true if and only if the workspace is enabled. Otherwise it will return false.
      • getWorkspace

        public static java.lang.String getWorkspace()

        Get the workspace identifier in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The workspace identifier.
      • getTaskGroup

        public static java.lang.String getTaskGroup()

        Get the task group identifier in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The task group identifier.
      • getTask

        public static java.lang.String getTask()

        Get the task identifier in the workspace environment. If it is not in the workspace environment, it returns null.

        Returns:
        String The task identifier.
      • setCurrentSchema

        public static void setCurrentSchema(java.lang.String schemaName)
                                     throws DataLoadException
        Change the current schema to the schemaName
        Parameters:
        schemaName - a schema name
        Throws:
        DataLoadException - this exception is thrown if there is a problem to get the database connection.
      • getUniqueIndexCoulumnsNames

        public static java.util.List getUniqueIndexCoulumnsNames(java.lang.String tableName)
                                                          throws DataLoadException
        Get the unique-Index columns name for a table. return empty list if unique index is not defined for a table.
        Parameters:
        tableName - a table name
        Returns:
        List a list of unique-index columns name
        Throws:
        DataLoadException - if more than one unique-index are defined for a table
      • warn

        public static void warn(java.lang.String messageKey,
                                java.lang.Object[] params,
                                java.util.ResourceBundle resourceBundle)
                         throws DataLoadException
        Warns the user and prompts the user whether to continue or not. If it is not in the J2SE environment, it just continues.
        Parameters:
        messageKey - the warning message key
        params - the message parameters to be substituted in into the localized message
        resourceBundle - the resource bundle
        Throws:
        DataLoadException - A system exception will be thrown if the user replied not to continue.
      • warn

        public static void warn(java.lang.String messageKey,
                                java.util.ResourceBundle resourceBundle)
                         throws DataLoadException
        Warns the user and prompts the user whether to continue or not. If it is not in the J2SE environment, it just continues.
        Parameters:
        messageKey - the warning message key
        resourceBundle - the resource bundle
        Throws:
        DataLoadException - A system exception will be thrown if the user replied not to continue.
      • isCloudscape

        public static boolean isCloudscape()
        The method will check the database set in the command line argument is cloudscape or derby.
        Returns:
        true if the database is cloudscape or derby otherwise false.
      • isOracle

        public static boolean isOracle()
        The method will check the database set in the command line argument is oracle.
        Returns:
        true if the database is oracle otherwise false.
      • tokenizeToList

        public static java.util.List<java.lang.String> tokenizeToList(java.lang.String value,
                                                                      java.lang.String delimiter,
                                                                      boolean trimWhiteSpace)
        Tokenizes the value string with the delimiter and convert it into a list.
        Parameters:
        value - a value string
        delimiter - a delimiter string
        trimWhiteSpace - indicate whether to trim the white pace for each token.
        Returns:
        tokenized list.
      • tokenizeToList

        public static java.util.List<java.lang.String> tokenizeToList(java.lang.String value,
                                                                      java.lang.String delimiter)
        Tokenizes the value string with the delimiter and convert it into a list.
        Parameters:
        value - a value string
        delimiter - a delimiter string
        Returns:
        tokenized list.
      • findByPrimaryKey

        public static TableDataObject findByPrimaryKey(java.lang.String tableName,
                                                       java.util.Map primaryKeyData)
                                                throws DataLoadException
        Find a row from the table by primary key and return the populated TableDataObject.
        Parameters:
        tableName - the table name
        primaryKeyData - a map containing the primary key name to the value map
        Returns:
        the populated TableDataObject, if the object is not found, a null will be return.
        Throws:
        DataLoadException - this error is thrown if the database throws the sql exception
      • findByUniqueIndex

        public static TableDataObject findByUniqueIndex(java.lang.String tableName,
                                                        java.util.Map uniqueIndexDataMap)
                                                 throws DataLoadException
        Finds the row of a table by unique index data and returns it as a table data object.
        Parameters:
        tableName - a table name
        uniqueIndexDataMap - a unique index data map
        Returns:
        a table data object representing a row matches the unique index data. It returns null if the data cannot be found in the database.
        Throws:
        DataLoadException - if there are some errors.
      • setParameterValue

        public static void setParameterValue(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement statement,
                                             java.lang.String tableName,
                                             java.util.List whereClauseColumnNames,
                                             java.util.Map whereClauseDataMap)
                                      throws DataLoadException
        Sets the parameter value in the prepared statement.
        Parameters:
        statement - the prepared statement
        tableName - a table name
        whereClauseColumnNames - a list of column names in the where clause
        whereClauseDataMap - the column value to be set for these columns in the where clause
        Throws:
        DataLoadException - if there are some errors.
      • buildWhereClause

        public static java.lang.String buildWhereClause(com.ibm.commerce.foundation.dataload.database.ColumnMetaData columnMetaData)
        Builds a where clause based on the column is nullable or not. If the column is not null, the where clause is simple as "columnName = ?". If the column is nullable, the where clause will be: "((columnName=?) OR ((0 = ?) AND (columnName IS NULL)))".
        Parameters:
        columnMetaData - a column meta data object
        Returns:
        the where clause.
      • setWhereClauseColumnValue

        public static int setWhereClauseColumnValue(com.ibm.commerce.foundation.dataload.database.DBPreparedStatement statement,
                                                    com.ibm.commerce.foundation.dataload.database.ColumnMetaData columnMetaData,
                                                    java.lang.Object columnValue,
                                                    int parameterIndex)
                                             throws DataLoadException
        Sets the where clause column value in the prepared statement. It will depend on the column is nullable or not. If the column is nullable, it will set two parameter values. See the buildWhereClause() method for the where clause it builds.
        Parameters:
        statement - a prepared statement
        columnMetaData - a column meta data object
        columnValue - a column value
        parameterIndex - a parameter index
        Returns:
        the parameter index for the next set value.
        Throws:
        DataLoadException - if there are some errors.
      • isStorePublish

        public static boolean isStorePublish()
        Checks if it is running inside the store publish.
        Returns:
        true if it is running inside the store publish.
      • resolveId

        public static java.lang.String resolveId(java.lang.String tableName,
                                                 java.util.Map uniqueIndexValueMap,
                                                 boolean bGenerateNewKey,
                                                 java.lang.Long startKey,
                                                 java.lang.Long endKey)
                                          throws DataLoadException

        This method will call resolveIds() to get the primary key map and then get the primary key value from the map. It can only be called if the primary key only contains one column. If the primary key contains multiple columns, it will return null.

        Parameters:
        tableName - The table name to check for a key to resolve.
        uniqueIndexValueMap - A Map containing unique index columns and their values.
        bGenerateNewKey - A boolean variable to indicate if an identifier should be fetched from the key manager.
        startKey - a start key for the key range. If startKey or endKey is null, the key range will be determined by KEYS table.
        endKey - an end key for the key range.
        Returns:
        String The resolved primary key for the table and unique index information.
        Throws:
        DataLoadException - An application exception will be thrown if there was a problem resolving the identifier.
      • resolveIds

        public static java.util.Map resolveIds(java.lang.String tableName,
                                               java.util.Map uniqueIndexValueMap,
                                               boolean generateNewKey,
                                               java.lang.Long startKey,
                                               java.lang.Long endKey,
                                               TableDataObject tableDataObject)
                                        throws DataLoadException

        This method will resolve identifiers for a physical object with multiple primary key columns.

        Parameters:
        tableName - The table name to check for a key to resolve.
        uniqueIndexValueMap - A Map containing unique index columns and their values.
        generateNewKey - A boolean variable to indicate if an identifier should be fetched from the key manager.
        startKey - a start key for the key range. If startKey or endKey is null, the key range will be determined by KEYS table.
        endKey - an end key for the key range.
        tableDataObject - an optional table data object. If it is not null, the unique index and the resolve primary will all be populated into the table data object.
        Returns:
        The resolved primary key for the table and unique index information.
        Throws:
        DataLoadException - An application exception will be thrown if there was a problem resolving the identifier.
      • resolveIds

        public static java.util.Map resolveIds(java.lang.String tableName,
                                               java.util.Map uniqueIndexValueMap,
                                               boolean generateNewKey,
                                               java.lang.Long startKey,
                                               java.lang.Long endKey)
                                        throws DataLoadException
        It just calls: resolveIds(tableName, uniqueIndexValueMap, generateNewKey, startKey, endKey, null).
        Parameters:
        tableName - The table name to check for a key to resolve.
        uniqueIndexValueMap - A Map containing unique index columns and their values.
        generateNewKey - A boolean variable to indicate if an identifier should be fetched from the key manager.
        startKey - a start key for the key range. If startKey or endKey is null, the key range will be determined by KEYS table.
        endKey - an end key for the key range. all be populated into the table data object.
        Returns:
        The resolved primary key for the table and unique index information.
        Throws:
        DataLoadException - An application exception will be thrown if there was a problem resolving the identifier.
      • checkId

        public static boolean checkId(java.lang.String tableName,
                                      java.util.Map primaryKeyMap)
                               throws DataLoadException
        Check the primary key to see if the row exists in the database or not.
        Parameters:
        tableName - the table name
        primaryKeyMap - a map contains the primary key data.
        Returns:
        true if the primary key exists in the database, false otherwise.
        Throws:
        DataLoadException - this exception is thrown if there are some problems during the check.
      • encrypt

        public static java.lang.String encrypt(java.lang.String value,
                                               java.lang.String customKeyConfigFilename,
                                               java.lang.String instanceXMLFilename)
                                        throws DataLoadException
        Encrypts the value using WC encryption algorithm. The merchant keys can be in either custom key configuration file or in the instance XML file. If neither of the files are specified, it will throw an exception. If they are both specified, the custom key configuration file will be used and the instance XML file will be ignored.
        Parameters:
        value - the value to be encrypted.
        customKeyConfigFilename - the file path to custom key configuration file
        instanceXMLFilename - to file path to the instance XML file.
        Returns:
        the encrypted value
        Throws:
        DataLoadException - if both custom key configuration file and instance XML file are not specified or the encryption has some errors.
      • getInputFile

        public static java.io.File getInputFile(java.lang.String loadItemName)
        Gets the input file from the configuration file. If it is not defined in the configuration, it returns null.
        Parameters:
        loadItemName - a load item name
        Returns:
        the input file
      • initCSVFileWriter

        public static java.io.BufferedWriter initCSVFileWriter(java.lang.String loadItemName,
                                                               java.lang.String appendName,
                                                               java.lang.String filePath,
                                                               java.lang.Object inputData,
                                                               java.lang.String uniqueIdCSVColumnName,
                                                               java.lang.String lineSeparator,
                                                               java.lang.String encoding,
                                                               java.lang.String tokenDelimiter)
                                                        throws DataLoadException
        Initializes the CSV file writer.
        Parameters:
        loadItemName - a load item name
        appendName - a name to be appended to the original CSV file if the new CSV file is not specified.
        filePath - a file path specified for the new CSV file.
        inputData - the input data from CSV reader
        uniqueIdCSVColumnName - a unique ID CSV column name.
        lineSeparator - the line separator for the CSV file
        encoding - the encoding for the CSV file
        tokenDelimiter - the taken delimiter for the CSV file
        Returns:
        the buffered writer for the CSV
        Throws:
        DataLoadException - if the buffered writer cannot be created due to file path or access denied.
      • getCSVInputDataLine

        public static java.lang.String getCSVInputDataLine(java.lang.Object inputData,
                                                           java.lang.String uniqueId,
                                                           java.lang.String tokenDelimiter)
        Gets the CSV line from the input Data. If the tokenDelimiter is specified, it will prefix the line with the unique ID and the token delimiter.
        Parameters:
        inputData - the input data map from the CSV reader.
        uniqueId - a unique ID
        tokenDelimiter - a token delimiter.
        Returns:
        the CSV line. It may include the prefix of the unique ID and the token delimiter.
      • isValidXmlFile

        public static boolean isValidXmlFile(java.lang.String filePath)
                                      throws DataLoadException
        Checks to see if the file is an XML file.
        Parameters:
        filePath - a file path
        Returns:
        true if it is an XML file.
        Throws:
        DataLoadException - if there are some errors.
      • getQuery

        public static com.ibm.commerce.foundation.dataload.config.QueryProperties getQuery(java.util.List<com.ibm.commerce.foundation.dataload.config.QueryProperties> queryList,
                                                                                           java.lang.String queryName)
        Returns the query from the list which has the specified name
        Parameters:
        queryList - a list of query
        queryName - a specified query name
        Returns:
        the query with the specified name
      • rebuildSQLWithMultipleQuestionMarks

        public static java.lang.String rebuildSQLWithMultipleQuestionMarks(java.lang.String sql,
                                                                           int position,
                                                                           int numberOfQuestionMarks)
                                                                    throws DataLoadException
        Rebuilds the SQL and change a single question mark to multiple question marks.
        Parameters:
        sql - the original SQL
        position - the position of the question mark to be replaced.
        numberOfQuestionMarks - the number of question marks to be replaced.
        Returns:
        the modified SQL
        Throws:
        DataLoadException - if there are no question mark specified in the position.
      • isDBConnectionOptional

        public static boolean isDBConnectionOptional()
        Checks if the dbConnectionOptional = true parameter is specified on the command line. If it is data extract, it is optional.
        Returns:
        true if it is specified. The default is false.
      • convertMapToPropertiesString

        public static java.lang.String convertMapToPropertiesString(java.util.Map parameterMap)
        Convert a map to a string which contains all name value pairs and each separated by &. It becomes name1=value1&name2=value2...
        Parameters:
        parameterMap - a map object
        Returns:
        a string which contains all name value pairs in the map.
      • initailizeAssociateQuery

        public static void initailizeAssociateQuery(java.util.List<com.ibm.commerce.foundation.dataload.config.QueryProperties> associatedQueryList,
                                                    java.util.List<com.ibm.commerce.foundation.dataload.database.DBPreparedStatement> psArray,
                                                    java.util.Map idMap)
                                             throws DataLoadException
        Initializes the associated query for the list type parameters. It may duplicated the number of queries in the list.
        Parameters:
        associatedQueryList - a list of queries
        psArray - a list of prepared satements.
        idMap - a map object.
        Throws:
        DataLoadException - if there are some errors.
      • updateParamValueOfListType

        public static void updateParamValueOfListType(int index,
                                                      com.ibm.commerce.foundation.dataload.config.QueryProperties query)
        Updates the parameter value for the list type
        Parameters:
        index - the index in the list value.
        query - a query properties
      • getCatalogIdFromCtx

        public static java.lang.String getCatalogIdFromCtx()
                                                    throws DataLoadException
        Gets the catalog ID from business context. If the catalog ID is not specified, a null will be returned.
        Returns:
        the catalog ID in the business context.
        Throws:
        DataLoadException - if there are some errors.
      • getStoreIdFromCtx

        public static java.lang.String getStoreIdFromCtx()
                                                  throws DataLoadException
        Gets the store ID from business context. If the store ID is not defined in the business context, a null will be returned.
        Returns:
        the store ID in business context.
        Throws:
        DataLoadException - if there are some errors.
      • getLangIdFromCtx

        public static java.lang.String getLangIdFromCtx()
                                                 throws DataLoadException
        Gets the language ID from business context. If the language ID is not defined in the business context, the English language ID -1 will be returned.
        Returns:
        the language ID in business context.
        Throws:
        DataLoadException - if there are some errors.
      • getCurrencyFromCtx

        public static java.lang.String getCurrencyFromCtx()
                                                   throws DataLoadException
        Gets the currency from business context. If the currency is not defined in the business context, a null will be returned.
        Returns:
        the currency in business context.
        Throws:
        DataLoadException - if there are some errors.
      • prunedString

        public static java.lang.String prunedString(java.lang.String string,
                                                    int maxUtf8Length)
        Gets a string pruned to length bytes. This method is usually used to truncate a string so that it can be fit into a database column. The string is converted to byte array using UTF-8 encoding, then use the specified length to truncate. To avoid cut half of a multiple bytes character, the pruned string may be slightly shorter than the specified length.
        Parameters:
        string - The string to be pruned.
        maxUtf8Length - The byte length to prune to. It is usually the size of database column.
        Returns:
        the string pruned to length bytes or slightly less.
      • convertStackTraceToString

        public static java.lang.String convertStackTraceToString(java.lang.Throwable throwable)
        This method converts the stack trace of the Throwable object to a String for tracing purposes.
        Parameters:
        throwable - The Throwable object.
        Returns:
        The String representation of the stack trace.
      • getListOfRegistryNameToRefresh

        public static java.util.List<java.lang.String> getListOfRegistryNameToRefresh(ConfigProperties configProperties)
        Gets a list of registry name to be refreshed from the configure properties.
        Parameters:
        configProperties - a configure properties
        Returns:
        a list of registry name.