Migrating Metrics data from Cognos® PowerCube to Elasticsearch

If you previously used Cognos® to provide Metrics data for your HCL Connections™ deployment, Metrics reports are stored in Cognos® PowerCube. You can migrate existing Metrics report data from Cognos® to Elasticsearch.  

Before you begin

Before you begin migrating data from PowerCube, verify that the following prerequisites have been satisfied:

  • The Metrics component was upgraded to the Elasticsearch-based version and functions correctly.
  • The Cognos PowerCube server is available, and is running WebSphere® Application Server 8.5.5.14 with JDK 7. For instructions on updating the server to use JDK 7, see Troubleshooting Cognos Metrics.

About this task

There will be 201 Global reports and 154 Community reports (for each community) to be migrated. Those reports include basic line chart, apps comparing pie chart, and the attributes comparing pie chart. After migration is complete, those reports can be queried in the new Metrics UI.

Migrating data from Cognos® PowerCube to Elasticsearch has the following limitations:
  • Only month-level and year-level reports are supported for a history data query from PowerCube (day-level and week-level reports are not supported).
  • Customized reports are not supported for a history data query from PowerCube.

Procedure

  1. Download and extract the migration tool package.
    Note: The Metrics Cognos Reports migration tool download can be found at Fix Central. Make sure to download the version for the IC 6.0 CR that is deployed on your system.
    1. Download the metrics migration tool package cubeMigration.zip .
    2. Extract it to a temporary location; for example: ~/migration_tool/
  2. Add migration information to the extracted config.ini file.
    1. Back in the extracted package (~/migration_tool/), edit the config.ini file and replace variables with the information for your deployment.
      The following code snippet shows a sample of the config.ini file:
      
### [ JAVA Env ]
JAVA_HOME = /opt/jdk1.8.0_162/      # The JAVA home directory which contains /bin/java
Xmx = 512m                          # The maximum java heap size
Xms = 128m                          # The minimum java heap size


### [ Cognos ]
cognos.installPath = /opt/cognos1022        # The Cognos install location
cognos.internalURL = http://cogserver.ibm.com/cognos/servlet/dispatch   # It's the internalDispatcher's value in <cognos.installPath>/cogstartup.xml
cognos.username = cognos
cognos.password = password


### [ Elasticsearch ]
es.hosts = host1, host2, host3      # The Elasticsearch server's host name or IP address. Use commas to separate if multiple hosts.
es.port = 9200
es.keyStore = /home/root/elasticsearch-metrics.p12  # The client key store used to connect to Elasticsearch server
es.keyStorePass = password          # The password of client key store


### [ Migration ]
mt.reset = false                    # Set true to restart migration from beginning. If false, will start from last break.
mt.interval = 0                     # Interval in seconds between migrating every community's cube.
mt.batchSize = 100                  # The number of communities to migrate in a batch.
      Note: Windows™ users:
      • Use an editor that recognizes the \n line-break symbol. Examples of supported editors include: Notepad++, Editplus, UltraEdit, Vi for Windows.
      • When specifying paths, use either a double backslash (\\) or a forward slash (/) as the separator between directories; for example: C:\\Program Files\\Cognos or C:/Program Files/Cognos
    2. Save and close the file.
  3. Move the MigrationPkg.zip to the BI deployment folder.
    1. In the root of the extracted migration tool package, locate the MigrationPkg.zip file (for example, in ~/migration_tool/MigrationPkg.zip).
    2. Copy the MigrationPkg.zip to the following location: Cognos_Install_Path/CognosBI/deployment/
  4. Start the migration.
    1. Open a command console or terminal window.
    2. Change to the root folder of the unzipped cubeMigration.zip (~/migration_tool/).
    3. Run the appropriate Java™ command for your operating system:

      If you did not add the Java path to your "path" environment settings, you can include the path in the command now. Use the full path of java to launch the migration tool; for example, C:\IBM\WebSphere\AppServer\java_1.8_64\jre\bin\java -classpath.

      Important: Do not set the JAVA_HOME environment settings in the operating system.
      • Linux®:
        java -classpath './lib/*:Cognos_lib_path/*' com.ibm.connections.metrics.cognos.cube.migration.CubeMigration
      • Windows™:
        java -classpath "./lib/*;Cognos_lib_path/*" com.ibm.connections.metrics.cognos.cube.migration.CubeMigration

      Where:

      • Cognos_lib_path points to the library path of the Cognos® server; for example:

        C:/IBM/Cognos/CognosBI/webapps/p2pd/WEB-INF/lib

      Attention: The migration process might run for hours or even days, depending on the number of communities and the amount of data to be migrated.
  5. Monitor the migration process.

    When the migration tool starts, it runs a properties check to verify the information provided in the config.ini file. If an invalid value is found, the migration process exits and posts the reason in both the command console and the log file.

    If the properties in the config.ini file are valid, then the migration begins. The process includes several phases; each phase provides status information in both the command console and the log file (which contains more detail). You can use the status information to monitor the migration progress.

    The following example shows typical output from the migration process:

    ########## Migration start at 2018-08-14 09:00:44 ##########>>> Load properties successfully>>> Connected to cognos server: Logon successful as wpsadmin>>> Connected to ES: ES cluster status is green>>> Cube report deployment completed successfully. Took 1 seconds>>> Global cube data migration completed successfully. Took 15 seconds>>> Cube date update completed successfully. Took 0 seconds>>> Community list initialization completed successfully. Took 3 secondsTotal 99, successful 0, failed 0 > (1/99) Migration of data for community a9ad618e-d74c-4ea4-b10c-ece66c20b9fd successfully > (2/99) Migration of data for community 365a17b2-e17e-47e0-9a71-8cf34000f8f0 successfully > (3/99) Migrate reports for community bacf08cc-2389-45ed-9634-aa0d4d3ac844 successfully ...... ...... > (97/99) Migration of data for community c52c05bd-5d62-4e1f-a766-1049b745a5fb successfully > (98/99) Migrate reports for community 865ddb3a-e28d-4fd9-8829-313ff282d758 successfully > (99/99) Migration of data for community e55d7fc1-e786-442a-a2ee-9f05ce0f83d3 successfullyTotal 99, successful 99, failed 0>>> Communities cube data migration completed successfully. Took 2 minutes 17 seconds

  6. Check the migration results.
    • Use the Metrics UI to view migrated reports and verify that they display correctly.
    • Review the migration logs, which are stored in the ~/migration_tool/log.out file.
    • Review the debug log, which is stored in ~/migration_tool/debug.out if you included the optional -Dlog4j.configuration=debug.properties parameter with the migration command.
    If any community data failed to migrate, one or more of the following codes might show in the migration logs:
    • {@code -1} -- Failed to create index
    • {@code -2} -- Failed to run Cognos® reports
    • {@code -3} -- Failed to transform reports
    • {@code -4} -- Failed to save reports to Elasticsearch
  7. Resolve any issues with data that failed to migrate.

    If the migration process completed but the data for some communities was not successfully migrated, resolve the data issues before remigrating. For information on resolving migration issues, see Troubleshooting Elasticsearch Metrics.

  8. Resume or restart the migration.
    Resume migration
    If any failures caused the migration process to stop (or if the process was manually stopped by a user), you can return to step 4 and restart the migration tool by running the migration command again. The migration tool resumes from the point where it was stopped.
    Remigrate failed communities
    If you corrected data that failed to migrate in the previous pass, you can remigrate just the failed data by returning to step 4 and running the migration command again with this additional parameter: -DmigrateFailed=true; for example:
    java -DmigrateFailed=true -classpath './lib/*:Cognos_lib_path/*' com.ibm.connections.metrics.cognos.cube.migration.CubeMigration
    Only the failed communities will have data remigrated to Elasticsearch.
    Restart migration
    If you want to start the migration from the beginning, edit the config.ini file and set mt.reset = true (in the [ Migration ] section of the file), and then return to step 4 to run the migration again. The previously migrated data will be removed, and all data will be migrated.
  9. Remove sensitive data from the config.ini file.