Migrating Metrics data from the RDBMS to Elasticsearch

If you previously used Cognos® to provide Metrics data for your HCL Connections™ deployment, Metrics events data is stored in your RDBMS, while older data is stored in the Cognos Power Cube. You can migrate the existing events data in your RDBMS to the new stand-alone Elasticsearch instance.

Procedure

  1. From the Deployment Manager system, load the Python script by entering the following commands: 
    cd /opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin
sudo sh wsadmin.sh -lang jython -user wasadmin_user -password wasadmin_password
execfile('metricsEventCapture.py')
  2. Migrate events and populate user attributes.

    Table 1 lists the possible migration commands and describes what they do; the list following Table 1 suggests which migration commands best suit different customer situations.

    Table 1. Migration task commands and descriptions
    Command name Description
    MigraterService.migrate() Migrate all of existing events data
    MigraterService.migrateFrom('YYYYMMDD') Migrate only data created after a certain date
    MigraterService.populateUserAttributes() Populate user attributes for events in Elasticsearch
    MigraterService.migrateAndPopulateUserAttributes() Migrate all of existing events data, and populate user attributes for events in Elasticsearch
    MigraterService.migrateAndPopulateUserAttributesFrom('YYYYMMDD') Migrate only data created after a certain date, and populate user attributes for events in Elasticsearch

    When choosing the migration command, review the following considerations:

    • If you have already migrated events data, populate user attributes for events in Elasticsearch by running the following command: MigraterService.populateUserAttributes()
    • If you have not migrated events data, you can migrate events and user attributes population for events together, by running one of the following commands:
      • MigraterService.migrateAndPopulateUserAttributes()
      • MigraterService.migrateAndPopulateUserAttributesFrom('YYYYMMDD')
    • If you have not migrated events data and you want to separate the tasks of migrating events data and populating user attributes, run the following commands separately:
      1. Migrate events data to Elasticsearch: MigraterService.migrate() or MigraterService.migrateFrom('YYYYMMDD')
      2. Populate user attributes to the events migrated in Elasticsearch: MigraterService.migrateAndPopulateUserAttributes()
    The command returns immediately, but the task continues to run on the server as necessary. Monitor the progress by viewing the logs at /opt/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/Metrics_Server:
    • MetricsMigration_YY.MM.DD_HH:MM:SS.log
    • PopulateUserAttributes_YY.YY.DD_HH:MM:SS.log
  3. Validate that the Elasticsearch user experience is functioning well by updating the browser URL to include the test context root; for example, by replacing "/metrics" with "/metricssc" in the URL for Global Metrics or Community Metrics.
  4. When validation is complete, run the following Python script to switch users to new metrics app.

    This script causes the RDBMS-based app to stop capturing data, and the Elasticsearch component to start capturing it.

    execfile('metricsEventCapture.py')
switchMetricsToElasticSearch()
  5. Migrate any content that was updated by users while this task was in progress.

    If Connections users updated content during the data migration and validation periods, that data was stored on the relational database, so repeat step 2 to migrate it. You can repeat step 2 as many times as necessary.

What to do next

When you are ready to back up your new metrics data, see Backing up and restoring data for Elasticsearch-based components.