HCL Commerce Version 9.1.7.0 or later

Persisting Ingest changes (NiFi Registry)

This topic describes how to make changes to a specific pipe by changing its property/configuration across all connectors. For example, changing SQL statement for a processor, Execute SQL, in a pipe, DatabaseCategoryStage1b, across all connector in NiFi.

About this task

Complete the following steps to do Ingest changes across all connectors in NiFi.

Procedure

  1. Find the processor that you want to change.
    Note: You can make only property/configuration changes and no logic change to the processor.
  2. Find all the pipes in NiFi containing the processor that you want to change and then make a list of pipe.name.
  3. Run the following command to navigate to NiFi Registry container/pod.
    docker exec -it {registryContainer} bash
  4. Run the following command to extract the pipe description JSON for the list of pipe.name created in Step 2.
    sh /opt/nifi-registry/scripts/export_flow.sh {pipe.name} > /opt/nifi-registry/{pipe.name}.json
  5. Open {pipe.name}.json and then search for the desired processor for editing and changing its property/configuration value.
  6. Run the following command to import the updated pipe description JSON for the list of pipe.name.
    sh /opt/nifi-registry/scripts/import_flow.sh {pipe.name} /opt/nifi-registry/{pipe.name}.json
  7. Run the following command to exit out of the container and backup the updated pipe description JSON.
    docker cp {registryContainer}:/opt/nifi-registry/{pipe.name}.json {pipe.name}.json
  8. Using the following URL, check NiFi Registry to verify that these pipe description JSON (flow) have a new version defined.
  9. Use any one of the following two ways (a or b) to update the pipes in NiFi with a new version:
    1. Using Upgrade API:
      1. Using the following URL, run Upgrade API from Ingest service to get the latest version of the pipe in a particular connector

        http://INGEST_HOSTNAME_PORT/connectors/{id}/upgrade

      Note:
      • Use an empty body.
      • Run Upgrade API multiple times if the upgraded pipe is in more than one connector.
      • Does not work with pipes in framework connectors, Bulk Services, Logging Service, etc.
    2. Using NiFi Console:
      1. In the NiFi console, navigate to the upgraded pipe which is displayed with a red arrow icon.
      2. Right-click the pipe and then navigate to Version > Change Version to choose the desired version from the list of versions.
      3. Click Change.