CI/CD strategy to deploy customizations

Stages of the CI/CD pipeline

While you can construct a CI/CD pipeline in many ways, a useful approach for HCL Commerce Search customizations involves the following steps.

Codify a standard data package structure for your customizations

To automate the delivery of your customizations, ensure that you consistently store changes to:

• Custom configurations,
• Custom logic code, and
• Search release code and patches.

When you push changes to the CI/CD pipeline, each of its stages must be able to locate all the resources needed to process the data and pass it on to the next step. Therefore we begin by creating a consistent package structure for customized code. For more information, see Example .sra file structure for CI/CD automation.

Customize your code

Customize the Ingest application image

You are likely to change or touch the following resources when you customize Ingest. All new and updated files from these locations need to be available when you rebuild the Ingest Docker container. For details, see Customizing Search for CI/CD automation.

• The custom connector descriptor, found at /profile/apps/search-ingest.ear/search-ingest.war/WEB-INF/classes/deployments/commerce, and the connector descriptor properties (used to override default connector pipe properties), found at/profile/apps/search-ingest.ear/search-ingest.war/WEB-INF/classes/deployments/customization. For a discussion of how to organize this task for CI/CD, see Example: deploying a connector for CI/CD. You can use the same general process for each of the following components.
• Flow customizations, in the Registry. These extensions are updated every five minutes in the /opt/nifi-registry/flows/extensions directory.
  Note: Ingest synchronizes NiFi canvas at each startup by importing (new) custom connectors and applying custom properties

Customize the Query application image

You are likely to change or touch the following resources when you customize the Query service. All new and updated files from these locations need to be available when you rebuild the Query Docker container.

• Ingest configurations (for global attributes), which can be found in the Query container at /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/ingest-configs.json.
• The Ingest profile for Query. This can be found in the Query container at /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/profiles/ingest.
• REST handlers, found in /profile/apps/search-query.ear/search-query.war/WEB-INF/classes.
• Component configurations, found in the /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/wc-component.json.
• Custom search profiles, which be under /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/profiles/in the productview, categoryview or sitecontent subdirectories.
• Search profile (Java) custom extension logic, located under /profile/apps/search-query.ear/search-query.war/WEB-INF/classes.
• Custom NLP profile, by default the file HCL_NLPProfile.json. This file is located in /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/profiles/nlp.
• Matchmaker files, found in the /profile/apps/search-query.ear/search-query.war/WEB-INF/classes/colors(_locale).properties, ...units(_locale).properties, and ...filter(_locale).properties directories.

Customize NiFi processors and NiFi flows

Construct any new NiFi processors and NiFi flows using the guidelines in Creating a custom NiFi process group. Resources include:

• The Ingest profile (Java) extension for NiFi. This can be found in the NiFi container at /opt/nifi/nifi-current/extensions as a NAR file.

Apply any service fixes to your custom images

Ensure that you have applied the most recent service fixes. This includes deploying hot fixes and image upgrades. Autopatching hot fixes involves picking up the following kinds of files from the /patches directory:

• Ingest: binary code, connector (Json) descriptor.
• Registry: flow (dynamic loading via file or Registry UI).
• Query: binary code, profile (Json) descriptor.

Upgrading images: When NiFi is configured to use Persistent Volume storage, the following sequence of deployment is required in order to allow the NiFi canvas update to be performed automatically.

1. NiFi + Registry + Query
2. NiFi + Ingest
   Note: When you are using a Persistent Volume, you must restart the pod one more time after upgrading it to a new image in order to complete NiFi canvas auto-upgrade.

For an up-to-date list of hot fixes, see Fixes that are included in HCL Commerce releases.

Deploy custom images to your test environments

Ensure that you test all of your changes before deploying them. See Testing and debugging in the Customizing Ingest profiles topic, and Logging and troubleshooting the Ingest and Query services for more information.

Push your changes to a source code repository

Push code to a source code repository. For example, Git. Refer to the Example .sra file structure for CI/CD automation

Use an automated scripting tool to pull images from the repository, build new Docker images and activate them

Set up CI/CD pipeline using the CI/CD tool, Jenkins. Create and implement Jenkins scripts so that the CI/CD pipeline pulls the customization package from the repository, creates new Docker images and then pushes the images to a Docker repository. For more information, see Automating the CI/CD pipeline.

Note: You can use any other CI/CD tool to set up and implement the CI/CD pipeline. Ensure that you research before deciding which CI/CD tool you want to implement to facilitate your CI/CD pipeline. Think about which CI/CD tool works best for your business.