Deploying Ingest Profile custom extensions on production runtime environment

This topic explains how to deploy a custom search NIFI image on production.

Procedure

Deploy Ingest Profile custom extensions on production runtime environment.

Create a custom search-nifi-app Docker image.
For runtime deployment of Ingest customization extensions, create a custom Docker image for search-nifi-app to include the customizations created using the NiFi toolkit.
1. Create a Dockerfile Docker configuration file with the following content.
```
FROM search-nifi-app docker image
COPY CusDeploy/ /opt/nifi/nifi-current/lib
```
  Where CusDeploy/ contains the commerce-custom-search-marketplace-seller-9.1.10.0.jar that you built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
  If patch needs to be applied:
```
FROM comlnx94.prod.hclpnp.com/9.1.10.0/search-nifi-app:v9-20220322-1524
COPY patches / /opt/nifi/nifi-current/patches
WORKDIR /opt/nifi/scripts
RUN ./patch.sh
COPY CusDeploy/ /opt/nifi/nifi-current/lib
```
  Where patches/ directory contains the patch (jar) to be applied in V9.1.10.0 and CusDeploy/ to contain commerce-custom-search-marketplace-seller-9.1.10.0.jar built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
```
FROM comlnx94.prod.hclpnp.com/9.1.10.0/search-nifi-app:v9-20220322-1524
COPY CusDeploy/ /opt/nifi/nifi-current/lib
```
  Where CusDeploy/ to contain commerce-custom-search-marketplace-seller-9.1.10.0.jar built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
2. Prepare the custom deployment files.
  - Create the CusDeploy subdirectory in the same directory as the Dockerfile created in Create a custom NiFi processor by extending the default Ingest processor in the NiFi Toolkit.
  - Copy the commerce-custom-search-marketplace-seller-9.1.x.x.jar into the CusDeploy directory.
  - Copy the ingest-api.jar obtained from the libs directory under the extracted commerce-custom-search-marketplace-seller.zip project template into the CusDeploy directory.
    Note: From V9.1.12.0 onwards, ingest-api.jar is not made available inside tutorial assets.
3. Build the custom NiFi server (search-nifi-app) Docker image.
  - Open a command prompt and navigate to the directory containing your Dockerfile, and its CusDeploy subdirectory.
  - Run the Docker build command to build a custom Docker image for the NiFi server (search-nifi-app)
```
docker build -t search-nifi-app:tag
```
    For example:
```
docker build -t search-nifi-app:nifi-extension .
```
    Note:
    
    Ensure that the period is present at the end of your Docker build command.
    
    The tag, like the deployment package build label, is used to identify the custom Docker image. You can modify this at any time to help identify the generated Docker image.
Deploy the runtime Elasticsearch data Server through CI/CD using custom Docker image created in Create a custom NiFi processor by extending the default Ingest processor in the NiFi Toolkit.
This task covers the minimum requirements for a deployment that is based on Docker Compose.
- Set up an HCL Commerce production environment using the Docker Compose deployment method. For information on setting up this environment, see Deploying HCL Commerce Version 9.1 with Docker Compose (for non-production usage).
- (Optional) For your custom NiFi server Docker image (search-nifi-app), ensure that you have the following deployment parameters correctly defined under the environment section of your docker-compose configuration file.
If your customization depends on the commerce-custom-search-processors-nar project, transfer the NAR file from the commerce-custom-search-processors-nar project’s "target" directory to the designated extension directory on Elasticsearch server and mount this file as a volume in the NiFi Docker container. For detailed instructions, refer to topic Building and deploying a custom NAR file.
```
volumes:
        - ./volumes/hcl-cache-local-redis/cache_cfg-ext.yaml:/SETUP/hcl-cache/cache_cfg-ext.yaml
        - ./volumes/hcl-cache-local-redis/redis_cfg.yaml:/SETUP/hcl-cache/redis_cfg.yaml
        - ./volumes/nifi-ext/commerce-custom-search-processors-nar-9.1.10.0.nar:/opt/nifi/nifi-current/extensions/commerce-custom-search-processors-nar-9.1.10.0.nar
```
following steps are to be performed on the runtime environment the same way that they were performed on the Development environment:
- Create an Ingest profile and bind it to a default connector (Creating a custom Ingest profile and binding to the default Connector Pipe).
- Override the default search query profile to include the additional seller_description.raw attribute in the query response. (Overriding Query search profiles to modify the response fields).
- Validate that the search profile response contains additional the attribute seller_description.raw (Testing the Ingest profile extensions).

This topic covered how to create a custom search NiFi image, including the ingest logic, and deploy it to production. The next step is to create a custom ingest profile and bind it to a default connector.