Creating a NiFi service connector

The pre-built pipelines provided in Version 9.1 are Store, Catalog, Attribute, Category, Product, SEO, Price, Inventory, Entitlement, and Merchandising.

Store

Name, description, store level defaults, supported languages and currencies

Catalog

Name, description, catalog filters

Attribute

Name, description, attribute values, and facet properties

Category

Name, short description, sales catalog hierarchy and navigation properties, facets, seo url

Product

Name, short description, brand, price lists, inventory counts, natural language, sales catalog hierarchy and navigation properties, seo url, spell correction, suggestion, entitlement, merchandising associations, attachments

URL

Search engine optimization properties and URL resolution for products and categories

Description

Long description for product and category.

You can set connectors to run once, or on a recurring schedule.

Name

Each descriptor (and by extension a connector) must have a unique name. If a connector with the given name already exists, a new connector will not be created and a 400: Bad Request will be returned.

Description

A description of what this connector does. It is recommended to give a connector a description so that it is easy to recall the purpose of a connector.

Pipes

The list of pipes that make up a connector. Each pipe in a connector is responsible for doing some sort of ETL operation(s). Each pipe must have a name, which corresponds to a pipe that exists in the NiFi Registry. For more information, see the Apache NiFi Registry documentation.

You can define the properties that you want set in a pipe; PROCESS_GROUP and CONTROLLER_SERVICE level properties are supported. To define a property, the name and value of the property should be specified alongside their scope (The immediate parent Process Group or Controller Service they should be defined inside).

When used with PROCESS_GROUP scope, the property defined inside of this section will be used as a variable to be set into the given process group. For example:

                                 {
					"name": "connector.wait.limit",
					"value": "3",
					"scope": {
						"name": "Wait for Completion",
						"type": "PROCESS_GROUP"
					}
				}

In this example, you are defining a variable with the name connector.wait.limit created inside of the process group with name Wait for Completion. This variable will have a value of 3 assigned to it. Note that variables in NiFi can be inherited to child processors and child process groups. Setting the variable at the pipe level should be sufficient for all of its sub-components to access it.

When used with CONTROLLER_SERVICE scope, the property defined inside of this section will be treated as one of the existing properties belonging to the specified controller service. For example,

                              {
					"name": "Database Connection URL",
					"value": "${AUTH_JDBC_URL}",
					"scope": {
						"name": "Database Connection Pool",
						"type": "CONTROLLER_SERVICE"
					}
				},

In this example, an existing property called "Database Connection URL" of the controller service with name "Database Connection Pool" will be updated with the given value ${AUTH_JDBC_URL}. This value can be an absolute value or an environment variable name such as the one illustrated in this example. Using an environment variable allows this value to be configured at deployment time when setting up the NiFi application container.

HCL Commerce Version 9.1 provides default support for a core set of Commerce-specific document types. These data specifications can be used for both full and delta updates. For an example of a Commerce data specification, see Product data specification.