HCL Commerce Version 9.1.10.0 or later

Release changes to NiFi

Changes to NiFi connectors have been made in recent releases. These are described in descending order from the most recent.

9.1.15 release changes

ElasticSearch Schema
  • Addition of fields in log_dynamic_mappings.json:

    Added fields history and language in log_dynamic_mappings.json to include change history event IDs in the run response.

  • New filters and custom analyzer in cas_product_elastic_schema.json:

    Introduced the pattern_replace_char_filter and whitespaces_replace_char_filter filters.

    Implemented a custom analyzer named es_pattern_replace for the sku and mpn fields in cas_product_elastic_schema.json.

NiFi Pipeline
  • Added WaitLink - Product Stage 1a[WaitLink], WaitLink - Enrich NLP[WaitLink], and moved Enrich NLP[NLPStage1] after WaitLink - Product Stage 1g[WaitLink] in the dataload.product.cas connector.
  • Added WaitLink - Category Stage 1a[WaitLink], moved Clear TI Delta Table[DatabaseClearTIDeltaTable] before WaitLink - Enrich NLP[WaitLink] in auth.reindex.cas, live.reindex.cas connectors.
  • Added WaitLink - Flow Control[WaitLink] and ContinueLink - Flow Control[ContinueLink] in auth.validate, live.validate connectors.
  • Added SplitLink[SplitLink] and WaitLink - SplitLink[WaitLink] in auth.reindex.cas, live.reindex.cas connectors.
  • Moved WaitLink - Product Stage 1a[WaitLink] before AliasLink - Catalog[AliasLink] in auth.reindex.cas, live.reindex.cas connectors.
  • Renamed CopyLink - Inventory (Copy To Product)[CopyLink] to CopyLink - Inventory (Copy To Product)[InventoryCopyLink] in auth.inventory, auth.reindex, live.reindex, push.to.live.cas, push.to.live connectors.
  • Added CacheLink[CacheLink] to refresh controller service caches before continuing in auth.reindex, live.reindex connectors.
  • Added product and category cache invalidation to dataload connectors. Added dataload support to InvalidateProduct and InvalidateCategory.
SQL file location change
In previous versions, custom SQL was kept in the same template schema in the NiFi registry as other customization code. To make it easier for you to customize the SQL by itself, this SQL has been moved out to separate files, such as DatabaseProductStage.sql, in the src/main/resources/sql directory. The previous ComposeDatabaseSQL processor has been replaced with a new processor, GenerateSQL, that is used to retrieve these files. A shell script is used to update the SQL in the comment section of every processor, making it easier for you to access it from anywhere.

9.1.14.1 release changes

ElasticSearch Schema
No Change.
NiFi Pipeline
Added WaitLink - Product Stage 1a[WaitLink], WaitLink - Enrich NLP[WaitLink], and moved Enrich NLP[NLPStage1] after WaitLink - Product Stage 1g[WaitLink] in the dataload.product.cas connector.

9.1.14 release changes

ElasticSearch Schema
  • Tokenizer switch in custom_splitter analyzer:

    Implemented a switch in the tokenizer used by the custom_splitter analyzer from keyword to whitespace in cas_product_elastic_schema.json.

  • New Split field addition in search index mapping:

    Introduced a new split field in the search index mapping of cas_product_elastic_schema.json with specific settings tailored for text analysis.

NiFi Pipeline
  • Added WaitLink - Enrich NLP[WaitLink] before Clear TI Delta Table[DatabaseClearTIDeltaTable] and moved Enrich NLP[NLPStage1] up before WaitLink - Enrich NLP[WaitLink] in dataload.product connector.
  • Added WaitLink - Product Stage 1a[WaitLink] and Clear TI Delta Table[DatabaseClearTIDeltaTable] in auth.inventory.cas connector.
  • Added Clear TI Delta Table[DatabaseClearTIDeltaTable] in auth.reindex.cas connector.
  • Added MoveLink - Store[MoveLink] and AliasLink - Store[AliasLink] in push-to-live and push-to-live.cas connectors.
  • Moved ContinueLink - Flow Control[ContinueLink] up before Inventory Stage 1a (Parent Inventories)[DatabaseInventoryStage1a].
  • Moved auth.postindex/live.postindex connector stages to the main auth.reindex/live.reindex connector, leaving auth.postindex/live.postindex connector now used for customization only.
  • Added new ContinueLink - Flow Control[ContinueLink] in auth.attribute and live.attribute connectors.

9.1.13.3 release changes

ElasticSearch Schema
  • Catalog filter enhancement:

    Added support for base contract inheritance for catalog filters by introducing a new child field in store_dynamic_mappings.json.

  • Default stemmer change to Snowball:

    Changed the default stemmer in cas_product_elastic_schema.json and product_elastic_schema.json from the existing one to the Snowball stemmer.

NiFi pipeline
  • Added WaitLink - Product Stage 1a[WaitLink] and Clear TI Delta Table[DatabaseClearTIDeltaTable] at end of auth.inventory.cas connector.
  • Added Clear TI Delta Table[DatabaseClearTIDeltaTable] at the end of the auth.reindex.cas connector.

9.1.13.2 release changes

ElasticSearch Schema
No changes.
NiFi pipeline
  • Added WaitLink - Category Stage 1c[WaitLink] and WaitLink - Product Stage 1g[WaitLink] to the dataload.category.cas connector. Relocated Clear TI Delta Table[DatabaseClearTIDeltaTable] to the end of the pipeline
  • Added Clear TI Delta Table[DatabaseClearTIDeltaTable] after ingest completion in auth.inventory, auth.reindex, dataload.category.cas, dataload.category, dataload.product.cas, dataload.product, live.inventory, live.reindex connectors.
  • Changed NRTLink[CASNRTLink] to DataloadLink[CASDataloadLink] in auth.inventory.cas, live.inventory.cas connectors.

9.1.13.1 release changes

ElasticSearch Schema
  • New field addition:

    Added a new field named key inside attribute.value in both cas_product_elastic_schema.json and product_elastic_schema.json.

9.1.13.0 release changes

ElasticSearch Schema
  • Natural keyword field mapping:

    Modified the cas_product_elastic_schema.json and product_elastic_schema.json to include natural keyword field mapping.

  • Asset store schema addition:

    Introduced the asset_store to the store_dynamic_mappings.json.

  • Text property details addition:

    Added text property details for default.correction in product_elastic_schema.json.

  • Parent field inclusion:

    Under the name and description fields in product_elastic_schema.json, a new parent field has been added.

NiFi pipeline
  • Removed Product Stage 1c (Find SEO URLs)[DatabaseCASProductStage1c] from live.reindex.cas, live.product.cas, dataload.product.cas, auth.reindex.cas, and auth.product.cas connectors.
  • Combined CAS post index into the reindex process in auth.reindex.cas and live.reindex.cas connectors.
  • Removed Category Stage 1e (Find SEO URL)[DatabaseCASCategoryStage1e] from auth.category.cas, auth.reindex.cas, dataload.category.cas, live.category.cas, and live.reindex.cas connectors.
  • Added WaitLink - Product Stage 1g[WaitLink] in the auth.reindex.cas and live.reindex.cas connectors.
  • Moved Store[DatabaseCASStoreStage1] stage before the Product Schema[CASProductSchema] stage in the auth.reindex.cas and live.reindex.cas connectors.
  • Added copy.condition.expression to all Auth and Live CopyLink for Price and Inventory.
  • Removed auth.postindex.cas and live.postindex.cas connectors.
  • Added Remove Workspace Information - Attribute[RemoveWorkspaceDocument], Remove Workspace Information - Category[RemoveWorkspaceDocument], Remove Workspace Information - Product[RemoveWorkspaceDocument], Remove Workspace Information - URL[RemoveWorkspaceDocument] and Remove Workspace Information - Description[RemoveWorkspaceDocument] stages in the push-to-live and push-to-live.cas connectors.
  • Added auth.alias and live.alias connectors to support switching catalog index aliases using API.
  • Changed SplitLink - Inventory[SplitLink] to CopyLink - Inventory (Copy To Product)[CopyLink] in the push-to-live connector.
  • Added update-live and update-live.cas connectors.
  • Removed Category Stage 1d (Build Hierarchy)[DatabaseCASCategoryStage1d] from the auth.product.cas and live.product.cas connectors.
  • Removed CopyLink - Price (Copy To Product)[CopyLink], WaitLink - Price CopyLink[WaitLink], CopyLink - Inventory (Copy To Product)[CopyLink], and WaitLink - Inventory CopyLink[WaitLink] from the update-live.cas and update-live connectors.
  • Added WaitLink - Inventory CopyLink[WaitLink] in auth.inventory and live.inventory connectors.
  • Removed LockLink - Store[LockLink], LockLink - Catalog[LockLink], LockLink - Attribute[LockLink], LockLink - Category[LockLink], LockLink - Product[LockLink], LockLink - Page[LockLink], LockLink - Description[LockLink], (Un)LockLink - Store[LockLink], (Un)LockLink - Catalog[LockLink], (Un)LockLink - Attribute[LockLink], (un)LockLink - Category[LockLink], (Un)LockLink - Product[LockLink], (Un)LockLink - Page[LockLink], and (Un)LockLink - Description[LockLink] from the push-to-live and push-to-live.cas connectors.
  • Added Clear TI Delta Table[DatabaseClearTIDeltaTable] after ingest completion in auth.inventoryauth.reindex, dataload.category.cas, dataload.category, dataload.product.cas, dataload.product, live.inventory, and live.reindex connectors.
  • Changed NRTLink[CASNRTLink] to DataloadLink[CASDataloadLink] in the auth.inventory.cas and live.inventory.cas connectors.

9.1.12 release changes

ElasticSearch Schema
The following changes apply to category, product and attribute schemas.
New analyzers
These analyzers use different tokenizers and filters, which are applied to different index fields to perform text analysis processing on terms during ingestion and at index time, and during query processing and search runtime.
custom_splitter
Uses keyword tokenizer.
custom_shingle
Uses standard tokenizer.
custom_analyzer
Uses standard tokenizer.
Refer to the Elasticsearch documentation for additional details concerning text analysis.
New filters
The following Elasticsearch filters can now be used inside custom analyzers to convert (normalize, pre-process, etc.) the index field values for use in the Elasticsearch index
  • html_strip_filter

    Strips html elements from text and replaces html entities with their decoded values.

  • asciifolding

    Converts alphabetic, numeric, and symbolic characters that are not in the Basic Latin Unicode block.

  • word_delimiter_filter

    Splits tokens at non-alphanumeric characters.

  • shingle_filter

    Adds shingles, or word n-grams, to a token stream by concatenating adjacent tokens.

  • trim_filter

    Removes leading and trailing whitespace from each token in a stream.

Normalizer property changed
The normalizer property has changed from lowercase to normalized, because it not only changes the word to lowercase but also performs ASCII folding. This normalizer is used in all normalized index fields.

9.1.9.0 release changes

ElasticSearch Schema
No changes.
NiFi pipeline
  • URLStage2a and URLStage2b have been removed along with their Waitlinks.

    URL Stage 1a has been updated to write to both URL Index and Category Index (add SEO) at the same time. This avoids issues where URL Stage 2a queries Elasticsearch prior to completion of the URL document.

    Since URL Stage 1a (copy to category) already adds SEO to the Category Index, URL Stage 2a is no longer needed and has been removed. The same applies to URL Stage 1b (copy to product) and it has been removed.

  • Removed CategoryStage4 (Category Deletion) in NRT and Live pipelines.

    CategoryStage4 (Category Deletion) has been removed from before Category Stage 1a (Main Document). This stage was previously used to handle edge cases for deletion. Since the catalog hierarchy now builds directly from the database, this stage is no longer needed and can be removed. This change improves overall indexing time.

9.1.8.0 release changes

ElasticSearch Schema
No changes.
NiFi pipeline
  • Removed ProductStage 1f.

    The rolling up (1e) and pushing down (1f) stages of the product attributes connector have been combined into one more performant stage, Product Stage 1e.

  • CategoryStage2 has been changed to Category Stage 1d and redesigned.

    Previously, CategoryStage2 built the category hierarchy by querying Elasticsearch. In Version 9.1.8, the category hierarchy is built by reading from the database for improved performance and reliability.

9.1.7.0 release changes

ElasticSearch Schema
No changes.
NiFi pipeline
  • The Find Child Items process group has been removed from Product Stage 1b.

    The Find Child Items process group is removed from Product Stage 1b. The logic of finding child items is moved to a new stage called Product Stage 1h.

  • Product Stage 1f (Rollup attributes) and its WaitLink and RefreshLink have been removed.

    The rollup logic has been combined into Product Stage 1e for better performance.

  • Product Stage 2 (pushdown) and its WaitLink and RefreshLink have been removed.

    The pushdown attribute logic has now been combined into Product Stage 1e for better performance. Other pushdown logic areas have been combined with other stages.

  • Product Stage 1c (override child attributes) and its WaitLink and RefreshLink have been removed.

    The override child attribute logic is no longer needed with the new logic of Product Stage 1e

  • Product Stage 1d (find item attachment) and its WaitLink and RefreshLink have been removed.

    The find item attachment logic is no longer needed with the new logic in Product Stage 1b.

  • NRT/Dataload ProductStage4, DatabaseProductStage1e-2 and DatabaseProductStage1f-2 have been removed.

    The logic of these stages are no longer needed with the new optimized logic. ProductStage4 and ProductStage1e-2 have also been removed from the data load pipeline.