Ingest Store index pipeline

Implementing a Store index field pipeline

The following steps illustrate how the Store indexing pipeline is implemented in Apache NiFi. The flow consists of mainly eight stages, each of which is illustrated using examples from the NiFi user interface. For more information on the interface, see the Apache NiFi User Guide.

Extensive example samples are available for each stage. These are provided in the bottom sections of this topic. The examples include SQL code, schema table information, and sample inputs and outputs.

Stage 1: Creating a Store document

This stage describes how to transform the Store data into the Store index. For example code, see Stage 1 samples.

Transforming the Store data starts with running the following SQL to retrieve Store data from the Commerce database.

         SELECT F.STORE_ID, F.STORETYPE, F.LANGUAGE_ID AS DEF_LANGUAGE_ID, F.FFMCENTER_ID, F.CONTRACT_ID, G.DIRECTORY,
		E.SETCCURR, E.IDENTIFIER AS STORE_IDENTIFIER, E.MEMBER_ID, N.LANGUAGE_ID, D.DISPLAYNAME, D.DESCRIPTION,
		L.LOCALENAME, F.INVENTORYSYSTEM, F.CRTDBYCNTR_ID
	   FROM STOREENT E, STOREENTDS D, STORELANG N, LANGUAGE L,
	        (SELECT F.CONTRACT_ID, S.STORE_ID, S.LANGUAGE_ID, S.STORETYPE, S.FFMCENTER_ID, S.INVENTORYSYSTEM, S.CRTDBYCNTR_ID, S.DIRECTORY 
	           FROM STORE S
	                LEFT OUTER JOIN STOREDEF F ON (S.STORE_ID = F.STORE_ID)
	          WHERE S.STORE_ID IN (SELECT R.RELATEDSTORE_ID FROM STOREREL R WHERE R.STATE = 1 AND R.STRELTYP_ID = -4 AND R.STORE_ID = ${param.storeId})
	            AND S.STATUS = 1) F,
	        (SELECT S.DIRECTORY 
	           FROM STORE S, STOREREL R
	          WHERE S.STORE_ID = R.RELATEDSTORE_ID AND R.STATE = 1 AND R.STRELTYP_ID = -4 AND R.RELATEDSTORE_ID <> R.STORE_ID
	            AND R.STORE_ID = ${param.storeId}) G
	  WHERE F.STORE_ID = E.STOREENT_ID AND E.MARKFORDELETE = 0 AND E.TYPE = 'S' AND N.STOREENT_ID = E.STOREENT_ID
		AND D.STOREENT_ID = E.STOREENT_ID AND D.LANGUAGE_ID = F.LANGUAGE_ID AND N.LANGUAGE_ID = L.LANGUAGE_ID
	  ORDER BY N.LANGUAGE_ID

Next, the result set is passed to the CreateStoreDocumentFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index.


Index Field Name	Index Field Type	Description
Document Identifier
id/store	id_string	Internal id of the owning (open and operational) store; mapped to STORE.STORE_ID
id/member	id_string	The internal reference number that identifies the owner of the store; mapped to STOREENT.MEMBER_ID
id/language	id_string	The internal reference number of the current language of this document; mapped to STORELANG.LANGUAGE_ID
id/contract	id_string	The internal reference number of the hosting contract between hosting service provider and the store operator; mapped to STORE.CRTDBYCNTR_ID
id/default/language	id_string	The default language for information displayed to customers shopping in the store; mapped to STORE.LANGUAGE_ID
id/default/fulfillment	id_string	The default fulfillment center for the current store; mapped to STORE.FFMCENTER_ID
id/default/contract	id_string	The default contract of the current store; mapped to STOREDEF.CONTRACT_ID
id/supported/language	id_string	The list of supported languages of the current store; mapped to STOREENTDSC.LANGUAGE_ID
identifier/language	id_string	The current language of this document
identifier/specification	id_string	Set to "store"
identifier/store	id_string	A string that uniquely identifies the owning store; mapped to STOREENT.IDENTIFIER
identifier/default/language	id_string	The language string of this supported language; mapped to STOREENTDSC.LANGUAGE_ID
identifier/default/currency	id_string	The default currency for the current store that will be used by a customer who has not specified a preferrred currency; mapped to STOREENT.SETCCURR
identifier/supported/language	id_string	The list of supported language strings; mapped to STOREENTDSC.LANGUAGE_ID and LANGUAGE.LANGUAGE
Language Sensitive Data
name/raw	raw	The language-dependent name of this catalog group; mapped to STOREENTDSC.NAME
name/normalized	normalized	Same as above
description/raw	raw	A short description of this catalog group; mapped to STOREENTDSC.SHORTDESCRIPTION
Properties
type	raw	The type of the current store
properties/name	raw	The predefined store property name
properties/value	raw	The store specific property value
Navigational Data
relationship/parent	id_string	The reference to its catalog asset store if used; mapped to STOREREL.RELATEDSTORE_ID
relationship/directory	raw	The base name of the directory in which store-specific Web assets, such as JSP files and property files, are found; mapped to STORE.DIRECTORY

Stage 2: Associating Store locations

This stage describes how to transform Store location related data and load it into the Store index. For example code, see Stage 2 samples.

Transforming and loading Store location related data starts with running the following SQL to retrieve location data from the Commerce database:

 SELECT D.NAME, D.DESCRIPTION, D.STLOC_ID, D.LANGUAGE_ID, S.IDENTIFIER, S.STOREENT_ID,
			  S.PHONE, S.ADDRESS1, S.ADDRESS2, S.ADDRESS3, S.ACTIVE, S.CITY, S.STATE, S.COUNTRY, S.ZIPCODE,
			  A.NAME ATTR_NAME, A.DISPLAYNAME ATTR_DISPLAYNAME, A.VALUE ATTR_VALUE, A.DISPLAYVALUE ATTR_DISPLAYVALUE,
			  B.NAME FFMC_NAME, B.FFMCENTER_ID
	   FROM STLOC S, STLOCDS D
			  LEFT OUTER JOIN (
				SELECT LISTAGG(NAME, '###') WITHIN GROUP (ORDER BY SEQUENCE) NAME,
					   LISTAGG(COALESCE(DISPLAYNAME, ' '), '###') DISPLAYNAME,
					   LISTAGG(VALUE, '###') VALUE,
					   LISTAGG(COALESCE(DISPLAYVALUE, ' '), '###') DISPLAYVALUE,
					   LANGUAGE_ID, STLOC_ID
				  FROM STLOCATTR
				 GROUP BY STLOC_ID, LANGUAGE_ID) A ON (A.LANGUAGE_ID IN (${ingest.languages}) AND A.STLOC_ID = D.STLOC_ID)
			  LEFT OUTER JOIN (
				SELECT LISTAGG(COALESCE(F.NAME, ' '), '###') NAME,
					   LISTAGG(R.FFMCENTER_ID, '###') FFMCENTER_ID,
					   STLOC_ID
				  FROM STLFFMREL R, FFMCENTER F
				 WHERE R.FFMCENTER_ID = F.FFMCENTER_ID
				 GROUP BY STLOC_ID) B ON (B.STLOC_ID = D.STLOC_ID)
	  WHERE S.STLOC_ID = D.STLOC_ID AND D.LANGUAGE_ID IN (${ingest.languages}) AND S.STOREENT_ID = ${param.storeId}

Next, the result set is passed to the FindStoreLocationsFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


Physical Store Location
locations/<id>/id	id_string	The internal store location id
locations/<id>/identifier	id_string	The external identifier for this store location
locations/<id>/name	raw	The name of this store location
locations/<id>/description	raw	The description of this store location
locations/<id>/phone	raw	The phone number of this store location
locations/<id>/address	raw	The list of addresses of this store location
locations/<id>/attributes/name	raw	A list of attribute names associated with this store location
locations/<id>/attributes/value	raw	The corresponding attribute value of this store attribute
locations/<id>/filfillment	id_string	A list of fulfillment center ids for this store location

Stage 3: Associating Store names

This stage describes how to transform Store name related data and load it into the Store index. For example code, see Stage 3 samples.

Transforming and loading Store name related data starts with running the following SQL to retrieve location data from the Commerce database:

SELECT SL.STOREENT_ID, SL.LANGUAGE_ID, DA.DESCRIPTION A_DESCRIPTION, DE.DESCRIPTION E_DESCRIPTION,
		       DA.DISPLAYNAME A_DISPLAYNAME, Start of deleted contentDE.DISPLAYNAME End of deleted contentStart of added contentDE.DISPLAYNAME End of added contentE_DISPLAYNAME
		  FROM STORELANG SL
		       LEFT JOIN 
		       (SELECT S.STORE_ID, R.RELATEDSTORE_ID, D.LANGUAGE_ID, D.DISPLAYNAME, D.DESCRIPTION
				  FROM STORE S, STOREENT E, STOREREL R, STOREENTDS D, STORELANG L
				 WHERE S.STATUS = 1 AND S.STORE_ID = E.STOREENT_ID AND E.MARKFORDELETE = 0 AND E.TYPE = 'S'
				   AND S.STORETYPE NOT IN ('BMP', 'MPS', 'HCP', 'PBS', 'SCP', 'CHS', 'RPS', 'BRP', 'DPS', 'DPX')
				   AND R.STATE = 1 AND R.STRELTYP_ID = -4 AND R.STORE_ID = S.STORE_ID AND R.RELATEDSTORE_ID = D.STOREENT_ID
				   AND L.STOREENT_ID = S.STORE_ID AND L.LANGUAGE_ID = D.LANGUAGE_ID AND R.STORE_ID <> R.RELATEDSTORE_ID) DA
				    ON DA.STORE_ID = SL.STOREENT_ID AND SL.LANGUAGE_ID = DA.LANGUAGE_ID
		       LEFT JOIN 
		       (SELECT S.STORE_ID, R.RELATEDSTORE_ID, D.LANGUAGE_ID, D.DISPLAYNAME, D.DESCRIPTION
				  FROM STORE S, STOREENT E, STOREREL R, STOREENTDS D, STORELANG L
				 WHERE S.STATUS = 1 AND S.STORE_ID = E.STOREENT_ID AND E.MARKFORDELETE = 0 AND E.TYPE = 'S'
				   AND S.STORETYPE NOT IN ('BMP', 'MPS', 'HCP', 'PBS', 'SCP', 'CHS', 'RPS', 'BRP', 'DPS', 'DPX')
				   AND R.STATE = 1 AND R.STRELTYP_ID = -4 AND R.STORE_ID = S.STORE_ID AND R.RELATEDSTORE_ID = D.STOREENT_ID
				   AND L.STOREENT_ID = S.STORE_ID AND L.LANGUAGE_ID = D.LANGUAGE_ID AND R.STORE_ID = R.RELATEDSTORE_ID) DE
				    ON DE.STORE_ID = SL.STOREENT_ID AND SL.LANGUAGE_ID = DE.LANGUAGE_ID
		 WHERE SL.STOREENT_ID = ${param.storeId} AND SL.LANGUAGE_ID IN (${ingest.languages})

Next, the result set is passed to the FindStoreNamesFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


Language Sensitive Data
name/raw	raw	The language-dependent name of this catalog group; mapped to STOREENTDSC.NAME
name/normalized	normalized	Same as above
name/text	text	Same as above
description/raw	raw	A short description of this catalog group; mapped to STOREENTDSC.SHORTDESCRIPTION
description/normalized	normalized	Same as above
description/text	text	Same as above

Stage 4: Associating Store level configurations

This stage describes how to transform the Store level configuration data and load it into the Store index. For example code, see Stage 4 samples.

Transforming the configuration data and loading it starts with running the following SQL to retrieve location data from the Commerce database:

SELECT LISTAGG(NAME, '@@@') NAME,
           LISTAGG(STOREENT_ID, '@@@') STOREENT_ID,
           LISTAGG(VALUE, '@@@') VALUE
      FROM (SELECT F.NAME,
	           LISTAGG(F.STOREENT_ID, ', ') WITHIN GROUP (ORDER BY F.SEQUENCE) STOREENT_ID,
		   LISTAGG(COALESCE(F.VALUE, ' '), '###') VALUE
	      FROM (SELECT C.NAME, C.STOREENT_ID , C.VALUE, R.SEQUENCE
		      FROM STORECONF C, STOREREL R
		     WHERE R.RELATEDSTORE_ID = C.STOREENT_ID
		       AND R.STATE = 1 AND R.STRELTYP_ID = -11 AND R.STORE_ID = ${param.storeId}
		       AND C.NAME NOT LIKE ('wc.externalContent.%')
		     UNION
		    SELECT C.NAME, C.STOREENT_ID , C.VALUE, 9999999
		      FROM STORECONF C
		     WHERE C.STOREENT_ID = 0
		       AND C.NAME NOT LIKE ('wc.externalContent.%')) F
	     GROUP BY F.NAME)

Next, the result set is passed to the FindConfigurationsFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


Properties
configurations/name	raw	The predefined store attribute name
configurations/value	raw	The store specific attribute value

Stage 5: Associating Store level features

This stage describes how to transform the Store level feature data and load it into the Store index. For example code, see Stage 5 samples.

The process starts with running the following SQL to retrieve location data from the Commerce database:

SELECT LISTAGG(NAME, '@@@') NAME,
             LISTAGG(STOREENT_ID, '@@@') STOREENT_ID,
	     LISTAGG(VALUE, '@@@') VALUE
	FROM (SELECT F.NAME,
	             LISTAGG(F.STOREENT_ID, ', ') WITHIN GROUP (ORDER BY F.SEQUENCE) STOREENT_ID,
	             LISTAGG(COALESCE(F.CONTENT, ' '), '###') VALUE
		FROM (SELECT E.NAME, D.CONTENT, E.STOREENT_ID , R.SEQUENCE
		        FROM EMSPOT E, DMEMSPOTDEF D, STOREREL R
		       WHERE R.RELATEDSTORE_ID = E.STOREENT_ID
			 AND R.STATE = 1 AND R.STRELTYP_ID = -11 AND R.STORE_ID = ${param.storeId}
			 AND E.USAGETYPE = 'STOREFEATURE'
			 AND E.EMSPOT_ID = D.EMSPOT_ID
			 AND D.CONTENTTYPE = 'FeatureEnabled'
		       UNION
		      SELECT E.NAME, D.CONTENT, E.STOREENT_ID , 9999999
			FROM EMSPOT E, DMEMSPOTDEF D
		       WHERE E.STOREENT_ID = 0
			 AND E.USAGETYPE = 'STOREFEATURE'
			 AND E.EMSPOT_ID = D.EMSPOT_ID
			 AND D.CONTENTTYPE = 'FeatureEnabled') F
	       GROUP BY F.NAME)

Next, the result set is passed to the FindFeaturesFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


Properties
features/name	raw	The store specific feature name
features/value	raw	Enabled status of this store specific feature

Stage 6: Determining supported languages

This stage describes how to determine the Store supported languages and load them into the Store index. For example code, see Stage 6 samples.

Determining the Store supported languages and loading them starts with running the following SQL to retrieve location data from the Commerce database:

SELECT N.STOREENT_ID, LISTAGG(N.LANGUAGE_ID, ', ') WITHIN GROUP (ORDER BY N.LANGUAGE_ID) LANGUAGE_ID,
		       LISTAGG(TRIM(L.LOCALENAME), '###') LOCALENAME
		  FROM STORELANG N, LANGUAGE L
		 WHERE N.STOREENT_ID = ${param.storeId} AND N.LANGUAGE_ID = L.LANGUAGE_ID
		 GROUP BY N.STOREENT_ID

Next, the result set is passed to the FindSupportedLanguagesFromDatabase processort for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


identifier/supported/language	id_string	The list of supported language strings.
id/supported/language	string	The list of supported language Ids.

Stage 7: Determining supported catalogs

This stage describes how to determine the Store supported and default catalogs and load them into the Store index. For example code, see Stage 7 samples.

Determining the Store supported and default catalogs, and loading them starts with running the following SQL to retrieve location data from the Commerce database:

 SELECT LISTAGG(S.STOREENT_ID, ', ') STOREENT_ID,
            LISTAGG(COALESCE(CHAR(D.CATALOG_ID), ' '), '###') DEFAULT_CATALOG_ID,
            LISTAGG(S.CATALOG_ID, ', ') CATALOG_ID,
            LISTAGG(S.IDENTIFIER, '###') IDENTIFIER,
            LISTAGG(S.MASTERCATALOG, '###') MASTERCATALOG
      FROM  (SELECT S.STOREENT_ID, C.CATALOG_ID, C.IDENTIFIER, S.MASTERCATALOG 
               FROM STORECAT S, CATALOG C
	      WHERE S.STOREENT_ID IN
	            (SELECT RELATEDSTORE_ID FROM STOREREL
	              WHERE STATE = 1 AND STRELTYP_ID = -4 AND STORE_ID = ${param.storeId})
	                AND S.CATALOG_ID = C.CATALOG_ID) S
            LEFT JOIN STOREDEFCAT D ON (D.STOREENT_ID = ${param.storeId} AND S.CATALOG_ID = D.CATALOG_ID)

Next, the result set is passed to the FindSupportedCatalogsFromDatabase processor for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


identifier/supported/catalog	id_string	The list of supported sales catalogs
identifier/default/catalog	id_string	The external identifier of the default sales catalog; mapped to CATALOG.IDENTIFIER
identifier/master/catalog	id_string	The master catalog of the current asset store; determined from STORECAT.MASTERCATALOG
id/supported/catalog	id_string	The list of supported sales catalogs.
id/default/catalog	id_string	The default sales catalog of the current store; mapped to STOREDEFCAT.CATALOG_ID
id/master/catalog	id_string	The master catalog from its current asset store; determined from STORECAT.MASTERCATALOG

Stage 8: Determining supported currencies

This stage describes how to determine the Store supported currencies and load them into the Store index. For example code, see Stage 8 samples.

Determining the Store supported currencies and loading them starts with running the following SQL to retrieve location data from the Commerce database:

		 SELECT C.STOREENT_ID, LISTAGG(C.CURRSTR, '###') CURRENCY
	       FROM CURLIST C
	 	  WHERE C.STOREENT_ID = ${param.storeId}
	 	  GROUP BY C.STOREENT_ID

Next, the result set is passed to the FindSupportedCurrenciesFromDatabase script for transformation, using the following table to map the database field returned from the SQL above to an index field in the Store index:


identifier/supported/currency	id_string	The list of supported currencies for the current store

Stage 1 samples

The following code is an example of the input data for the CreateStoreDocumentFromDatabase processor:


                    {
                    "STORE_ID": 1,
                    "STORETYPE": "MHS",
                    "DEF_LANGUAGE_ID": -1,
                    "FFMCENTER_ID": 10501,
                    "CONTRACT_ID": 10005,
                    "DIRECTORY": "ExtendedSitesCatalogAssetStore",
                    "RELATEDSTORE_ID": 10501,
                    "SETCCURR": "USD",
                    "STORE_IDENTIFIER": "AuroraESite",
                    "MEMBER_ID": -5000,
                    "LANGUAGE_ID": -1,
                    "DISPLAYNAME": "AuroraESite",
                    "DESCRIPTION": "Commerce Model Store entity",
                    "LOCALENAME": "en_US           ",
                    "INVENTORYSYSTEM": -2,
                    "CRTDBYCNTR_ID": 10004
                    }

The CreateStoreDocumentFromDatabase processor transforms the input data into the following output data:


                    { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                    {
                    "doc": {
                    "identifier": {
                    "default": {
                    "currency": "USD"
                    },
                    "specification": "store",
                    "language": "en_US",
                    "store": "AuroraESite",
                    "supported": {
                    "language": "en_US"
                    }
                    },
                    "name": {
                    "normalized": "AuroraESite",
                    "raw": "AuroraESite"
                    },
                    "description": {
                    "raw": "Commerce Model Store entity"
                    },
                    "id": {
                    "default": {
                    "contract": "10005",
                    "language": "-1",
                    "fulfillment": "10501"
                    },
                    "contract": "10004",
                    "member": "-5000",
                    "language": "-1",
                    "store": "1",
                    "supported": {
                    "language": -1
                    }
                    },
                    "type": "MHS",
                    "relationship": {
                    "parent": "10501",
                    "directory": "ExtendedSitesCatalogAssetStore"
                    },
                    "properties": [
                    {
                    "name": "inventory",
                    "value": "-2"
                    }
                    ],
                    "__meta": {
                    "created": "2020-07-31T17:36:57.059Z",
                    "modified": "2020-07-31T17:36:57.079Z",
                    "version": {
                    "min": 0,
                    "max": 0
                    }
                    }
                    },
                    "doc_as_upsert": true
                    }

Stage 2 samples

The following code is an example of the input data for the FindStoreLocationsFromDatabase processor:


                    {
                    "NAME": "Warden Plaza",
                    "DESCRIPTION": "Warden Plaza",
                    "STLOC_ID": 10039,
                    "LANGUAGE_ID": -1,
                    "IDENTIFIER": "Warden Plaza",
                    "STOREENT_ID": 1,
                    "PHONE": "905.326.8647                    ",
                    "ADDRESS1": "8250 Warden Ave",
                    "ADDRESS2": null,
                    "ADDRESS3": null,
                    "ACTIVE": 1,
                    "CITY": "Markham",
                    "STATE": "Ontario",
                    "COUNTRY": "Canada",
                    "ZIPCODE": "L6G 1C7                                 ",
                    "ATTR_NAME": "BeautyCenter###CappuccinoBar###SushiRestaurant###Type###StoreHours",
                    "ATTR_DISPLAYNAME": "Beauty center###Cappuccino bar###Sushi restaurant###Type###Store hours",
                    "ATTR_VALUE": "true###true###true###Regular Store###Mon-Fri: 10am - 9pm<br />Sat: 9am - 7pm<br />Sun: 11am - 6pm<br />",
                    "ATTR_DISPLAYVALUE": "true###true###true###Regular Store###Mon-Fri: 10am - 9pm<br />Sat: 9am - 7pm<br />Sun: 11am - 6pm<br />",
                    "FFMC_NAME": "Warden Plaza",
                    "FFMCENTER_ID": "11539"
                    }

The FindStoreLocationsFromDatabase processor transfers the input data with the store id passed into the class from NiFi FlowFile class as attribute into following output data:


                    { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                    {
                    "doc": {
                    "locations": {
                    "10039": {
                    "identifier": "Warden Plaza",
                    "address": [
                    "8250 Warden Ave",
                    "Markham",
                    "Ontario",
                    "Canada",
                    "L6G 1C7"
                    ],
                    "phone": "905.326.8647",
                    "name": "Warden Plaza",
                    "description": "Warden Plaza",
                    "attributes": [
                    {
                    "name": "Beauty center",
                    "value": "true"
                    },
                    {
                    "name": "Cappuccino bar",
                    "value": "true"
                    },
                    {
                    "name": "Sushi restaurant",
                    "value": "true"
                    },
                    {
                    "name": "Type",
                    "value": "Regular Store"
                    },
                    {
                    "name": "Store hours",
                    "value": "Mon-Fri: 10am - 9pm<br />Sat: 9am - 7pm<br />Sun: 11am - 6pm<br />"
                    }
                    ],
                    "id": "10039",
                    "fulfillment": [
                    {
                    "name": "Warden Plaza",
                    "id": "11539"
                    }
                    ]
                    }
                    },
                    "__meta": {
                    "modified": "2020-07-31T17:55:37.124Z"
                    }
                    }
                    }

Stage 3 samples

The following code is an example of the input data from SQL for the FindStoreNamesFromDatabase processor:


                    {
                    "STOREENT_ID": 1,
                    "LANGUAGE_ID": -1,
                    "A_DESCRIPTION": "Extended Sites Catalog Asset Store",
                    "E_DESCRIPTION": "Commerce Model Store entity",
                    "A_DISPLAYNAME": "Extended Sites Catalog Asset Store",
                    "E_DISPLAYNAME": "AuroraESite"
                    }

The processor transforms the input data with store id passed from NiFi FlowFile class as the attribute into the following output data:


                    { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                    {
                    "doc": {
                    "name": {
                    "normalized": "AuroraESite",
                    "raw": "AuroraESite",
                    "text": "AuroraESite"
                    },
                    "description": {
                    "normalized": "Commerce Model Store entity",
                    "raw": "Commerce Model Store entity",
                    "text": "Commerce Model Store entity"
                    },
                    "__meta": {
                    "modified": "2020-07-31T18:44:35.288Z"
                    }
                    }
                    }

Stage 4 samples

The following code is an example of the input data for the FindConfigurationsFromDatabase processor:


                                {
                                "NAME": "CK_imagePath@@@is_ext_order_system@@@wc.cmc.storefunctions.hide@@@wc.price.variant@@@wc.search.entitlement@@@wc.search.priceMode@@@wc.search.priceMode.compatiblePriceIndex@@@wc.seo.defaultURLLangId",
                                "STOREENT_ID": "11001@@@0@@@11001@@@1@@@11001, 0@@@11001, 0@@@1@@@11001",
                                "VALUE": "/wcsstore/AuroraStorefrontAssetStore/@@@N@@@RequisitionList,allowOrgRegistration,accountParticipantRole,MobilePhone@@@enabled@@@0###0@@@1###1@@@1.0@@@-1"
                                }

The FindConfigurationsFromDatabase processor transforms the input data into the following output data:


                                        { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                                        {
                                        "doc": {
                                        "configurations": [
                                        {
                                        "name": "wc.search.priceMode.compatiblePriceIndex",
                                        "value": "1.0"
                                        }
                                        ],
                                        "__meta": {
                                        "modified": "2020-07-31T18:56:04.119Z"
                                        }
                                        }
                                        }

Stage 5 samples

A sample input data for FindFeaturesFromDatabase as the following:

{
  "NAME": "AccountActivation@@@AdvancedSearch@@@Age@@@AgeOptions@@@AllowReOrder@@@Analytics@@@ApplePay@@@BOPIS@@@BrowsingHistory@@@CDNCaching@@@CategoryProductBlogs@@@CategorySubscriptions@@@ConsentOptions@@@CouponWallet@@@CustomLogo@@@DateOfBirth@@@EmailOption@@@EnableQuotes@@@ExpandedCategoryNavigation@@@ExpeditedOrders@@@FacebookIntegration@@@FutureOrders@@@Gender@@@InventoryAvailability@@@KeywordSuggestions@@@MarketingConsent@@@MobilePhone@@@MultipleActiveOrders@@@MultipleShipments@@@PaymentPromotion@@@Permanent@@@PhotoGallery@@@ProductRankings@@@ProductRatingsReview@@@ProductSuggestions@@@ProductionServer@@@PunchoutPayment@@@QuickOrder@@@RatingReviewIntegration@@@RecurringOrders@@@RemoteWidget@@@SOAWishlist@@@SearchBasedNavigation@@@Session@@@SharedShippingBillingPage@@@ShipAsComplete@@@ShippingChargeType@@@ShippingInstructions@@@ShowHideOrderItems@@@SideBySideIntegration@@@StoreLocator@@@Subscription@@@Tealeaf@@@TrackingStatus@@@UseCommerceComposer@@@UseSynonyms@@@WCHIntegration@@@accountParticipantRole@@@allowOrgRegistration@@@contractSelection@@@on-behalf-of@@@on-behalf-of-csr@@@preferredCurrency@@@preferredLanguage@@@promotionCode@@@quickCheckout@@@search@@@sterlingConfigurator",
  "STOREENT_ID": "11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001, 11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001@@@11001, 11001@@@11001@@@11001@@@11001@@@11001",
  "VALUE": "false@@@true@@@false@@@true@@@true@@@false@@@false@@@true@@@false@@@false@@@false@@@false@@@false@@@true@@@false@@@true@@@true@@@false@@@false@@@false@@@false@@@true@@@true@@@true@@@true@@@false@@@false###true@@@false@@@true@@@false@@@false@@@false@@@false@@@false@@@false@@@false@@@false@@@false@@@false@@@true@@@true@@@true@@@true@@@false@@@true@@@true@@@false@@@true@@@false@@@false@@@true@@@true@@@false@@@true@@@true@@@true@@@false@@@false@@@false@@@false@@@false@@@true@@@true@@@false###true@@@true@@@true@@@true@@@false"
}

The FindFeaturesFromDatabase processor transforms the input data into the following output data:

{ "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
{
  "doc": {
    "features": [
      {
        "name": "AccountActivation",
        "value": "false"
      },
      {
        "name": "AdvancedSearch",
        "value": "true"
      },
      {
        "name": "Age",
        "value": "false"
      },
      {
        "name": "AgeOptions",
        "value": "true"
      },
      {
        "name": "AllowReOrder",
        "value": "true"
      },
      {
        "name": "Analytics",
        "value": "false"
      },
      {
        "name": "ApplePay",
        "value": "false"
      },
      {
        "name": "BOPIS",
        "value": "true"
      },
      {
        "name": "BrowsingHistory",
        "value": "false"
      },
      {
        "name": "CDNCaching",
        "value": "false"
      },
      {
        "name": "CategoryProductBlogs",
        "value": "false"
      },
      {
        "name": "CategorySubscriptions",
        "value": "false"
      },
      {
        "name": "ConsentOptions",
        "value": "false"
      },
      {
        "name": "CouponWallet",
        "value": "true"
      },
      {
        "name": "CustomLogo",
        "value": "false"
      },
      {
        "name": "DateOfBirth",
        "value": "true"
      },
      {
        "name": "EmailOption",
        "value": "true"
      },
      {
        "name": "EnableQuotes",
        "value": "false"
      },
      {
        "name": "ExpandedCategoryNavigation",
        "value": "false"
      },
      {
        "name": "ExpeditedOrders",
        "value": "false"
      },
      {
        "name": "FacebookIntegration",
        "value": "false"
      },
      {
        "name": "FutureOrders",
        "value": "true"
      },
      {
        "name": "Gender",
        "value": "true"
      },
      {
        "name": "InventoryAvailability",
        "value": "true"
      },
      {
        "name": "KeywordSuggestions",
        "value": "true"
      },
      {
        "name": "MarketingConsent",
        "value": "false"
      },
      {
        "name": "MobilePhone",
        "value": "false"
      },
      {
        "name": "MultipleActiveOrders",
        "value": "false"
      },
      {
        "name": "MultipleShipments",
        "value": "true"
      },
      {
        "name": "PaymentPromotion",
        "value": "false"
      },
      {
        "name": "Permanent",
        "value": "false"
      },
      {
        "name": "PhotoGallery",
        "value": "false"
      },
      {
        "name": "ProductRankings",
        "value": "false"
      },
      {
        "name": "ProductRatingsReview",
        "value": "false"
      },
      {
        "name": "ProductSuggestions",
        "value": "false"
      },
      {
        "name": "ProductionServer",
        "value": "false"
      },
      {
        "name": "PunchoutPayment",
        "value": "false"
      },
      {
        "name": "QuickOrder",
        "value": "false"
      },
      {
        "name": "RatingReviewIntegration",
        "value": "false"
      },
      {
        "name": "RecurringOrders",
        "value": "true"
      },
      {
        "name": "RemoteWidget",
        "value": "true"
      },
      {
        "name": "SOAWishlist",
        "value": "true"
      },
      {
        "name": "SearchBasedNavigation",
        "value": "true"
      },
      {
        "name": "Session",
        "value": "false"
      },
      {
        "name": "SharedShippingBillingPage",
        "value": "true"
      },
      {
        "name": "ShipAsComplete",
        "value": "true"
      },
      {
        "name": "ShippingChargeType",
        "value": "false"
      },
      {
        "name": "ShippingInstructions",
        "value": "true"
      },
      {
        "name": "ShowHideOrderItems",
        "value": "false"
      },
      {
        "name": "SideBySideIntegration",
        "value": "false"
      },
      {
        "name": "StoreLocator",
        "value": "true"
      },
      {
        "name": "Subscription",
        "value": "true"
      },
      {
        "name": "Tealeaf",
        "value": "false"
      },
      {
        "name": "TrackingStatus",
        "value": "true"
      },
      {
        "name": "UseCommerceComposer",
        "value": "true"
      },
      {
        "name": "UseSynonyms",
        "value": "true"
      },
      {
        "name": "WCHIntegration",
        "value": "false"
      },
      {
        "name": "accountParticipantRole",
        "value": "false"
      },
      {
        "name": "allowOrgRegistration",
        "value": "false"
      },
      {
        "name": "contractSelection",
        "value": "false"
      },
      {
        "name": "on-behalf-of",
        "value": "false"
      },
      {
        "name": "on-behalf-of-csr",
        "value": "true"
      },
      {
        "name": "preferredCurrency",
        "value": "true"
      },
      {
        "name": "preferredLanguage",
        "value": "false"
      },
      {
        "name": "promotionCode",
        "value": "true"
      },
      {
        "name": "quickCheckout",
        "value": "true"
      },
      {
        "name": "search",
        "value": "true"
      },
      {
        "name": "sterlingConfigurator",
        "value": "false"
      }
    ],
    "__meta": {
      "modified": "2020-09-11T06:13:00.050Z"
    }
  }
}

Stage 6 samples

The following code is an example of the input data for the FindSupportedLanguagesFromDatabase processor:


                                {
                                "STOREENT_ID": 1,
                                "LANGUAGE_ID": "-1",
                                "LOCALENAME": "en_US"
                                }

The FindSupportedLanguagesFromDatabase processor transforms the input data into the following output data:


                                        { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                                        {
                                        "doc": {
                                        "identifier": {
                                        "supported": {
                                        "language": [
                                        "en_US"
                                        ]
                                        }
                                        },
                                        "id": {
                                        "supported": {
                                        "language": [
                                        "-1"
                                        ]
                                        }
                                        },
                                        "__meta": {
                                        "modified": "2020-08-03T13:19:38.979Z"
                                        }
                                        }
                                        }

Stage 7 samples

The following code is an example of the input data for the FindSupportedCatalogsFromDatabase processor:


            {
            "STOREENT_ID" : "10501, 10501, 10501",
            "DEFAULT_CATALOG_ID" : "######10502               ",
            "CATALOG_ID" : "10001, 10501, 10502",
            "IDENTIFIER" : "Extended Sites Catalog Asset Store###Extended Sites Catalog Asset Store B2B Direct###Extended Sites Catalog Asset Store Consumer Direct",
            "MASTERCATALOG" : "1###0###0"
            }

The FindSupportedCatalogsFromDatabase processor transforms the input data with the store id and language id passed in from NiFi FlowFile class as attributes into the following output data:


                { "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
                {
                "doc": {
                "identifier": {
                "default": {
                "catalog": "Extended Sites Catalog Asset Store Consumer Direct"
                },
                "master": {
                "catalog": "Extended Sites Catalog Asset Store"
                },
                "supported": {
                "catalog": [
                "Extended Sites Catalog Asset Store",
                "Extended Sites Catalog Asset Store Consumer Direct"
                ]
                }
                },
                "id": {
                "default": {
                "catalog": "10502"
                },
                "master": {
                "catalog": "10001"
                },
                "supported": {
                "catalog": [
                "10001",
                "10502"
                ]
                }
                },
                "__meta": {
                "modified": "2020-08-08T03:37:21.743Z"
                }
                }
                }

Stage 8 samples

The following code is an example of the input data for the FindSupportedCurrenciesFromDatabase processor:


{
  "STOREENT_ID": 1,
  "CURRENCY": "USD###EUR"
}

The FindSupportedCurrenciesFromDatabase processor transforms the input data with the store id, and languages id passed into from NiFi FlowFile class as attributes into the following output data:


{ "update": { "_id": "1--1", "_index": "auth.store", "retry_on_conflict": 5, "_source": false } }
{
  "doc": {
    "identifier": {
      "supported": {
        "currency": [
          "USD",
          "EUR"
        ]
      }
    },
    "__meta": {
      "modified": "2020-08-03T16:05:32.234Z"
    }
  }
}