Home
Documentation
HCL Commerce is a high-availability, highly scalable and customizable e-commerce platform. Able to support hundreds of thousands of transactions per day, HCL Commerce allows you to do business with consumers (B2C) or directly with businesses (B2B). HCL Commerce uses cloud friendly technology to make deployment and operation both easy and efficient. It provides easy-to-use tools for business users to centrally manage a cross-channel strategy. Business users can create and manage precision marketing campaigns, promotions, catalog, and merchandising across all sales channels. Business users can also use AI enabled content management capabilities.
Customizing
The topics in the Customizing section describe tasks performed by an application developer to customize HCL Commerce.
Search
HCL Commerce comes with a powerful and fully integrated search function. The search functions in HCL Commerce provide an enriched customer experience, with features such as automatic search term suggestions and spelling correction. Since it is built on industry standards, HCL Commerce Search is highly flexible and extensible. Starter stores can use the search engine's most sophisticated features without requiring extra customization. Starting with Version 9.1, HCL Commerce Search with Elasticsearch microservices manage indexing and other crucial tasks, with no performance impact on the storefront or transaction server.
Elasticsearch-based solution
The topics in this section describe tasks performed by an application developer to customize the Elasticsearch-based search solution. You can extend the Ingest Service and Query Service to meet your business requirements.
Extending the Ingest service
The Version 9.1 HCL Commerce Search service includes Ingest and Elasticsearch, which together form a distributed, scalable search and analytics engine. You can chain together the Ingest service's rule-based aggregations to boost relevancy for hero SKUs or certain facets, and use sophisticated metrics to analyze your search performance.
Custom database tables for Ingest
You can ingest catalog data from your database by adding a custom database table for CATENTRY and CATGROUP. You can also control how database data is serialized into Elasticsearch to bring in large datasets.
Using List Aggregation with Ingest
You can issue SQL calls to retrieve database data from inside the NiFi pipeline. This information is converted from the "2d" tabular format in which the database stores it, to the one-dimensional string that the Elasticsearch process expects. Normally this list aggregation takes place inside the database, however each database imposes a limit on how long a returned string can be; in the case of DB2, for example, this length is 32k. If the SQL tries to serialize a longer string, it will be truncated.

Documentation
HCL Commerce is a high-availability, highly scalable and customizable e-commerce platform. Able to support hundreds of thousands of transactions per day, HCL Commerce allows you to do business with consumers (B2C) or directly with businesses (B2B). HCL Commerce uses cloud friendly technology to make deployment and operation both easy and efficient. It provides easy-to-use tools for business users to centrally manage a cross-channel strategy. Business users can create and manage precision marketing campaigns, promotions, catalog, and merchandising across all sales channels. Business users can also use AI enabled content management capabilities.
- Getting started
  HCL Commerce has different advantages for business users, administrators and developers. HCL Commerce targets each of these roles with a tailored set of offerings so that each of your users can get maximum benefit.
- Installing and deploying
  Learn how to install and deploy HCL Commerce development environments and HCL Commerce production environments.
- Migrating
  Before you migrate to HCL Commerce Version 9.1, review this information to help plan and execute your migration.
- Operating
  Topics in the Operating category highlight tasks that are typically performed by business users, customer support representatives, to complete their day-to-day tasks in the operation of the HCL Commerce site.
- Integrating
  Topics in the Integrating category highlight the tasks that are commonly performed for using HCL Commerce in combination with other products.
- Administering
  Topics in the Administering category highlight tasks that are typically performed by the Site Administrator, to support daily operations of the HCL Commerce site.
- Customizing
  The topics in the Customizing section describe tasks performed by an application developer to customize HCL Commerce.
  - Customizing Management Center for HCL Commerce
    HCL Commerce allows you to pull the GIT bundle of Management Center source code and start the application locally to customize the Management Center.
  - Managed asset deployment
    Managed assets are files that are uploaded by business users to be used for store marketing, or to supplement products. They are added to HCL Commerce through the Assets tool or the Marketing tool in Management Center. By default, managed assets are extracted and deployed through the HCL Commerce EAR. To maintain performance in a large-scale production environment, site administrators must switch to alternative extraction and deployment methods for managed assets.
  - HCL Commerce development environment
    HCL Commerce Developer is the development toolkit for customizing a HCL Commerce application.
  - Functional architecture
    Functional architecture provides both the set of patterns used to implement the business functionality and the frameworks in which these business functions execute.
  - Persistent object model
    HCL Commerce deals with a large amount of persistent data. There are numerous tables defined in the current database schema. Even with this extensive schema, however, you might need to extend or customize the database schema for your particular business needs.
  - Creating your custom store
    After you install and set up your programming environment, you can create your custom store and customize your storefront. You must ensure that the store server is properly configured, and that your store assets are moved to the Store server.
  - Presentation layer
    HCL Commerce uses Java Server Pages (JSP) to implement the view layer of the Model-View-Controller (MVC) design pattern. The view layer is in charge of retrieving data from the database through the use of data beans and formatting it to meet the display requirements. The view layers determines whether the request is sent to a browser or streamed out as XML. JSP files present a clean separation between data content and presentation.
  - Controller layer
    The Controller layer is the conductor of operations for a request. It controls the transaction scope and manages the session related information for the request. The controller first dispatches to a command and then calls the appropriate view processing logic to render the response.
  - Business logic layer
    The business logic layer is the business components that provide OAGIS services to return data or start business processes. The presentation layer uses these OAGIS services to display data, or to invoke a business process. The business logic provides data required by the presentation layer. The business logic layer exists because more than just fetching and updating data is required by an application; there is also additional business logic independent of the presentation layer.
  - Persistence layer
    The interaction between the business objects and persistence layer is isolated in an object called the Business Object Mediator. Business object document (BOD) commands interact with the Business Object Mediator to handle the interaction with the logical objects and how they are persisted.
  - Business model information model
    A business model, a representation of the business processes used throughout the site, provides a sample commerce solution which includes an organization structure, default user roles and access control policies, one or more starter stores, administration tools, and business processes that demonstrate best practices. A business model can be customized to support business requirements and scenarios. HCL Commerce provides sample business models that show some common commerce solutions. These business models are created by setting up an organization hierarchy structure, access control policies, stores, and contracts that help satisfy the necessary business requirements.
  - Business models
    Before starting to develop your site with HCL Commerce, you need to determine the business model supported by HCL Commerce that best represents the purpose of your site. Usually sites created with HCL Commerce will be implemented based on of one of these business models.
  - Store data information model
    Store data is the information that is loaded into the Transaction server database, which allows your store to function. The URL Registry Entries and View Registry Entries packages are included in the diagram, but they are not database assets. These entries are presentation configuration (that is, struts actions and forwards) that must be deployed. URL registry entries are shown in the diagram to illustrate the entire store data information model. To operate properly, a store must have the data in place to support all customer activities. For example, in order for a customer to make a purchase, your store must contain a catalog of goods for sale (catalog data), the data associated with processing orders (tax and shipping data), and the inventory to fulfill the request (inventory and fulfillment data).
  - Customizing HCL Commerce
    You can extend the HCL Commerce product to fit your business needs. This topic describes the prerequisite skills and required knowledge that you need to customize business logic. After you have the required knowledge, use HCL Commerce Developer to take tutorials that guide you step-by-step through various customization scenarios.
  - Run Engine command framework
    The Run Engine command framework provides predefined commands, that you can use to change environment parameters or container configurations. This framework is built into the HCL provided Docker images.
  - GraphQL for HCL Commerce
    The GraphQL markup language is available for any API. It is a server-side interpreter for processing queries using a data type system you design. With GraphQL, your data and code are independent of any database or storage system.
  - Representational State Transfer (REST) services
    HCL Commerce uses Representational State Transfer (REST) services to provide a framework that can be used to develop RESTful applications on several platforms. These platforms can include web, mobile, kiosks, and social applications.
  - Web services
  - Search
    HCL Commerce comes with a powerful and fully integrated search function. The search functions in HCL Commerce provide an enriched customer experience, with features such as automatic search term suggestions and spelling correction. Since it is built on industry standards, HCL Commerce Search is highly flexible and extensible. Starter stores can use the search engine's most sophisticated features without requiring extra customization. Starting with Version 9.1, HCL Commerce Search with Elasticsearch microservices manage indexing and other crucial tasks, with no performance impact on the storefront or transaction server.
    - Search terminology
      HCL Commerce Search uses a number of specialized terms. A concise list of the most commonly used terms is provided to get you started.
    - Migrating to Elasticsearch
      You have several options in migrating from the Solr search platform to the Version 9.1 Elasticsearch solution.
    - Elasticsearch-based solution
      The topics in this section describe tasks performed by an application developer to customize the Elasticsearch-based search solution. You can extend the Ingest Service and Query Service to meet your business requirements.
      - Indexing choices
        You have two options for indexing Extended Site (eSite) stores, the legacy eSite Store indexing model, or the Catalog Asset Store (CAS) model. The CAS model is recommended because of its speed and scalability.
      - Extending the Ingest service
        The Version 9.1 HCL Commerce Search service includes Ingest and Elasticsearch, which together form a distributed, scalable search and analytics engine. You can chain together the Ingest service's rule-based aggregations to boost relevancy for hero SKUs or certain facets, and use sophisticated metrics to analyze your search performance.
        Managing connectors in the Ingest service
        How to manage your descriptors and connectors, so as to optimize your search indexing for Elasticsearch.
        Verifying index integrity
        You can use a specialized Ingest check pipeline to verify that the Elasticsearch index contains all the required data. This pipeline verifies that all incoming data is correctly inserted in the index. The pipeline compares the results of the Ingest process with the original input to ensure reliable data intake.
        Upgrading batch version for pipes
        To use the latest version of your Ingest components, the preferred approach is to upgrade your pipelines using the Version context menu in the NiFi editor.
        Creating a custom index
        You may want to incorporate searches for data that is specific to your business and is not related to the commerce catalog. Aternatively, you may want to extend the existing product index in some way. In such cases, you can create a custom index for Elasticsearch or for the Query service to use. You can use this custom index when appropriate to include extra data in your product or category search result.
        Using Maven and Project Object Modeling
        Maven's Project Object Modeling (POM) system lets you manage project builds, reporting, and documentation all in one place.
        Customizing Ingest ETL SQL
        You can use the Java ETL when integrating and extending NiFi and Elasticsearch. ETL processes take place in the side flow of data input. In the side flow, extraction is managed by SQL queries, which you can customize.
        Ingest profiles
        You can use Ingest profiles to define Java extensions for SQL database data extraction and to define data transformation logic. Ingest profiles are bound to connector pipes and executes as part of the pipe process.
        Ingest configuration via REST
        You can configure all your Ingest connectors via a single REST endpoint. All connectors inherit a set of configurations such as global attributes and make them available as NiFi flowfile attributes inside NiFi. Having access to this endpoint, you can add or remove NiFi capabilities from outside the main NiFi process flow.
        Creating a NiFi connector
        Connectors are sets of data pipes with built-in processing, which together form a pipeline. To create a new connector, define a descriptor to describe the schema, the pipeline the connector will use, and the schedule (interval, recurrence) that it follows.
        Creating a custom NiFi process group
        NiFi process groups and their connections are the building blocks of dataflow pipeline that perform data ingestion and transformation tasks to ready the data for the search index. Creating your own custom process group provides a way to perform data ingestion and transformation tasks according to specific business requirements. Here are the four default templates to help you to get started with the creation of a specific process group according to your business logic.
        Custom NiFi processors
        NiFi Processors are the basic building blocks of dataflow pipelines. Processors perform specific tasks within the pipeline, such as listening for incoming data, pulling data from external sources, publishing data to external sources, and routing, transforming, or extracting information from flow files. Creating your own custom processor provides a way to perform different operations or to transform flow file content according to specific business requirements.
        Using the NiFi status API
        Use the NiFi status API to obtain information regarding NiFi. You can specifically obtain information about NiFi threads, queues, components, or versions.
        Extending Ingest connectors
        HCL Commerce provides default Ingest connectors that perform data ingestion and transformation tasks to ready the data for the search index. You can use them as is or can customize these default Ingest connectors as per your business requirements using the Search Ingest Service API. This topic describes what customizations you can make and how you can manage these customizations and move them across different versions.
        Updating NiFi process group, processor, controller service using Ingest connector descriptor
        Persisting Ingest changes (NiFi Registry)
        This topic describes how to make changes to a specific pipe by changing its property/configuration across all connectors. For example, changing SQL statement for a processor, Execute SQL, in a pipe, DatabaseCategoryStage1b, across all connector in NiFi.
        Ingesting unstructured content
        You can customize the Ingest service to enable it to crawl website data and ingest selected content into Elasticsearch. This capability can be added in the HCL Commerce Search V9.1 NiFi container.
        Custom database tables for Ingest
        You can ingest catalog data from your database by adding a custom database table for CATENTRY and CATGROUP. You can also control how database data is serialized into Elasticsearch to bring in large datasets.
        Using List Aggregation with Ingest
        You can issue SQL calls to retrieve database data from inside the NiFi pipeline. This information is converted from the "2d" tabular format in which the database stores it, to the one-dimensional string that the Elasticsearch process expects. Normally this list aggregation takes place inside the database, however each database imposes a limit on how long a returned string can be; in the case of DB2, for example, this length is 32k. If the SQL tries to serialize a longer string, it will be truncated.
      - Extending the Query Service
        The Query service builds the search expressions and then hands the expression to Elasticsearch. You can customize this service to create your own expression providers, pre-processors, post-processors, and custom handlers as per your business requirements. There is no JDBC connectivity from the Query service to the Commerce database. This architecture ensures that the application can be stateless and scaled independently. Business data can still be provided to the Query service either by indexing or through other micro-services.
      - Extending Elasticsearch
        You can customize the behavior of Elasticsearch through ZooKeeper, or by adding your own REST API endpoints. A variety of approaches to extending the Elasticsearch service are discussed and methods are provided.
      - Fine-tuning search results
        HCL Commerce Search helps you to enhance the search experience of shoppers by adjusting your search results based on your shoppers' interactions at the storefront.
      - CI/CD strategy to deploy customizations
        The HCL Commerce Search Elasticsearch solution does not dictate the data repository or scripting needed to automate a Continuous Integration/Continuous Delivery (CI/CD) pipeline. You have the flexibility to use your own data repository and build structure. The following topics describe a recommended approach including a suggested data pipeline structure.
      - Migrating Elasticsearch customizations
        You can migrate existing Solr extensions to the data service model, and keep your custom connectors and NiFi flows up to date with the latest version of HCL Commerce Search.
    - Solr-based solution
      In keeping with HCL's commitment to current and open standards, HCL Commerce Search uses Apache Lucene as the basis of its Search framework. Lucene powers the Apache Solr search engine and the Elasticsearch search engine. The indexing pipeline is a more open, flexible and scalable and is tightly integrated with the data service. Using the underlying dataflow technology and architecture, you can easily customize the pipelines. This open-standards approach considerably eases the process of integrating Search with existing and third-party applications.
- Tutorials
  HCL Commerce provides many tutorials to help you customize and understand your HCL Commerce instance and stores.
- Samples
  Topics in the Samples category highlight the various samples that are provided with HCL Commerce.
- Compliance
  The following section describes how you can leverage HCL Commerce features and functionality to help your site be compliant with different privacy and security standards.
- Securing
  These topics describe the security features of HCL Commerce and how to configure these features.
- Performance
  Topics in the Performance section describe the means by which to plan, implement, test, and re-visit the optimization of HCL Commerce site performance.
- Troubleshooting
  Topics in the Troubleshooting section highlight common issues that are encountered with HCL Commerce, and how they can be addressed or mitigated.
- Reference
  Topics in the Reference section contain all of the HCL Commerce reference documentation.

HCL Commerce Version 9.1.7.0 or later

Using List Aggregation with Ingest

You can issue SQL calls to retrieve database data from inside the NiFi pipeline. This information is converted from the "2d" tabular format in which the database stores it, to the one-dimensional string that the Elasticsearch process expects. Normally this list aggregation takes place inside the database, however each database imposes a limit on how long a returned string can be; in the case of DB2, for example, this length is 32k. If the SQL tries to serialize a longer string, it will be truncated.

To avoid this problem, HCL Commerce Search provides an application-level function to perform the serialization rather than having the database do it. You control this behavior using a variable that you can set at the flow level or, as a global switch, at the level of the reindex link or the NRT link. This variable is flow.database.listagg, and the default value is True. Setting it in ReindexLink, NRTLink, or DataloadLink means to define this property globally throughout the entire dataflow, while setting this property against a connector pipe instead will scope the operation only against that given stage.

In general you should use the database LISTAGG function and only disable it when the limit is reached. This avoids the unnecessary overhead of performing the list aggregration inside the NiFi application.