CI/CD Introduction

The HCL Commerce Version 9 continuous integration and delivery (CI/CD) pipeline combines containerization, cloud-service integration and extensive customization. Since it uses loosely coupled open-source tools, the pipeline is highly flexible and customizable. This combination of technical capabilities enables you to seamlessly develop and deliver updates, enhancements and fixes.

If you want to continuously deploy changes or improvements to your HCL Commerce platform in an agile way, consider implementing a continuous integration and delivery pipeline. CI/CD is not for everyone; if you only occasionally make changes to your sites, you may not need it. CI/CD is compatible with the traditional model of infrequent major releases, but even in such environments it supports rapid fixes and continuous uptime.

There are many ways to partially automate the development and delivery process. In the HCL Commerce context, integration, delivery, and deployment have very specific meanings. The HCL Commerce Version 9 CI/CD system defines its terms as follows:
Continuous integration (CI)
In CI, teams integrate their work more frequently (usually on a daily basis), using automation of tests and techniques to merge their code.
Continuous delivery (CD)
Builds are produced in short cycles with a high degree of reliability. All builds are essentially preproduction builds.
Continuous deployment
Builds are automatically deployed to production.
Toolchain
A kind of CI/CD pipeline that is more focused on the software stack than team organization.

Continuous delivery is not the same as continuous deployment. The term CI/CD is often used to describe a continuous integration and continuous delivery process, and this definition is applicable for a certain set of HCL Commerce users. However, although HCL Commerce is capable of providing continuous deployment, it is continuous delivery that is more appropriate for the majority of HCL Commerce implementations.

Continuous integration is focused on delivering new code for features and fixes. Changes are delivered faster and allow continuous feedback on the code and features being delivered. CI is achieved through automation of testing, integrations, provisioning, and the reduction of bottlenecks to complete that testing. You are reducing the overhead and the amount of rework during testing and provisioning of environments while ensuring consistency and accuracy. Through the use of software and processes such as Git, Jenkins for automation, and agile methods, you can share code within a development team with consistency, speed and in a coordinated manner.

Continuous delivery is the capability of delivering the code changes by deploying to environments and provisioning at any step of a development chain at any time. A good example of this is providing fixes for a bug. With CD, you can reduce the cost and time to deliver, test, provision environments, and deploy such fixes. A CI pipeline and a CD pipeline working in concert and collaboratively make a pipeline.

Integration and delivery form an iterative process comprising process, automation and software. You are automating aspects of the project and infrastructure to allow CI/CD, however the HCL Commerce pipeline is not a simple toolchain that includes continuous deployment. Continuous deployment is essentially adding a step, after CI/CD, that directly and automatically deploys your changes to production. For most production environments, this is not desired. In most cases, Line of Business oversight and some form of release management are involved, for instance when establishing holiday readiness and for sales campaigns.

Containerization and horizontal scaling

The main innovation that makes continuous integration and delivery possible for HCL Commerce is containerization, and specifically Docker. Encapsulating the server structure of HCL Commerce in Docker containers enables rapid development, turnaround, and horizontal rather than vertical scaling of server resources.

'Dockerization' provides' several immediate benefits:
  • Get up and running faster with an immutable image, with no OS overhead. Such an image encapsulates not only the application logic, but also the configuration variables for middleware and even the OS. For example, front-end changes can reside in your store servers as a container, search customizations can reside in your search containers and your back-end customizations live in the Externalized Customizations server.
  • Accelerate innovation by removing infrastructure concerns from application development. Developers can deploy customizations to target environments, in an automated way, without having to rely on IT operation team.
  • Decrease costs by reducing platform footprint, for less hardware and fewer required skills. Administrators and developers can load environments in minutes versus hours or even days, and easily keep them updated.
  • Scale more easily and cost effectively by quickly spinning up new images. You can solve most performance problems by 'throwing more containers at the problem.' This is known as horizontal scaling.
Vertical scaling is the ability to increase the capacity of existing hardware or software by adding resources. Vertical scaling is limited by the fact that you can only get as big as the size of the server. Horizontal scaling affords the ability to scale wider to deal with traffic. It is the ability to connect multiple hardware or software entities, such as servers, so that they work as a single logical unit. Simply by adding extra hardware and containers, you can avoid having to maintain the complex clustering and load-balancing configurations that were necessary for traffic management in previous versions of HCL Commerce.
As long as you follow the iterative HCL Commerce CI/CD pipeline process, you can choose the appropriate tools for each step in the cycle. For example, you can customize the pipeline by:
  • Using any of the popular Docker orchestration tools (Mesos/Marathon, Nomad, Docker Compose, Docker Swarm, Kubernetes, AZK)
  • Serving containers from one of the popular Cloud Platform (AWS, Azure, Google Cloud, Softlayer)
  • Customizing the pipeline for different scenarios (from development to deployment)
  • Build up your infrastructure based on your own business model.
The following diagram illustrates a generic CI/CD pipeline.
The pipeline starts with a development environment in which you can check-in code for merging and branching. In this example GitHub is used to ensure that new code passes integration testing.

To help with the testing, you package your customizations. In previous versions of HCL Commerce, you publish EAR and WAR files to application servers in environments that have to be available, and you try to keep them consistent. That is a very manual method of testing and is error prone; it is hard to find and maintain the environments. In a Docker world, you use Docker images that provide the services for your solution, such as store containers, search containers etc. You can also build your own containers to work with legacy code or to use third-party services. (For example, in the GDPR context you could integrate a Continuous Security Validation pipeline. This pipeline consists of tools, processes and controls that ensure security compliance, auditing, pen-testing and vulnerability testing of the code.) The overarching cycle repeats as new code and features are delivered.

You iteratively build and deploy your customizations to the containers and after successful testing, you add the customized Docker images to your Docker Registry. Using tools such as Vault or Consul, you store your configurations, settings, and security configurations outside of the containers so that the containers can acquire the correct settings to run on the appropriate environment. Since each individual server does not have to maintain its own configuration information, servers can easily be added and replaced, updated or removed.

You then deploy your customized images to the appropriate environment using a Docker orchestration solution such as Kubernetes. The Docker orchestration solution, along with vendor options or third party options, helps load balance and optimize workloads.

For more information about Docker, see Docker Containerization Unlocks the Potential for Dev and Ops. For information about implementing a containerization plan for your operation, see Planning the Docker container startup.

Planning and best practices

Begin by assessing whether CI/CD provides any benefits to your operation. Continuous delivery is popular with end-users, but it can be unsettling if relied-upon features are constantly changing. For the development team, it may require organizational or institutional changes and you must ensure that these are appropriate.

Not everything can or should be automated. The pipeline may be an automated process from an organizational standpoint, but many hands-on checks and balances need to be in place to ensure quality and security. In particular, do not confuse continuous release with continuous deployment: good practices must be place to ensure that code is only released when it is ready. It is also counterproductive to automate the wrong processes, or to automate a process just because it can be automated.

For a thorough discussion of pipeline architecture including how to design a system appropriate to your operation, see Planning a CI/CD pipeline.