Derivatives of KafkaProducerService

KafkaProducerService lets publishing messages/records onto Kafka topics. Following are the specialized types available for creating Kafka publisher service –

com.hcl.unica.system.integration.service.kafka.AbstractKafkaProducerService<K, V>The type parameter K & V represents the key & value of the message to be sent onto target Kafka topic. Refer the com.example.service.kafka.events.producer.ExampleKafkaProducerService from asset-integration-starter project for a typical service definition. In addition to declaring type arguments, this service must implement the getServiceInterface() method to be able to invoke Kafka producer service programmatically. Apart from these declarations, AbstractKafkaProducerService requires no other business logic.

Platform configuration

Once Kafka producer service is defined & declared in custom-plugin-services.yml, certain additional parameters must be set up to be able to publish to any Kafka topic. Note that Kafka connectivity details are obtained from Kafka configuration settings made for the respective system. Following additional parameters must be configured in “Additional parameters” section of respective system configuration –


Parameter	Description
{service-name}.kafka.topics	Comma separated list of target topic names.
{service-name}.kafka.topics.{topic-name}.value.format	Expected message format. Supported formats are Avro & Json.
{service-name}.kafka.batch.size	Same as standard batch.size property used for Apache Kafka producers.
{service-name}. kafka.linger.ms	Same as standard linger.ms property used for Apache Kafka producers.

Note: Refer https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html for standard Kafka producer configurations.

{service-name} represents the name of service declared in custom-plugin-services.yml file.

For example,

message-publisher-service.kafka.topics: topic1, topic2

message-publisher-service.kafka.topics.topic1.value.format: Json

message-publisher-service.kafka.topics.topic2.value.format: Avro

message-publisher-service.kafka.batch.size: 16900

message-publisher-service.kafka.linger.ms: 10

Invocation of KafkaProducerService

Content Integration Framework does not directly invoke any Kafka producer service. Plugin can define & use such service if required. Refer com.example.service.client.ExampleKafkaProducerClient from asset-integration-starter project to get clarity on how already defined Kafka producer service can be invoked from inside any other service. By default, Content Integration Framework publishes the message onto the topic configured in Platform configuration. As demonstrated in ExampleKafkaProducerClient, it is important to note that, caller of the service can completely override the target topic specifications if required.