Creating Entry sources

About this task

To create Entry Sources, complete the following steps.

Procedure

  1. In the Entry Sources menu, click + Entry source.
    The Create entry source page appears.
  2. Provide appropriate values for the following fields.
    • Name- Mandatory
    • Description- Optional
  3. If you want to save the Entry Source in a location different than the root (/) folder, click Browse and select the required folder.
  4. In the Import data from section, select from one of the following options:
    1. Upload File - CSV, TSV, JSON.
    2. Real Time - REST API, KAFKA, Unica Campaign, Unica Interact, Unica Discover
    Note:
    • Use CSV, TSV, and JSON when you have a static data of customers. Use REST API, Kafka, Unica Campaign, Unica Interact and Unica Discoverfor transactional mode.
    • In case of CSV, TSV, and JSON, you can add multiple files. If Journey encounters duplicate data, it will either update the data or discard the data based on the settings made in Journey Settings or Deduplication. Even if you have not configured Deduplication settings, Journey accepts the new data and processes the newly added files.
    • CSV - If you select this option, upload the appropriate CSV data file.
    • TSV - If you select this option, upload the appropriate TSV data file.
    • JSON - If you select this option, upload the appropriate JSON data file.
    • REST API - If you select this option, you will receive a static URL to which you can push the data dynamically.
      1. Generate REST API keys. For more information, see Creating a new REST integration.
      2. Generate the authentication token using the URL http://<localhost>:<port-number>/journey/api/thirdpartylogin.
      3. To authenticate this URL, use the REST API keys, generated in step (a).
      4. To send the data to Journey use Rest API mentioned above use the authentication token and entrySourceCode while sending the data.
      5. entrySourceCode is mandatory to send data on REST Entry Source.
    • Kafka - If you select this option, you will see a dedicated Kafka topic name on which you can post your data. By default, STREAMING_IMPORT is selected as Kafka topic name. New Kafka topic name can be added by clicking on the button. A new topic of the given name will be created on KAFKA. If for any reason (example server down) the topic is not created on Kafka, then an error message will be displayed saying Unable to create KAFKA topic ES creation fails. User will then have to choose a different name or retry after sometime.
      Note: When user wants to use multiple kafka brokers and they start their engine with single kafka borker, then topics by default get created on this single broker. Later, when user updates the broker list then kafka doesn’t replicate the topics on other brokers because at the time of installation the replication value was set to one. Hence, during installation only, user needs to specify whether they want to use kafka with multiple brokers or with single broker.

      Cluster Kafka:

      Kafka Broker list: <BROKER_HOST1>:<PORT>, <BROKER_HOST2>:<PORT>, <BROKER_HOST3>:<PORT>

      Information message on the broker list format.

      What it shall do:
      1. Update engine and web application properties with below parameter

        spring.kafka.bootstrap-servers=<BROKER_HOST1>:<PORT>, <BROKER_HOST2>:<PORT>, <BROKER_HOST3>:<PORT>

      2. JourneyEngine :

        We added below property in journey_master_config.properties file in Journey Engine. Need to update from installer according to number of broker.

        journey.kafka.replications = <Number of Brokers>

      3. JourneyWeb :

        We added below property in JounreyWeb/application.properties file in Journey web. Need to update from installer according to number of broker.

        journey.kafka.customtopic.replications = <Number of Brokers>

      Note: Internal kafka topic name cannot be used for creating the custom kafka topic ES. Once the topic name is configured and saved, then topic name cannot be edited. In case of deletion of Entry Source the underlying KAFKA topic will not be deleted.

      User can integrate their choice of external sources without requesting an enhancement to Unica. They can build this integration with Journey by adding the plugin in CIF. Please refer the Unica Content Integration Developer Guide for more information.

      Also, we ship integration with mailchimp out of the box, for more information on how to configure this integration, please refer Unica Content Integration Installation and Configuration Guide

      Users can also add external entry source with the help of Sync external source button present next to the Kafka Topic Name.

      1. Click on Sync external entry source

        The Add source page appears.

      2. Provide appropriate values for the following fields.
        • Select source repository - Select repository from the dropdown of the field.
        • Select audience or event type - Select audience or event type from the dropdown of the field.
        • Select Journey data definition - click Browse and select the required data definition.
      3. The Field Mapping dialog appears.
      4. Map the Journey data definition fields with the appropriate source repository fields.

      5. Click Save & close

      Selected external source along with date and time appears on the create entry source page. Select the icon for viewing external source details. Select the icon for editing external source details. Select the icon to delete the selected entry source.

    • Unica Campaign - If you select this option, then you can send data from Unica Campaign to Journey. The token validity of platform should be increased manually to 10800 seconds (3 hours) because the default token validity of platform is 15 seconds. Due to this an error message is displayed on the campaign entry source screen that Campaign login failed OR session expired. Try Relogin into application.
    Path
    Platform -> Settings -> Configuration -> General -> Miscellaneous -> Token lifetime
    Note: After increasing the validity restart the Campaign application.
    • Unica Interact - If you select this option, then you can send data from Unica Interact to Journey.
    • Unica Discover - If you select this option, then you can send data from Unica Discover to Journey.
  5. Click Create entry source.
    Note: For CSV and TSV type file entry source, the first row in file type entry source should be the data definition field names (with exact case and spelling as mentioned in corresponding data definition). It is not applicable for Json type file entry source.
    Note: Data passed in entry source in case of date fields (if the field is of datatype date in data definition to be used in Journeys) should match with the date format entered in data definition to avoid any inconsistencies in data processing.