Applications, Aggregates & Enrichments
All of the feed applications configured in HCL Detect can be seen by
clicking on the Administration
link in the top navigation bar and
then by clicking the Settings
menu item. We have to use
Feed Management tab
to see feed applications.
This dashboard shows the applications and their associated data model, e.g., ‘Topup', ‘Usage', etc.
Users with the appropriate permissions can use this page to view, edit, add and/or delete the aggregates and enrichments associated with a given feed application.
The existing aggregates configured for a feed application can be seen by clicking on
the ShowAll
button in the Aggregations
row for
that application.
The dialog showing the aggregates associated with the TopupDemo
application is below:
The aggregates associated with a feed can be edited and removed.
To explain the aggregates a bit further, let us consider the aggregate called
topupAmountBySubscriber
. The dialog that opens up upon clicking
on the edit button is shown below:
This aggregate is designed to count the number of calls by subscriber over various
time windows (e.g., currentDay
, lastDay
,
etc.).
topupAmountBySubscriber
, i.e. the aggregate Name
,
is the name that is used to refer to the aggregate when using it in a trigger.
The Aggregationtype
, in this case Sum
, is used to
accumulate the topup amounts, and the group-by attribute, in this case,
MSISDN
, is used to produce per-user tallies, since it
identifies individual subscribers.
Additional group-by attributes can be added in a manner similar to the group-by clause used in traditional relational database management system (RDBMS) query languages.
A filter condition can also be used when only a subset of tuples is to be considered for an aggregation.
For instance, if the incoming tuples in the feed contain a Boolean
isDroppedCall
, indicating a dropped call, then one can
calculate droppedCallsBySubscriber
by using
isDroppedCall
equals to True
as a filter
condition with the MSISDN
as the group-by attribute and
Count
as the aggregation type.
To add a new aggregate, we need to click on Add
button for that feed
application. A dialog with form needs to be filled by filling Aggregate
Name
, Aggregation Type
, Attribute to
Aggregate
, Group by Attribute
and Aggregation
Filter
:
The list of enrichments associated with a feed can be seen by clicking on the
Show
button in the Enrichments
row. The
enrichments associated with the Topup Demo
feed can be seen
below:
HCL Detect supports 4 kinds of enrichments:
- The transformed attributes enrichment enables one to add attributes to the tuple that have either a constant value or a value that can easily be derived from an UEL expression. Such transformed attribute may be needed by other enrichments or retained as an attribute in the enriched tuple. Note that retaining and forwarding the constant attributes as part of the output tuple from the enrichment step is optional.
- The lookup-based enrichment, which retrieves attributes from the profile
store and possibly add them to the outgoing tuple. Note that retaining and
forwarding the looked up attributes as part of the output tuple from the
enrichment step is optional. We have to first select a key attribute present in
the tuple, the associate
Pinpoint
table will be listed inLookup Table
drop-down, finally we have to select the attribute from the lookup table to be looked up. - The aggregate-based enrichment, which fetches data from
FastPast
and possibly adds such attributes to the outgoing tuple. As part of configuring an aggregated attribute, it is necessary to (1) select an aggregate from the list of available aggregates for the feed, (2) select the window length unit, which can beMinute
,Hour
,Day
,Month
orYear
, (3) select the period which can be one ofCurrent
orLast
and (4) optionally selectWindow length
, which can be used to indicate that only a subset of an aggregate is to be used. For instance, when theDay
window length unit and theLast
period withWindow Length
set to 7 is chosen, the aggregated value for the last 7 days can be computed. Note that when using theMinute
window length unit,Window Length
needs to be multiple of 10-minute i.e., backend configured sub-hour buckets length. Similarly to lookup-based enrichment, aggregate-based enrichment attributes may or may not be retained and forwarded as part of the output tuple. - The derived attributes enrichment, which enables the execution of an external Python function. One such a function takes in a tuple (as well as any other required additional parameters, if any), performs a user-defined computation, and produces a result. This function invocation's return value can then be retained and forwarded as part of an outgoing tuple. This capability can be used, for instance, to invoke a scoring function on an incoming tuple, which computes a score (e.g., based on previously mined data) that can then be added as an attribute to the corresponding outgoing tuple. The derived attributes enrichment are basically of three types, i.e., enrichment function based, expression based and scorer function based. expression based takes an UEL expression explained in Miscellaneous » UEL section.
The interface for editing enrichments is shown below:
The interface for editing derived attribute type enrichments is shown below:
Enrichment operations can be configured to before aggregations are performed or after
aggregations are performed. The value of a enriched aggreate-based attribute
may change based on whether it configured to run before or after the aggregation
operation. We can choose this while creating an enrichment by checking the
Post-Aggregation Enrichment
check-bit as shown in the below
image: