Feed Applications Models
Each feed application have to configure application model and can optionally configure some other feed application model file as described below.
feed application models are configured in
etc/models/applications/<feed-name>/
.
below is the example of a solution containing a feed application definition:
$ tree etc/model/applications/ericsson_usage/
etc/model/applications/ericsson_usage/
├── aggregations.json
├── application.json
├── enrichment_functions.json
└── enrichments.json
The name of feed application is Ericsson Usage .
Below sub-sections will explain each of the above files.
Application Model
Application model defines the key attribute and feed data model that this feed implements, and it is defined in application.json file.
Example application Model:
{"feedApplication":{"dataModels":["Identity","Usage"],"keyAttribute":"MSISDN","name":"Ericsson Usage"}}
dataModels
a list of feed data model that are implemented by this feed.keyAttribute
an attribute from the feed that is a key attribute.name
is name of feed application.
Aggregations
This model describes the aggregations that are computed on a feed and stored in
FastPast. Aggregates are configured in
etc/model/applications/<application_name>/aggregation.json
file.
Example Aggregation definition is as below:
{"aggregations":[{"name":"avgCallDuration","operation":{"aggregationKind":"Average","groupByAttributes":["callingNumber"],"valueAttribute":"duration"}},{"name":"numCallsByCell","operation":{"aggregationKind":"Sum","groupByAttributes":["cellId"],"value":1.0}},....]}
aggregations
List of aggregations.name
A name (aggregations defined on different feeds can have the same name) that is in the form of an identifier (aNameLikeThis).groupByAttributes
A potentially empty list of group by attributes.aggregationKind
An aggregation kind (Average, Maximum, Minimum, Sum, Variance).valueAttribute
orvalue
A value (such as "1" for counting) or value attribute (such as "callDuration").tupleFilter
An optional tupleFilter, which is a Boolean UEL expression
Loaded once during Detect initialization, managed from the UI there on.
Enirchment Functions
Enrichment Function model file defines the meta data of an enrichment function. It defines the signature of a python function to be used in an enricher.
Enrichment Function definition is configured in
etc/model/applications/<application_name>/enrichment_functions.json
file.
Example Enrichment Function:
{"functions":[{"module":"acme.application_helpers.ericsson_usage.enrichment_helpers","name":"device_change","parameters":[{"name":"deviceName","required":false,"type":"String"},{"name":"isSmartphone","required":false,"type":"Bool"}],"type":"Bool","usedAttributes":[{"name":"IMEI","required":true,"type":"String"},{"name":"lastIMEI","required":true,"type":"String"}]},....]}
signature of python funciton is as below:
defdevice_change_(tuple_,deviceName=None,isSmartphone=None):"""Returns the last time the subscriber has changed their device"""imei=tuple_.IMEIlastIMEI=tuple_.lastIMEI....returnxyz
functions
list of functions defined.module
a python module path.name
name of enrichment function.type
return type of the function.parameters
list of additional parameters to enrichment function, tuple is a default and first parameter to function. we need to only mention any additional parameters/usedAttributes
the list of attributes used from tuple. This is required for Detect to know if a particular function can be applied on a feed or not.
Enrichments
This model describes the enrichments that are performed on a feed, whose results
may be stored in PinPoint. Enrichments are configured in
etc/model/applications/<application_name>/enrichments.json
.
It has list of enrichments, where each enrichment has: * A name (enrichments
defined on different feeds can have the same name)
- A list of transformed attributes
- A list of lookup attributes
- A list of aggregated attributes
- A list of derived attributes
There are two kinds of enrichments: Pre-aggregation are list of enrichments that are applied before aggregation takes place. Post-aggregation are list of enrichments that are applied after aggregation takes place.
This file is loaded once during Drive initialization, managed from the UI there on.
Enrichment structure is as below:
{"preAggregationEnrichments":[{"name":"IMEIChangeEnrichmentPre","operation":{"derivedAttributes":[....],"aggregatedAttribute":[....],"lookupAttribute":[....],"transformedAttributes":[....]}},{..}],"postAggregationEnrichments":[{"name":"IMEIChangeEnrichmentPost","operation":{"derivedAttributes":[....],"aggregatedAttribute":[....],"lookupAttribute":[....],"transformedAttributes":[....]}},{..}]}
Its an UEL expression to derive a value for a new attribute.
Example:
{"expression":"\"FOO\"","name":"identity","retained":false,"type":"String"}
expression
UEL expression for to derive value.name
name of enriched attribute.retained
flag to retain this new attribute in the tuple for next operator in the flow.type
is the data type of this attribute.
This is used to lookup from a profile.
Example:
{"name":"licenseId","retained":false,"tableAttribute":{"keyAttribute":"identity","name":"licenseId","table":"CUSTOMER"},"type":"String"}
tableAttribute
is to define the table and attribute to lookup.tableAttribute.keyAttribute
is key attribute in the tuple which will be used to perform lookup from table.tableAttribute.name
is the name attribute from table to be looked up.tableAttribute.table
is the table name form pinpoint to be looked up.type
is the data type of the enrichmented attribute.
This enrichment attribute gets value by fetching a aggregate from FastPast.
Example:
{"aggregate":"sumOfTransactionAmountByCustomerAndTransactionType","groupByAttributes":["customerId","transactionType"],"name":"transactionAmountInLast7Days","period":"Last","retained":true,"type":"Double","windowLength":7,"windowLengthUnit":"Day"}
aggregate
is the name of aggregate in the FastPast.groupByAttributes
is a list of attributes to be used for passing for groupby attributes.name
is the name of enriched attribute.period
is the period for FastPast query. e.g. Current, Last or AllTime.windowLength
is length of window for a given unit.windowLengthUnit
is the unit for which query to be made. e.g., Day, Month, Year, Minute, Hour
The derived attributes enrichment enables the execution of an external Python function. One such a function takes in a tuple (as well as any other required additional parameters, if any), performs a user-defined computation, and produces a result. This function invocation's return value can then be retained and forwarded as part of an outgoing tuple.
Example:
{"function":{"module":"acme.application_helpers.ericsson_usage.enrichment_helpers","name":"device_change"},"name":"deviceChanged","retained":true,"type":"Bool"},{"function":{"module":"acme.application_helpers.ericsson_usage.enrichment_helpers","name":"device_change_time"},"name":"deviceChangeTime","retained":true,"storeBackAttribute":{"keyAttribute":"callingNumber","name":"lastIMEIModificationTime","table":"CUSTOMER_PROFILE"},"type":"Int64"},
function
This refers to enrichment function model discussed in previous sub section.storeBackAttribute
the derived attribute can optionally stored back to profile in pinpoint.