The Ledger Services of the DDA Infrastructure
The DDA Infrastructure on Edge Gateways – will rely on Ledger Services exposed by the Distributed Ledger enabler. Ledger Services play a key role in configuring DDA processes, by means of Blockchain “smart contracts”. In particular:
- DDA configurations will be represented as smart contracts. Each smart contract will configure one or (usually) more analytics instances, through the Distributed Ledger. In particular, the Distributed Ledger infrastructure will manage smart contracts, while using them to configure multiple instances. Hence, by managing multiple smart contracts, the Distributed Ledger infrastructure will be able to keep track of multiple analytics configurations. The latter configurations will be executed by the DDA to support analytics for various production processes that are typically executed simultaneously in the scope of the factory and space multiple edge nodes (e.g., multiple stations across the factory).
- The Publishing Services of the Distributed Ledger will have a direct interaction with Edge Analytics Orchestrator, in order to implement analytics tasks that span distributed analytics functions placed across multiple EGs (Edge Gateways).
Edge Analytics Engine (EAE)
The EAE engine is a runtime environment hosted in an EG i.e. at the edge of a FAR-EDGE deployment. It is the programmable and configurable environment that executes data analytics logic locally in order to meet stringent performance requirements, mainly in terms of latency.
While the Ledger Services are in charge of managing smart contracts and executing distributed analytics across EGs, the EAE is in charge of data analytics within a single EG. The EAE is also configurable while comprising multiple analytics instances that are driven by multiple smart contracts. It consists of the following main components:
- the EA-Orchestrator;
- the EA-Processor;
- the Local EA-Repository,
EA-Orchestrator
The EA-Orchestrator provides the run-time environment that controls and executes edge analytics instances, which are specified in a format that is conveniently called EaSpec (i.e. which stands for Edge Analytics Specification). In particular, the EA-Orchestrator is able to parse and execute analytics functions and rules specified in an EaSpec.
The following statements define the EA-Orchestrator main operation:
- An EaSpec defines a set of edge analytics functionalities, as a graph of processing functions, which can be executed by the EA-Processor.
- The EA-Orchestrator parses EaSpaces and executes the analytics functions that they comprise.
- The EA-Orchestrator is able to concurrently execute multiple analytics instances (specified in corresponding EaSpecs).
From an implementation perspective, EaSpecs can be represented in different forms such as:
- A configuration file or an entry in a database. This is the implementation approach adopted in this first release of the EAE.
- A part of a smart contract in the blockchain. This is the representation that will be implemented as part of the final version of EAE in deliverable D3.6. In latter deliverable analytics functions spanning multiple Edge Nodes will be specified in smart contracts and executed based on the Ledger Services.
Specifically, the EaSpec includes the information needed to drive the operation of the EA-Orchestration, including for example the attributes and sequences needed to setup the required jobs on the EA-Processor.
EA-Processor
The EA-Processor implements the data processing functionalities that are necessary to implement an edge analytics task. These functionalities include:
- Pre-processor, which prepares data stream for analysis, based on the specifications of the target analytics tasks. A pre-processor interacts with a Data Bus in order to acquire streaming data from the field. At the same time, it also produces and registers new streams in the same Data Bus.
- Analytics Processor, which applies analytics algorithms to one or more data streams. Similar to the pre-processor, the analytics processor consumes and produces data through interaction with the Data Bus.
- Store Processors, which is used to store streams to repositories.
Pre-processors, analytics processors and store processors define three different types of functionalities that are supported by the EAE i.e. pre-processing, analytics and storage of streams. Given these processor types, a specific instance of Edge Analytics (EA) is implemented by setting up multiple processors, which are connected in a graph-like fashion thus forming a topology. The topology is specified in the EaSpec, which will be represented as a smart contact. Moreover, the topology and overall process are controlled by the EA orchestrator.
SIMPLE EXAMPLE
The Figure illustrates an example topology and runtime operations for EA Processor. In this example, two streams (CPS1 and CPS2) are pre-processed from Processor Job 1 (i.e. Pre-Processor) and Processor Job 2 (i.e. Pre-Processor) equivalently in order for an analytics algorithm (i.e. Processor Job 3) (i.e. Analytics Processor) to be applied to them. Finally, the result needs to be stored in a Data Storage with the help of Processor Job 4 (i.e. Storage Processor). The setup and runtime operation of the EA-Processor entails the following steps:
- Step1 (Set-up): Based on the description of the topology and required processors in the EaSpec, the EA-Orchestrator instantiates and configures the required Processor jobs.
- Step2 (Runtime): Processor Job 1 consumes and pre-processes streams coming from CPS1. Likewise, Processor Job 2 consumes and pre-processes streams coming from CPS2.
- Step3 (Runtime): Analytics Processor Job 3 consumes the produced streams from Processor Job 1 and 2 for applying the analytics algorithm.
- Step4 (Runtime): Store Processor Job 4 consumes the data stream produced from Processor Job 3 and forwards it to the Data Storage.
- Step5 (Runtime): Data Storage persists the Data coming from Store Processor Job 4.