Data Management Strategies for Near Real-Time Edge Analytics

Developing an Edge Data Management framework (EDMframe)

Ivan Lujic

Data Management Strategies for Near Real-Time Edge Analytics

Förderjahr 2018 / Stipendien Call #13 / Stipendien ID: 3793

Internet of things (IoT) sensors are used in wide set of applications, such as smart cities, eHealth monitoring or intelligent traffic management systems. As a consequence, massive amounts of data are generated continuously from a growing number of IoT sensors. Decision-making processes depend on information obtained through the analysis of collected data. Traditionally, managing such systems includes data processing in the cloud. However, performing data analytics in cloud data centers brings serious challenges including the transfer of the astoundingly large amounts of sensor data over the Internet to the cloud and strict latency and accuracy requirements of IoT applications. To overcome aforementioned challenges, we investigate strategies of adaptive data management for near real-time data analytics performed on edge nodes that are closer to the source of data.

Uni | FH [Universität]

Technische Universität Wien

Themengebiet

Big Data

,

IoT

Zielgruppe

Systemintegratoren

,

Techniker:innen

Gesamtklassifikation

Dissertation | PhD

,

Studie|Konzept

Technologie

Big Data

,

Cloud Service

,

R

,

Sensorik

Lizenz

CC-BY

Projektergebnisse

Dissertation | PhD CC-BY

Dissertation mit dem Titel "Foundations for Sustainable and Trustworthy Edge Data Analytics" von Ivan Lujic an der TU Wien.

Diss_Lujic_Ivan_preprint.pdf
15.16 MB

Endbericht CC-BY

Endbericht des noch nicht abgeschlossenen Dissertationsprojektes

stip3793_Call13_Endbericht_V01.pdf
1.02 MB

Summary CC-BY

Zusammenfassung des Endberichts

stip3793_Call13_Zusammenfassung_V01.pdf
497.81 KB

Zwischenbericht CC-BY

Stipendium-Zwischenbericht (midterm report) May 2019

stip3793_Call13_Zwischenbericht_V01_0.pdf
770.38 KB

Paper

Der Konferenzbeitrag für die ECSA 2019 mit dem Titel "Architecturing Elastic Edge Storage Services for Data-Driven Decision Making" ist unter Springer-Copyright in Lecture Notes in Computer Science (LNCS, volume 11681) Buchreihe veröffentlicht. Zusätzlich dazu findet ihr hier die aktualisierte Konferenz-Präsentation.

2019-09 ECSA19 Paris - Lujic2019Architecturing.pdf
891.77 KB

Paper

Der Artikel "Resilient Edge Data Management Framework" wird in der zukünftigen Ausgabe der wissenschaftlichen Fachzeitschrift "IEEE Transactions on Services Computing" veröffentlicht. Derzeit ist der Artikel unter IEEE-Copyright als Early Access Beitrag verfügbar. Hier wird eine pre-print Version veröffentlicht.

TSC_Resilient Edge Data Management Framework_0.pdf
10.18 MB

Code MIT

A prototype implementation of edge data management framework components

https://github.com/lujic/EDMFrame

Blogbeiträge

Netidee Blog Bild

This blog introduces a high-level architecture model overview of the SAPLaw system, aiming to illustrate the main components and steps for achieving self-adaptive analytics placement at the edge.

(15.04.2020 )
Self-adaptive Analytics Placement – Architecture Overview

Netidee Blog Bild

This blog introduces a use case for self-adaptive placements of data analytics in the edge-cloud environment based on data locality, aiming to efficiently place the analysis where data is located.

(20.03.2020 )
Self-adaptive Analytics Placement - Use Case

Netidee Blog Bild

This blog introduces a testbed for experimentally evaluating edge data management strategies, aiming to have a better overview of needed technologies and physical resources.

(15.02.2020 )
A Testbed Setup for Edge Data Management Strategies

Netidee Blog Bild

Paper "Resilient Edge Data Management Framework" has been accepted for publication in IEEE Transactions on Services Computing. This blog post gives an overview of what the most significant outcomes of the evaluation were.

(15.01.2020 )
Contributions accepted for publication in IEEE Transactions on Services Computing

Netidee Blog Bild

This blog post introduces further methodology and environment for implementing and realizing some of the proposed edge data management strategies, aiming to provide their practical applicability besides theoretical contribution.

(15.12.2019 )
Practical applicability of EDM strategies

Netidee Blog Bild

EDMFrame is designed as a service built on top of existing IoT data processing services to enhance data recovery and analytics features.

(15.11.2019 )
Edge Data Management Solutions and Limitations

Netidee Blog Bild

Developing intuitive systems where we must make near-real-time decisions based on sensor data while dealing with limited space and time for data processing is a serious challenge facing the future of IoT.

(15.10.2019 )
Intuitive and Proactive IoT Systems

Netidee Blog Bild

The contribution and results of recent research are presented at the 13th European Conference on Software Architecture (ECSA). In this blog post, one can find a short tooling overview for proposed engineering principles.

(15.09.2019 )
Impact on Architectural Design of Elastic Edge Storage Services

Netidee Blog Bild

Many critical data analytics tasks nowadays require data processing near the infrastructure devices. We aim to investigate the benefits of the symbolic data conversion as a support for edge analytics, storage and transmission bandwidth scarcity.

(15.08.2019 )
From the Raw Time Series to a Symbolic Data Representation

Netidee Blog Bild

Exceeding network capacity with excessive and incomplete data might cause serious challenges including degradation of QoS and slowing down the response time in critical applications such as smart grids, connected cars or eHealth.

(15.07.2019 )
Dealing with Approximate Data Representation and Analytics

A way of combining inputs, such as application information and data workload characteristics, must enable dynamic, software-defined components for the overall edge storage management.

(15.06.2019 )
Engineering Principles for Edge Data Services

To achieve optimized design of elastic edge storage services, requirements coming from edge data and system characterization, application-specific analytics context, and edging system operations, should be considered.

(15.05.2019 )
Investigating Elasticity for Edge Storage Services

In the IoT, a massive number of smart devices produce a variety of data at unprecedented scale. The next generation edge storage service must make sure that the edge analytics is always served with relevant and suitable data.

(15.04.2019 )
Need for Elastic Edge Data Services

One of the solutions for efficient and automatic edge data recovery of different gaps might include pre-calculated projection recovery maps (PRMs) that recommend techniques and needed ranges of data for detected gaps.

(17.03.2019 )
Towards Self-adaptive Technique Selection for Edge Data Recovery

To timely manage accurate and automatic decisions, future management of IoT systems must deal with incomplete data, the big data volume and limited capacity of storage resources, imposing a solution for efficient data management, called EDMFrame.

(18.02.2019 )
From raw sensor data to smart actuator decisions

Netidee Blog Bild

Although, there is a number of studies exploring emerging edge computing technology, the state-of-the-art still lacks contributions in efficient data management strategies for near real-time edge analytics.

(20.01.2019 )
State-of-the-art

Netidee Blog Bild

While cloud computing offers dynamic and scalable resources, edge computing pushes data processing to the edge of the network, in proximity of data sources. However, many requirements and limitations must be considered for reliable data analytics.

(26.12.2018 )
Finding methodologies for overcoming emerging challenges

Netidee Blog Bild

Movement of data analytics from distant cloud data centers to local edge nodes requires reliable data management strategies with support from the IoT gathering layer.

(30.11.2018 )
Moving to the edge...

Ähnliche Stipendien

Building Knowledge Graphs for Under Resourced Languages

Building Knowledge Graphs for Under Resourced Languages

Ziel meiner Dissertation ist es, Internet Standards sichtbar zu machen, indem ich messe, wie sie von Betreibern implementiert werden und ob dies auf einer sicheren Art und Weise geschieht.

Measuring the Gap between Standardization, Implementation and Adoption of Internet Standards

Picture of a cellular base station

This thesis focuses on active measurements in live cellular networks to improve both security (e.g., protection against hackers) and transparency for customers and the public.

Active Measurements in Cellular Networks

Mai AL-Zu'bi

Real-Time Detection of Side-Channel Attacks Using Machine Learning and Micro-Architectural Traces

Attack Detection using Micro-architectural Traces and Machine Learning

Bild aus einem eduLAB Workshop, man sieht Kinder die Ligretto Karten sortieren.

Schon einmal einen Algorithmus entwickelt und angewendet? Wahrscheinlich ja – und das vielleicht ohne es zu merken! Diese Arbeit erforscht, wie intuitive algorithmische Problemlösestrategien gezielt gefördert werden können.

Algorithmic Problem Solving in Unplugged Computer Science Outreach Activities