Netidee Blog Bild
Scalability and Robustness in IoT Monitoring Systems Using Kubernetes
Exploring Gossip-Based Communication for Efficient Information Exchange (09.07.2024)
Förderjahr 2023 / Stipendien Call #18 / ProjektID: 6747 / Projekt: DEMon

In order to test decentralized communication systems, it is necessary to emulate or simulate the participating nodes. In my work, it is primarily about emulating the systems in order to investigate scalability and other properties.

In order to guarantee the scalability of the systems, a framework was created that automatically generates any number of pods with the desired functionality using Kubernetes based on previously defined configurations. A pod can be seen as an independent IoT device that executes the DEMon prototype. In my experiments with Kubernetes, up to 150 such pods are created, which then exchange information about system properties (such as remaining memory and processor capacities). Each pod has a list of the other pods stored in the system and selects the next communication partner at random. This gossip-based principle of randomness helps to eliminate dependencies on specific pods that rulesets (other algorithms) would entail and ensures that the DEMon prototype is extremely robust to topology changes and failures. The core variables of the configuration are as follows:

  • System size: Depending on how large a system is, the communication variables change accordingly: The time and number of messages until information in systems is intuitively related to the system size itself. Scalability describes how a method performs with different (increasing) system sizes.   
  • Number of conversation partners: As mentioned above, a pod randomly selects another node to communicate its own and additionally stored information from others to this other node. This number can be changed. In future it will be referred to as the gossip count. As intuitively assumed, an increase in the gossip count inevitably leads to an acceleration in the spread of information.  
  • Waiting time between conversations: As the establishment of a new communication with other participants is a cyclical event, the waiting time between messages can be changed here: Shortening it leads to an acceleration and lengthening it decelerates the exchange of information. This waiting time is known as the gossip rate and the number of cycles already completed is known as the round or gossip round.

The question now arises as to how we can capture and quantify the "information" of a system: There are of course different ways to do this. For me, since we are dealing with a monitoring system, the following are the most important:

  • Convergence: We describe a system as converged as soon as all participants have received at least one piece of information from every other participant. (Please note that the exchange of information behaves transitively: If A receives a message from B and then sends a message to C, C now stores information from A as well as from B).   
  • Age of information: Since the monitoring data we record is time-stamped, the "age" of the information can be measured at any time by comparing it with the current system. If we now receive the status (data) of another randomly selected participant, how up-to-date is this data seen via the system? 

Initial Results:   

Impact of gossip count on the time to convergence
Influence of gossip count on the time to convergence
Influence of gossip rate on the time to convergence
Influence of gossip rate on the time to convergence
Influence of gossip count on the number of messages till convergence
Influence of gossip count on the number of messages till convergence
Influence of gossip rate on the number of messages till convergence
Influence of gossip rate on the number of messages till convergence

 

CAPTCHA
Diese Frage dient der Überprüfung, ob Sie ein menschlicher Besucher sind und um automatisierten SPAM zu verhindern.
    Datenschutzinformation
    Der datenschutzrechtliche Verantwortliche (Internet Privatstiftung Austria - Internet Foundation Austria, Österreich) würde gerne mit folgenden Diensten Ihre personenbezogenen Daten verarbeiten. Zur Personalisierung können Technologien wie Cookies, LocalStorage usw. verwendet werden. Dies ist für die Nutzung der Website nicht notwendig, ermöglicht aber eine noch engere Interaktion mit Ihnen. Falls gewünscht, können Sie Ihre Einwilligung jederzeit via unserer Datenschutzerklärung anpassen oder widerrufen.