Förderjahr 2021 / Projekt Call #16 / ProjektID: 5899 / Projekt: PrototAIp
We have now identified the requirements and specified the software architecture.
We decided on a notebook interface as the programming environment. The most widely used notebook software with the largest user community is Jupyter notebook. The goal behind Project Jupyter is "develop open-source software, open-standards, and services for interactive computing across dozens of programming languages" - a perfect match for our project with similar ideas. Other notebook solutions are the open-source project Apache Zeppelin or customized commercial versions of Jupyter Notebook by major cloud providers Amazon AWS, Google and Microsoft Azure.
However, a Jupyter Notebook is only intended for one user at a time. Fortunately, Project Jupyter offers a solution for this as well: JupyterHub. It serves as a central hub for the users and provides each user with a notebook. There are two versions available: a base version called The Littlest Jupyter Hub (TLJH) and the more advanced version Zero to JupyterHub (Z2JH). TLJH is easier to use and less complex in design, but can only serve a maximum of 100 users. However, we want to be scalable and not have to limit ourselves to a certain number of users. For this reason, we have chosen Z2SH as the backbone of our platform.
As soon as a user signs in at the platform, a virtual instance of a Jupyter notebook is started and assigned to the user. And now back to another principle of our project: Batteries included - our notebooks will be preloaded with the most important libraries for data scientists, so that the students can start right away without long waiting time. You can find out how we get the list of the most important libraries in the last blog post.
Another principle is not to create a finished monolith software solution, but also to give as much back to the community as possible. For this reason, we design all parts of our software to be reusable. An example are the mentioned preloaded notebooks. Project Jupyter already offers a selection here (see below).
We will develop a new notebook template, the prototAIp-notebook, built on the already existing notebooks and we will share it directly with the Jupyter community and everyone else. You can already find the first version on our GitHub page and our Dockerhub. It is continuously expanded, so make sure to check back regularly.