We began developing ARTverse in close collaboration with artists, aiming to create tools they could use to produce, teach, and exhibit art within social VR platforms. Our initial focus was on the development of creative tools. In the process of developing these tools, our attempts to explore VR as a creative medium faced significant challenges. Many features envisioned by our collaborators could not be implemented due to technical limitations of the available platforms.

Challenges in VRChat SDK

The primary limitation of developing creative tools using the VRChat SDK is that objects and materials must be preloaded with the scene. This restriction prevents the creation of new materials or objects during runtime.

3D Drawing Tools: Our first approach was to replicate features found in existing 3D drawing apps within VRChat. While VRChat's starting room includes a basic 3D pencil and color wheel, expanding this tool was highly constrained. For instance: Every line with a different color or weight requires a separate material in the Unity scene. Since the VRChat SDK does not support the creation of materials at runtime, implementing features such as dynamic line styles and colors required an impractical number of preloaded materials.

Modeling Tools: We also explored developing a modeling tool to manipulate volumes within the scene. While scaling objects along three axes is straightforward, the SDK does not support runtime mesh editing. As a result, creating dynamic shape distortions was impossible.

Multimedia Integration: Artists expressed a strong desire to mix multiple media formats—images, videos, and drawings—within a single installation. This approach is common in digital art. To address this, we attempted to build a tool that could search and load images from an external database, such as Europeana. This would allow artists to create collages or thematic exhibitions in real time. However, the VRChat SDK prohibits API calls. Although it is technically possible to load images from external sources, the links must be hosted in public repositories (e.g., GitHub), which greatly limits flexibility.

Saving Changes in Runtime: Another significant limitation was the inability to save changes made during runtime. While simple text files (e.g., for game stats) can be saved, the SDK does not support storing more complex data structures. This means that any created or edited art pieces could not be preserved for future sessions.

Implemented Features

In the end, we created a basic scene with several features, based in VRChat existing tools from tutorials and community assets improved by us. Ideally, an artist could duplicate an instance of our scene and reuse its features to generate new artwork. However, due to the inability to save changes within the scene, it currently serves more as a demo scene for the features that we developed. That said, it is possible to clone the Unity project from the Git Repository, make some adjustments, and use the features to create a VR installation for a VRChat room. 

List of implemented creative features in ARTverse

It's important to note that this is not a critique of VRChat. Many of the challenges we encountered while using the VRChat SDK would have arisen on any other platform. In general, these platforms allow the replication of rooms but content is typically created in desktop applications with limited options for online or in-VR changes.