Generative Virtual Worlds

Program Description

The most promising Extended Reality (XR) applications—such as Virtual Reality exposure therapy for mental health, motor learning in physical rehabilitation, or remote collaboration through augmented avatars—depend on the ability to create personalized and rapidly generated XR content. Currently, we are witnessing a major shift from manual to automated programming of XR environments driven by the integration of generative AI, which enables the creation of those personalized environments. For example, De La Torre et al. demonstrated real-time prompting of interactive worlds, and Generio 1 allows for highly customizable near-instant object generation via natural language prompts.

Because of this rapid technical progress, development often occurs in isolated silos, with a strong focus on technical innovation but limited attention to broader implications. Key challenges include: How do we ensure scalability, control, and customization of generated content? What are effective modalities for interacting with generative XR, and how should an AI be represented in collaborative XR environments? What are the ethical, societal, and environmental implications of generative XR?

Currently, there is a lack of guidelines and frameworks that synthesize the potential of generative XR while addressing these implications on research practices and society. We propose the workshop series “Generating Virtual Worlds” with the goal of building an international community around these open questions.

The aim is to investigate the following objectives: (1) to gather and critically evaluate tools and platforms for generative XR development, (2) define guidelines for interacting with generated virtual worlds, and (3) identify key areas concerning the societal implications of generative XR.

The envisioned outcomes are: (a) establish an international community focusing on implications of generative XR with a strong lead at the Pioneer Centre for AI, (b) collect, develop, and share tools and prototypes for research and teaching, and (c) disseminate a comprehensive vision paper that articulates the opportunities, challenges, and strategic directions for the future of generative XR.