Abstract:

With the revolution of high-speed networks and immersive technologies, the concept of metaverse has been proposed and it has the potential to become the next generation of Internet.

We could imagine that some of the virtual humans we interact with in the metaverse could possibly be generated by computer algorithms. However, the current technology is not capable of generating lifelike virtual humans, especially when interacting with users is required. Therefore, in order to improve the plausibility of the environments that are inhabited by non-interactive characters, we developed a system that enables an actor to take over control of multiple avatars when interaction with a real user is required. In our system, the actor is able to jump between multiple avatars in a virtual cafe scenario, acting in the roles of bartender and customers.

To evaluate our takeover system, we conducted an interaction experiment and a usability test. In the experiment, we assigned tasks to the participants and investigated their social presence in each situation through questionnaires and interviews. The results verified that the avatars in our system were more plausible than the ones with pre-recorded actions and reactions. In the usability test, the time taken by novices to take over the avatar was recorded. The results show that the time required to take over decreases with practice. All users are able to take over an avatar within 35s after practice. The shortest time to take over is 5.59s.

Developed to a paper submitted to ISMAR 2023