Is High-Fidelity Important for Human-like Virtual Avatars in Human Computer Interactions?

Qiongdan Cao; Hui Yu; Paul Charisse; Si Qiao; Brett Stevens

doi:10.53941/ijndi0201008

Abstract

As virtual avatars have become increasingly popular in recent years, current needs indicate that “interactivity” is crucial for inducing a positive response from users towards these avatars, especially in human computer interactions (HCI). This paper reviews recent works on high-fidelity human-like virtual avatars (e.g. high visual and motion fidelity) and discusses the critical question——“Is high-fidelity a positive choice for virtual avatars to achieve better interactions in HCI”. Furthermore, we summarise current technical approaches to developing those virtual avatars. We investigate the advantages and disadvantages of high-fidelity virtual avatars in different areas focusing on addressing the effect of motion, especially the upper body. Research shows that high-fidelity is a positive choice for virtual avatars, although it may depend on the application.

References

1.
Autodesk. Arnoldrenderer Research. Available online: https://arnoldrenderer.com/research/ (accessed on 10 December 2022).
2.
Park, S.W.; Park, H.S.; Kim, J. H.; et al. 3D displacement measurement model for health monitoring of structures using a motion capture system. Measurement, 2015, 59: 352−362.
3.
Zibrek, K.; Kokkinara, E.; McDonnell, R. The effect of realistic appearance of virtual characters in immersive environments-does the character's personality play a role? IEEE Trans. Vis. Comput. Graph., 2018, 24: 1681−1690.
4.
Jo, D.; Kim, K.H.; Kim, G.J. Effects of avatar and background representation forms to co-presence in mixed reality (MR) tele-conference systems. In SIGGRAPH ASIA 2016 Virtual Reality Meets Physical Reality: Modelling and Simulating Virtual Humans and Environments, Macau, China, 5–8 December 2016; ACM: Macau, China, 2016; pp. 12. doi:10.1145/2992138.2992146
5.
Latoschik, M.E.; Roth, D.; Gall, D.; et al. The effect of avatar realism in immersive social virtual realities. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology, Gothenburg, Sweden, 8–10 November 2017; ACM: Gothenburg, 2017; pp. 39. doi:10.1145/3139131.3139156
6.
Garau, M. The Impact of Avatar Fidelity on Social Interaction in Virtual Environments. Ph.D. Thesis, University College London, London, UK, 2003.
7.
Mori, M.; MacDorman, K.F.; Kageki, N. The uncanny valley [from the field]. IEEE Robot. Autom. Mag., 2012, 19: 98−100.
8.
Hanson, D.; Olney, A.; Prilliman, S.; et al. Upending the uncanny valley. In Proceedings of the 20th National Conference on Artificial Intelligence, Pittsburgh, Pennsylvania, USA, 9–13 July 2005; AAAI Press: Pittsburgh, 2005; pp. 1728–1729.
9.
Geller, T. Overcoming the uncanny valley. IEEE Comput. Graph. Appl., 2008, 28: 11−17.
10.
Brenton, H.; Gillies, M.; Ballin, D.; et al. The uncanny valley: Does it exist. In Proceedings of the 11th Conference of Human Computer Interaction, Workshop on Human Animated Character Interaction, Las Vegas, NV, USA, Lawrence Erlbaum Associates: Las Vegas, 2005.
11.
Volante, M.; Babu, S.V.; Chaturvedi, H.; et al. Effects of virtual human appearance fidelity on emotion contagion in affective inter-personal simulations. IEEE Trans. Vis. Comput. Graph., 2016, 22: 1326−1335.
12.
Ries, B.; Interrante, V.; Kaeding, M.; et al. Analyzing the effect of a virtual avatar's geometric and motion fidelity on ego-centric spatial perception in immersive virtual environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, Kyoto, Japan, 18–20 November 2009; ACM: Kyoto, 2009; pp. 59–66. doi:10.1145/1643928.1643943
13.
BANDAI NAMCO. Elden Ring. FromSoftware, Inc. 2022. Available online: https://en.bandainamcoent.eu/elden-ring/elden-ring (accessed on 4 July 2022).
14.
Cao, C.; Agrawal, V.; De La Torre, F.; et al. Real-time 3D neural facial animation from binocular video. ACM Trans. Graph., 2021, 40: 87.
15.
Yu, H.; Liu, H.H. Regression-based facial expression optimization. IEEE Trans. Hum. Mach. Syst., 2014, 44: 386−394.
16.
Wang, L.Z.; Chen, Z.Y.; Yu, T.; et al. FaceVerse: A fine-grained and detail-controllable 3D face morphable model from a hybrid dataset. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 20220; IEEE: New Orleans, 2022; pp. 20301–20310. doi:10.1109/CVPR52688.2022.01969
17.
Zhang, S.; Yu, H.; Wang, T.; et al. Linearly augmented real-time 4D expressional face capture. Inf. Sci., 2021, 545: 331−343.
18.
Zall, R.; Kangavari, M.R. Comparative analytical survey on cognitive agents with emotional intelligence. Cognit. Comput., 2022, 14: 1223−1246.
19.
D’Avella, S.; Camacho-Gonzalez, G.; Tripicchio, P. On Multi-Agent Cognitive Cooperation: Can virtual agents behave like humans? Neurocomputing, 2022, 480: 27−38.
20.
Thompson, J.; White, S.; Chapman, S. Interactive clinical avatar use in pharmacist preregistration training: Design and review. J. Med. Internet Res., 2020, 22: e17146.
21.
Sinatra, A.M.; Pollard, K.A.; Files, B.T.; et al. Social fidelity in virtual agents: Impacts on presence and learning. Comput. Hum. Behav., 2021, 114: 106562.
22.
Machidon, O.M.; Duguleana, M.; Carrozzino, M. Virtual humans in cultural heritage ICT applications: A review. J. Cult. Herit., 2018, 33: 249−260.
23.
Foni, A.E.; Papagiannakis, G.; Magnenat-Thalmann, N. Virtual Hagia Sophia: Restitution, visualization and virtual life simulation. In UNESCO World Heritage Congress (Vol. 2); 2002.
24.
Stefanova, M.; Pillan, M.; Gallace, A. Influence of realistic virtual environments and humanlike avatars on patients with social phobia. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 17–19 August 2021; ASME, 2021; pp. V002T02A081. doi:10.1115/DETC2021-70265
25.
Mousas, C.; Koilias, A.; Rekabdar, B.; et al. Toward understanding the effects of virtual character appearance on avoidance movement behavior. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), Lisboa, Portugal, 27 March 2021–1 April 2021; IEEE: Lisboa, 2021; pp. 40–49. doi:10.1109/VR50410.2021.00024
26.
Hube, N.; Angerbauer, K.; Pohlandt, D.; et al. VR collaboration in large companies: An interview study on the role of avatars. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Bari, Italy, 4–8 October 2021; IEEE: Bari, 2021; pp. 139–144. doi:10.1109/ISMAR-Adjunct54149.2021.00037
27.
Waltemate, T.; Gall, D.; Roth, D.; et al. The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response. IEEE Trans. Vis. Comput. Graph., 2018, 24: 1643−1652.
28.
Gochfeld, D.; Benzing, K.; Laibson, K.; et al. Avatar selection for live performance in virtual reality: A case study. In 2019 IEEE Games, Entertainment, Media Conference (GEM), New Haven, CT, USA, 18–21 June 2019; IEEE: New Haven, 2019; pp. 1–5. doi:10.1109/GEM.2019.8811548
29.
Ma, F.; Pan, X.N. Visual fidelity effects on expressive self-avatar in virtual reality: First impressions matter. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Christchurch, New Zealand, 12–16 March 2022; IEEE: Christchurch, 2022; pp. 57–65. doi:10.1109/VR51125.2022.00023
30.
Herrera, F.; Oh, S.Y.; Bailenson, J.N. Effect of behavioral realism on social interactions inside collaborative virtual environments. Presence: Teleoperat. Virtual Environ., 2018, 27: 163−182.
31.
Rogers, S.L.; Broadbent, R.; Brown, J.; et al. Realistic motion avatars are the future for social interaction in virtual reality. Front. Virtual Real., 2022, 2: 750729.
32.
Tinwell, A.; Grimshaw, M.; Nabi, D.A.; et al. Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput. Hum. Behav., 2011, 27: 741−749.
33.
McDonnell, R.; Larkin, M.; Hernández, B.; et al. Eye-catching crowds: Saliency based selective variation. ACM Trans. Graph., 2009, 28: 55.
34.
Jack, R.E.; Garrod, O.G.B.; Yu, H.; et al. Facial expressions of emotion are not culturally universal. Proc. Natl. Acad. Sci. USA, 2012, 109: 7241−7244.
35.
Gonzalez-Franco, M.; Steed, A.; Hoogendyk, S.; et al. Using facial animation to increase the enfacement illusion and avatar self-identification. IEEE Trans. Vis. Comput. Graph., 2020, 26: 2023−2029.
36.
Bai, Z.C.; Yao, N.M.; Mishra, N.; et al. Enhancing emotional experience by building emotional virtual characters in VR volleyball games. Comput. Animat. Virtual Worlds, 2021, 32: e2008.
37.
Yun, C.; Deng, Z.G.; Hiscock, M. Can local avatars satisfy a global audience? A case study of high-fidelity 3D facial avatar animation in subject identification and emotion perception by US and international groups Comput. Entertain., 2009, 7: 21.
38.
Hodgins, J.; Jörg, S.; O'Sullivan, C.; et al. The saliency of anomalies in animated human characters. ACM Trans. Appl. Percept., 2010, 7: 22.
39.
Ruhland, K.; Zibrek, K.; McDonnell, R. Perception of personality through eye gaze of realistic and cartoon models. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception, Tübingen, Germany, 13–14 September 2015; ACM: Tübingen, 2015; pp. 19–23. doi:10.1145/2804408.2804424
40.
Ruhland, K.; Peters, C.E.; Andrist, S.; et al. A review of eye gaze in virtual agents, social robotics and HCI: Behaviour generation, user interaction and perception. Comput. Graph. Forum, 2015, 34: 299−326.
41.
Kokkinara, E.; McDonnell, R. Animation realism affects perceived character appeal of a self-virtual face. In Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games, Paris, France, 16–18 November 2015; ACM: Paris, 2015; pp. 221–226. doi:10.1145/2822013.2822035
42.
Cao, Q.D.; Yu, H.; Nduka, C. Perception of head motion effect on emotional facial expression in virtual reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, 22–26 March 2020; IEEE: Atlanta, 2020; pp. 750–751. doi:10.1109/VRW50115.2020.00226
43.
Garau, M.; Slater, M.; Vinayagamoorthy, V.; et al. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA, 5–10 April 2003; ACM: Ft. Lauderdale, 2003; pp. 529–536. doi:10.1145/642611.642703
44.
Petrović, V.M. Artificial intelligence and virtual worlds-toward human-level AI agents. IEEE Access, 2018, 6: 39976−39988.
45.
Norouzi, N.; Kim, K.; Hochreiter, J.; et al. A systematic survey of 15 years of user studies published in the intelligent virtual agents conference. In Proceedings of the 18th International Conference on Intelligent Virtual Agents, Sydney, Australia, 5–8 November 2018; ACM: Sydney, 2018; pp. 17–22. doi:10.1145/3267851.3267901
46.
Luerssen, M.H.; Hawke, T. Virtual agents as a service: Applications in healthcare. In Proceedings of the 18th International Conference on Intelligent Virtual Agents, Sydney, Australia, 5–8 November 2018; ACM: Sydney, 2018; pp. 107–112. doi:10.1145/3267851.3267858
47.
Alicja, K.; Maciej, M. Can AI see bias in X-ray images. Int. J. Network Dyn. Intell., 2022, 1: 48−64.
48.
Jiang, F.; Jiang, Y.; Zhi, H.; et al. Artificial intelligence in healthcare: Past, present and future. Stroke Vasc. Neurol., 2017, 2: 230−243.
49.
Zhao, G.Y.; Li, Y.T.; Xu, Q.R. From emotion AI to cognitive AI. Int. J. Network Dyn. Intell., 2022, 1: 65−72.
50.
Bosse, T.; Hartmann, T.; Blankendaal, R.A.M.; et al. Virtually bad: A study on virtual agents that physically threaten human beings. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, Stockholm, Sweden, 10–15 July 2018; International Foundation for Autonomous Agents and Multiagent Systems: Stockholm, 2018; pp. 1258–1266.
51.
Schmidt, S.; Zimmermann, S.; Mason, C.; et al. Simulating human imprecision in temporal statements of intelligent virtual agents. In CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022–5 May 2022; ACM: New Orleans, 2022; pp. 422. doi:10.1145/3491102.3517625
52.
Aneja, D.; McDuff, D.; Shah, S. A high-fidelity open embodied avatar with lip syncing and expression capabilities. In 2019 International Conference on Multimodal Interaction, Suzhou, China, 14–18 October 2019; ACM: Suzhou, 2019; pp. 69–73. doi:10.1145/3340555.3353744
53.
Niu, Z.Y.; Zhong, G.Q.; Yu, H. A review on the attention mechanism of deep learning. Neurocomputing, 2021, 452: 48−62.
54.
Liu, S.M.; Xia, Y.F.; Shi, Z.S.; et al. Deep learning in sheet metal bending with a novel theory-guided deep neural network. IEEE/CAA J. Autom. Sin., 2021, 8: 565−581.
55.
Vaitonytė, J.; Blomsma, P.A.; Alimardani, M.; et al. Realism of the face lies in skin and eyes: Evidence from virtual and human agents. Comput. Hum. Behav. Rep., 2021, 3: 100065.
56.
Lu, P.; Song, B.Y.; Xu, L. Human face recognition based on convolutional neural network and augmented dataset. Syst. Sci. Control Eng., 2021, 9: 29−37.
57.
Xia, Y.F.; Yu, H.; Wang, X.; et al. Relation-aware facial expression recognition. IEEE Trans. Cognit. Dev. Syst., 2022, 14: 1143−1154.
58.
Ahmed, I.; Din, S.; Jeon, G.; et al. Towards collaborative robotics in top view surveillance: A framework for multiple object tracking by detection using deep learning. IEEE/CAA J. Autom. Sin., 2021, 8: 1253−1270.
59.
Movella Inc. Motion Capture. Available online: https://www.xsens.com/motion-capture (accessed on 10 2022).
60.
Xia, Y.F.; Zheng, W.B.; Wang, Y.M.; et al. Local and global perception generative adversarial network for facial expression synthesis. IEEE Trans. Circuits Syst. Video Technol., 2022, 32: 1443−1452.
61.
Lou, J.W.; Wang, Y.M.; Nduka, C.; et al. Realistic facial expression reconstruction for VR HMD users. IEEE Trans. Multimedia, 2020, 22: 730−743.
62.
Laustsen, M.; Andersen, M.; Xue, R.; et al. Tracking of rigid head motion during MRI using an EEG system. Magn. Reson. Med., 2022, 88: 986−1001.
63.
Zhang, J.Y.; Chen, K.Y.; Zheng, J.M. Facial expression retargeting from human to avatar made easy. IEEE Trans. Vis. Comput. Graph., 2022, 28: 1274−1287.
64.
Zhang, X.G.; Yang, X.X.; Zhang, W.G.; et al. Crowd emotion evaluation based on fuzzy inference of arousal and valence. Neurocomputing, 2021, 445: 194−205.
65.
Ye, F.; Zhang, S.; Wang, P.; et al. A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles. In 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan, 11–17 July 2021; IEEE: Nagoya, 2021; pp. 1073–1080. doi:10.1109/IV48863.2021.9575880
66.
Yue, W.B.; Wang, Z.D.; Zhang, J.Y.; et al. An overview of recommendation techniques and their applications in healthcare. IEEE/CAA J. Autom. Sin., 2021, 8: 701−717.
67.
He, X.; Pan, Q.K.; Gao, L.; et al. A greedy cooperative Co-evolution ary algorithm with problem-specific knowledge for multi-objective flowshop group scheduling problems. IEEE Trans. Evol. Comput. 2021, in press. doi:10.1109/TEVC.2021.3115795
68.
Liu, W.B.; Wang, Z.D.; Zeng, N.Y.; et al. A novel randomised particle swarm optimizer. Int. J. Mach. Learn. Cybern., 2021, 12: 529−540.
69.
Zhang, K.; Su, Y.K.; Guo, X.W.; et al. MU-GAN: Facial attribute editing based on multi-attention mechanism. IEEE/CAA J. Autom. Sin., 2021, 8: 1614−1626.
70.
Wang, Y.M.; Dong, X.H.; Li, G.F.; et al. Cascade regression-based face frontalization for dynamic facial expression analysis. Cognit. Comput., 2022, 14: 1571−1584.
71.
Gou, C.; Zhou, Y.C.; Xiao, Y.; et al. Cascade learning for driver facial monitoring. IEEE Trans. Intell. Veh., 2023, 8: 404−412.
72.
Sharma, S.; Kumar, V. Voxel-based 3D face reconstruction and its application to face recognition using sequential deep learning. Multimed. Tools Appl., 2020, 79: 17303−17330.
73.
Sherman, W.R.; Craig, A.B. Understanding Virtual Reality: Interface, Application, and Design; Morgan Kaufmann: Boston, USA, 2003.
74.
Zell, E.; Aliaga, C.; Jarabo, A.; et al. To stylize or not to stylize?: The effect of shape and material stylization on the perception of computer-generated faces ACM Trans. Graph., 2015, 34: 184.
75.
Yu, H.; Garrod, O.G.B.; Schyns, P.G. Perception-driven facial expression synthesis. Comput. Graph., 2012, 36: 152−162.
76.
Andrist, S.; Pejsa, T.; Mutlu, B.; et al. Designing effective gaze mechanisms for virtual agents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; ACM: Austin, 2012; pp. 705–714. doi:10.1145/2207676.2207777
77.
Bopp, J.A.; Müller, L.J.; Aeschbach, L.F.; et al. Exploring emotional attachment to game characters. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play, Barcelona, Spain, 22–25 October 2019; ACM: Barcelona, 2019; pp. 313–324. doi:10.1145/3311350.3347169
78.
Wisessing, P.; Zibrek, K.; Cunningham, D.W.; et al. Enlighten me: Importance of brightness and shadow for character emotion and appeal. ACM Trans. Graph., 2020, 39: 19.
79.
Bailenson, J.N.; Beall, A.C.; Blascovich, J. Gaze and task performance in shared virtual environments. J. Visual. Comput. Anim., 2002, 13: 313−320.
80.
Sharma, M.; Vemuri, K. Accepting human-like avatars in social and professional roles. ACM Trans. Hum. Robot Interact., 2022, 11: 28.
81.
Freeman, G.; Zamanifard, S.; Maloney, D.; et al. My body, my avatar: How people perceive their avatars in social virtual reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; ACM: Honolulu, 2020; pp. 1–8. doi:10.1145/3334480.3382923
82.
Ducheneaut, N.; Wen, M.H.; Yee, N.; et al. Body and mind: A study of avatar personalization in three virtual worlds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; ACM: Boston, 2009; pp. 1151–1160. doi:10.1145/1518701.1518877
83.
Pan, X.N.; Hamilton, A.F.D.C. Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. Br. J. Psychol., 2018, 109: 395−417.

Scilight Press

Author Information

Abstract

Graphical Abstract

Keywords

References

About Scilight

Journals

Publishing Policies

Contact Us