Open Access
Survey/Review Study
Is High-Fidelity Important for Human-like Virtual Avatars in Human Computer Interactions?
Qiongdan Cao
, Hui Yu*
, Paul Charisse
, Si Qiao
, Brett Stevens
Author Information
Submitted: 7 Feb 2023 | Accepted: 6 Mar 2023 | Published: 27 Mar 2023
Abstract
As virtual avatars have become increasingly popular in recent years, current needs indicate that “interactivity” is crucial for inducing a positive response from users towards these avatars, especially in human computer interactions (HCI). This paper reviews recent works on high-fidelity human-like virtual avatars (e.g. high visual and motion fidelity) and discusses the critical question——“Is high-fidelity a positive choice for virtual avatars to achieve better interactions in HCI”. Furthermore, we summarise current technical approaches to developing those virtual avatars. We investigate the advantages and disadvantages of high-fidelity virtual avatars in different areas focusing on addressing the effect of motion, especially the upper body. Research shows that high-fidelity is a positive choice for virtual avatars, although it may depend on the application.
Graphical Abstract
Keywords
References
1.
Autodesk. Arnoldrenderer Research. Available online: https://arnoldrenderer.com/research/ (accessed on 10 December 2022).
2.
Park, S.W.; Park, H.S.; Kim, J. H.; et al. 3D displacement measurement model for health monitoring of structures using a motion capture system. Measurement, 2015, 59: 352−362.
3.
Zibrek, K.; Kokkinara, E.; McDonnell, R. The effect of realistic appearance of virtual characters in immersive environments-does the character's personality play a role? IEEE Trans. Vis. Comput. Graph., 2018, 24: 1681−1690.
4.
Jo, D.; Kim, K.H.; Kim, G.J. Effects of avatar and background representation forms to co-presence in mixed reality (MR) tele-conference systems. In SIGGRAPH ASIA 2016 Virtual Reality Meets Physical Reality : Modelling and Simulating Virtual Humans and Environments , Macau , China , 5 –8 December 2016 ; ACM: Macau, China, 2016; pp. 12. doi:10.1145/2992138.2992146
5.
Latoschik, M.E.; Roth, D.; Gall, D.; et al. The effect of avatar realism in immersive social virtual realities. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology , Gothenburg , Sweden , 8 –10 November 2017 ; ACM: Gothenburg, 2017; pp. 39. doi:10.1145/3139131.3139156
6.
Garau, M. The Impact of Avatar Fidelity on Social Interaction in Virtual Environments. Ph.D. Thesis, University College London, London, UK, 2003.
7.
Mori, M.; MacDorman, K.F.; Kageki, N. The uncanny valley [from the field]. IEEE Robot. Autom. Mag., 2012, 19: 98−100.
8.
Hanson, D.; Olney, A.; Prilliman, S.; et al. Upending the uncanny valley. In Proceedings of the 20th National Conference on Artificial Intelligence , Pittsburgh , Pennsylvania , USA , 9 –13 July 2005 ; AAAI Press: Pittsburgh, 2005; pp. 1728–1729.
9.
Geller, T. Overcoming the uncanny valley. IEEE Comput. Graph. Appl., 2008, 28: 11−17.
10.
Brenton, H.; Gillies, M.; Ballin, D.; et al. The uncanny valley: Does it exist. In Proceedings of the 11th Conference of Human Computer Interaction , Workshop on Human Animated Character Interaction , Las Vegas , NV , USA , Lawrence Erlbaum Associates: Las Vegas, 2005.
11.
Volante, M.; Babu, S.V.; Chaturvedi, H.; et al. Effects of virtual human appearance fidelity on emotion contagion in affective inter-personal simulations. IEEE Trans. Vis. Comput. Graph., 2016, 22: 1326−1335.
12.
Ries, B.; Interrante, V.; Kaeding, M.; et al. Analyzing the effect of a virtual avatar's geometric and motion fidelity on ego-centric spatial perception in immersive virtual environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology , Kyoto , Japan , 18 –20 November 2009 ; ACM: Kyoto, 2009; pp. 59–66. doi:10.1145/1643928.1643943
13.
BANDAI NAMCO. Elden Ring. FromSoftware, Inc. 2022. Available online: https://en.bandainamcoent.eu/elden-ring/elden-ring (accessed on 4 July 2022).
14.
Cao, C.; Agrawal, V.; De La Torre, F.; et al. Real-time 3D neural facial animation from binocular video. ACM Trans. Graph., 2021, 40: 87.
15.
Yu, H.; Liu, H.H. Regression-based facial expression optimization. IEEE Trans. Hum. Mach. Syst., 2014, 44: 386−394.
16.
Wang, L.Z.; Chen, Z.Y.; Yu, T.; et al. FaceVerse: A fine-grained and detail-controllable 3D face morphable model from a hybrid dataset. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , New Orleans , LA , USA , 18 –24 June 2022 0; IEEE: New Orleans, 2022; pp. 20301–20310. doi:10.1109/CVPR52688.2022.01969
17.
Zhang, S.; Yu, H.; Wang, T.; et al. Linearly augmented real-time 4D expressional face capture. Inf. Sci., 2021, 545: 331−343.
18.
Zall, R.; Kangavari, M.R. Comparative analytical survey on cognitive agents with emotional intelligence. Cognit. Comput., 2022, 14: 1223−1246.
19.
D’Avella, S.; Camacho-Gonzalez, G.; Tripicchio, P. On Multi-Agent Cognitive Cooperation: Can virtual agents behave like humans? Neurocomputing, 2022, 480: 27−38.
20.
Thompson, J.; White, S.; Chapman, S. Interactive clinical avatar use in pharmacist preregistration training: Design and review. J. Med. Internet Res., 2020, 22: e17146.
21.
Sinatra, A.M.; Pollard, K.A.; Files, B.T.; et al. Social fidelity in virtual agents: Impacts on presence and learning. Comput. Hum. Behav., 2021, 114: 106562.
22.
Machidon, O.M.; Duguleana, M.; Carrozzino, M. Virtual humans in cultural heritage ICT applications: A review. J. Cult. Herit., 2018, 33: 249−260.
23.
Foni, A.E.; Papagiannakis, G.; Magnenat-Thalmann, N. Virtual Hagia Sophia: Restitution, visualization and virtual life simulation. In UNESCO World Heritage Congress (Vol . 2 ); 2002.
24.
Stefanova, M.; Pillan, M.; Gallace, A. Influence of realistic virtual environments and humanlike avatars on patients with social phobia. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference , 17–19 August 2021 ; ASME, 2021; pp. V002T02A081. doi:10.1115/DETC2021-70265
25.
Mousas, C.; Koilias, A.; Rekabdar, B.; et al. Toward understanding the effects of virtual character appearance on avoidance movement behavior. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR ), Lisboa , Portugal , 27 March 2021–1 April 2021 ; IEEE: Lisboa, 2021; pp. 40–49. doi:10.1109/VR50410.2021.00024
26.
Hube, N.; Angerbauer, K.; Pohlandt, D.; et al. VR collaboration in large companies: An interview study on the role of avatars. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct ), Bari , Italy , 4 –8 October 2021 ; IEEE: Bari, 2021; pp. 139–144. doi:10.1109/ISMAR-Adjunct54149.2021.00037
27.
Waltemate, T.; Gall, D.; Roth, D.; et al. The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response. IEEE Trans. Vis. Comput. Graph., 2018, 24: 1643−1652.
28.
Gochfeld, D.; Benzing, K.; Laibson, K.; et al. Avatar selection for live performance in virtual reality: A case study. In 2019 IEEE Games , Entertainment , Media Conference (GEM ), New Haven , CT , USA , 18 –21 June 2019 ; IEEE: New Haven, 2019; pp. 1–5. doi:10.1109/GEM.2019.8811548
29.
Ma, F.; Pan, X.N. Visual fidelity effects on expressive self-avatar in virtual reality: First impressions matter. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR ), Christchurch , New Zealand , 12 –16 March 2022 ; IEEE: Christchurch, 2022; pp. 57–65. doi:10.1109/VR51125.2022.00023
30.
Herrera, F.; Oh, S.Y.; Bailenson, J.N. Effect of behavioral realism on social interactions inside collaborative virtual environments. Presence: Teleoperat. Virtual Environ., 2018, 27: 163−182.
31.
Rogers, S.L.; Broadbent, R.; Brown, J.; et al. Realistic motion avatars are the future for social interaction in virtual reality. Front. Virtual Real., 2022, 2: 750729.
32.
Tinwell, A.; Grimshaw, M.; Nabi, D.A.; et al. Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput. Hum. Behav., 2011, 27: 741−749.
33.
McDonnell, R.; Larkin, M.; Hernández, B.; et al. Eye-catching crowds: Saliency based selective variation. ACM Trans. Graph., 2009, 28: 55.
34.
Jack, R.E.; Garrod, O.G.B.; Yu, H.; et al. Facial expressions of emotion are not culturally universal. Proc. Natl. Acad. Sci. USA, 2012, 109: 7241−7244.
35.
Gonzalez-Franco, M.; Steed, A.; Hoogendyk, S.; et al. Using facial animation to increase the enfacement illusion and avatar self-identification. IEEE Trans. Vis. Comput. Graph., 2020, 26: 2023−2029.
36.
Bai, Z.C.; Yao, N.M.; Mishra, N.; et al. Enhancing emotional experience by building emotional virtual characters in VR volleyball games. Comput. Animat. Virtual Worlds, 2021, 32: e2008.
37.
Yun, C.; Deng, Z.G.; Hiscock, M. Can local avatars satisfy a global audience? A case study of high-fidelity 3D facial avatar animation in subject identification and emotion perception by US and international groups Comput. Entertain., 2009, 7: 21.
38.
Hodgins, J.; Jörg, S.; O'Sullivan, C.; et al. The saliency of anomalies in animated human characters. ACM Trans. Appl. Percept., 2010, 7: 22.
39.
Ruhland, K.; Zibrek, K.; McDonnell, R. Perception of personality through eye gaze of realistic and cartoon models. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception , Tübingen , Germany , 13 –14 September 2015 ; ACM: Tübingen, 2015; pp. 19–23. doi:10.1145/2804408.2804424
40.
Ruhland, K.; Peters, C.E.; Andrist, S.; et al. A review of eye gaze in virtual agents, social robotics and HCI: Behaviour generation, user interaction and perception. Comput. Graph. Forum, 2015, 34: 299−326.
41.
Kokkinara, E.; McDonnell, R. Animation realism affects perceived character appeal of a self-virtual face. In Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games , Paris , France , 16 –18 November 2015 ; ACM: Paris, 2015; pp. 221–226. doi:10.1145/2822013.2822035
42.
Cao, Q.D.; Yu, H.; Nduka, C. Perception of head motion effect on emotional facial expression in virtual reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW ), Atlanta , GA , USA , 22 –26 March 2020 ; IEEE: Atlanta, 2020; pp. 750–751. doi:10.1109/VRW50115.2020.00226
43.
Garau, M.; Slater, M.; Vinayagamoorthy, V.; et al. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , Ft . Lauderdale , FL , USA , 5 –10 April 2003 ; ACM: Ft. Lauderdale, 2003; pp. 529–536. doi:10.1145/642611.642703
44.
Petrović, V.M. Artificial intelligence and virtual worlds-toward human-level AI agents. IEEE Access, 2018, 6: 39976−39988.
45.
Norouzi, N.; Kim, K.; Hochreiter, J.; et al. A systematic survey of 15 years of user studies published in the intelligent virtual agents conference. In Proceedings of the 18th International Conference on Intelligent Virtual Agents , Sydney , Australia , 5 –8 November 2018 ; ACM: Sydney, 2018; pp. 17–22. doi:10.1145/3267851.3267901
46.
Luerssen, M.H.; Hawke, T. Virtual agents as a service: Applications in healthcare. In Proceedings of the 18th International Conference on Intelligent Virtual Agents , Sydney , Australia , 5 –8 November 2018 ; ACM: Sydney, 2018; pp. 107–112. doi:10.1145/3267851.3267858
47.
Alicja, K.; Maciej, M. Can AI see bias in X-ray images. Int. J. Network Dyn. Intell., 2022, 1: 48−64.
48.
Jiang, F.; Jiang, Y.; Zhi, H.; et al. Artificial intelligence in healthcare: Past, present and future. Stroke Vasc. Neurol., 2017, 2: 230−243.
49.
Zhao, G.Y.; Li, Y.T.; Xu, Q.R. From emotion AI to cognitive AI. Int. J. Network Dyn. Intell., 2022, 1: 65−72.
50.
Bosse, T.; Hartmann, T.; Blankendaal, R.A.M.; et al. Virtually bad: A study on virtual agents that physically threaten human beings. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems , Stockholm , Sweden , 10 –15 July 2018 ; International Foundation for Autonomous Agents and Multiagent Systems: Stockholm, 2018; pp. 1258–1266.
51.
Schmidt, S.; Zimmermann, S.; Mason, C.; et al. Simulating human imprecision in temporal statements of intelligent virtual agents. In CHI Conference on Human Factors in Computing Systems , New Orleans , LA , USA , 29 April 2022 –5 May 2022 ; ACM: New Orleans, 2022; pp. 422. doi:10.1145/3491102.3517625
52.
Aneja, D.; McDuff, D.; Shah, S. A high-fidelity open embodied avatar with lip syncing and expression capabilities. In 2019 International Conference on Multimodal Interaction , Suzhou , China , 14 –18 October 2019 ; ACM: Suzhou, 2019; pp. 69–73. doi:10.1145/3340555.3353744
53.
Niu, Z.Y.; Zhong, G.Q.; Yu, H. A review on the attention mechanism of deep learning. Neurocomputing, 2021, 452: 48−62.
54.
Liu, S.M.; Xia, Y.F.; Shi, Z.S.; et al. Deep learning in sheet metal bending with a novel theory-guided deep neural network. IEEE/CAA J. Autom. Sin., 2021, 8: 565−581.
55.
Vaitonytė, J.; Blomsma, P.A.; Alimardani, M.; et al. Realism of the face lies in skin and eyes: Evidence from virtual and human agents. Comput. Hum. Behav. Rep., 2021, 3: 100065.
56.
Lu, P.; Song, B.Y.; Xu, L. Human face recognition based on convolutional neural network and augmented dataset. Syst. Sci. Control Eng., 2021, 9: 29−37.
57.
Xia, Y.F.; Yu, H.; Wang, X.; et al. Relation-aware facial expression recognition. IEEE Trans. Cognit. Dev. Syst., 2022, 14: 1143−1154.
58.
Ahmed, I.; Din, S.; Jeon, G.; et al. Towards collaborative robotics in top view surveillance: A framework for multiple object tracking by detection using deep learning. IEEE/CAA J. Autom. Sin., 2021, 8: 1253−1270.
59.
Movella Inc. Motion Capture. Available online: https://www.xsens.com/motion-capture (accessed on 10 2022).
60.
Xia, Y.F.; Zheng, W.B.; Wang, Y.M.; et al. Local and global perception generative adversarial network for facial expression synthesis. IEEE Trans. Circuits Syst. Video Technol., 2022, 32: 1443−1452.
61.
Lou, J.W.; Wang, Y.M.; Nduka, C.; et al. Realistic facial expression reconstruction for VR HMD users. IEEE Trans. Multimedia, 2020, 22: 730−743.
62.
Laustsen, M.; Andersen, M.; Xue, R.; et al. Tracking of rigid head motion during MRI using an EEG system. Magn. Reson. Med., 2022, 88: 986−1001.
63.
Zhang, J.Y.; Chen, K.Y.; Zheng, J.M. Facial expression retargeting from human to avatar made easy. IEEE Trans. Vis. Comput. Graph., 2022, 28: 1274−1287.
64.
Zhang, X.G.; Yang, X.X.; Zhang, W.G.; et al. Crowd emotion evaluation based on fuzzy inference of arousal and valence. Neurocomputing, 2021, 445: 194−205.
65.
Ye, F.; Zhang, S.; Wang, P.; et al. A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles. In 2021 IEEE Intelligent Vehicles Symposium (IV ), Nagoya , Japan , 11 –17 July 2021 ; IEEE: Nagoya, 2021; pp. 1073–1080. doi:10.1109/IV48863.2021.9575880
66.
Yue, W.B.; Wang, Z.D.; Zhang, J.Y.; et al. An overview of recommendation techniques and their applications in healthcare. IEEE/CAA J. Autom. Sin., 2021, 8: 701−717.
67.
He, X.; Pan, Q.K.; Gao, L.; et al. A greedy cooperative Co-evolution ary algorithm with problem-specific knowledge for multi-objective flowshop group scheduling problems. IEEE Trans . Evol . Comput . 2021 , in press. doi:10.1109/TEVC.2021.3115795
68.
Liu, W.B.; Wang, Z.D.; Zeng, N.Y.; et al. A novel randomised particle swarm optimizer. Int. J. Mach. Learn. Cybern., 2021, 12: 529−540.
69.
Zhang, K.; Su, Y.K.; Guo, X.W.; et al. MU-GAN: Facial attribute editing based on multi-attention mechanism. IEEE/CAA J. Autom. Sin., 2021, 8: 1614−1626.
70.
Wang, Y.M.; Dong, X.H.; Li, G.F.; et al. Cascade regression-based face frontalization for dynamic facial expression analysis. Cognit. Comput., 2022, 14: 1571−1584.
71.
Gou, C.; Zhou, Y.C.; Xiao, Y.; et al. Cascade learning for driver facial monitoring. IEEE Trans. Intell. Veh., 2023, 8: 404−412.
72.
Sharma, S.; Kumar, V. Voxel-based 3D face reconstruction and its application to face recognition using sequential deep learning. Multimed. Tools Appl., 2020, 79: 17303−17330.
73.
Sherman, W.R.; Craig, A.B. Understanding Virtual Reality : Interface , Application , and Design ; Morgan Kaufmann: Boston, USA, 2003.
74.
Zell, E.; Aliaga, C.; Jarabo, A.; et al. To stylize or not to stylize?: The effect of shape and material stylization on the perception of computer-generated faces ACM Trans. Graph., 2015, 34: 184.
75.
Yu, H.; Garrod, O.G.B.; Schyns, P.G. Perception-driven facial expression synthesis. Comput. Graph., 2012, 36: 152−162.
76.
Andrist, S.; Pejsa, T.; Mutlu, B.; et al. Designing effective gaze mechanisms for virtual agents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , Austin , TX , USA , 5 –10 May 2012 ; ACM: Austin, 2012; pp. 705–714. doi:10.1145/2207676.2207777
77.
Bopp, J.A.; Müller, L.J.; Aeschbach, L.F.; et al. Exploring emotional attachment to game characters. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play , Barcelona , Spain , 22 –25 October 2019 ; ACM: Barcelona, 2019; pp. 313–324. doi:10.1145/3311350.3347169
78.
Wisessing, P.; Zibrek, K.; Cunningham, D.W.; et al. Enlighten me: Importance of brightness and shadow for character emotion and appeal. ACM Trans. Graph., 2020, 39: 19.
79.
Bailenson, J.N.; Beall, A.C.; Blascovich, J. Gaze and task performance in shared virtual environments. J. Visual. Comput. Anim., 2002, 13: 313−320.
80.
Sharma, M.; Vemuri, K. Accepting human-like avatars in social and professional roles. ACM Trans. Hum. Robot Interact., 2022, 11: 28.
81.
Freeman, G.; Zamanifard, S.; Maloney, D.; et al. My body, my avatar: How people perceive their avatars in social virtual reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems , Honolulu , HI , USA , 25 –30 April 2020 ; ACM: Honolulu, 2020; pp. 1–8. doi:10.1145/3334480.3382923
82.
Ducheneaut, N.; Wen, M.H.; Yee, N.; et al. Body and mind: A study of avatar personalization in three virtual worlds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , Boston , MA , USA , 4 –9 April 2009 ; ACM: Boston, 2009; pp. 1151–1160. doi:10.1145/1518701.1518877
83.
Pan, X.N.; Hamilton, A.F.D.C. Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. Br. J. Psychol., 2018, 109: 395−417.
Issue
Volume 2, Issue 1How to Cite
Cao, Q., Yu, H., Charisse, P., Qiao, S., & Stevens, B. (2023). Is High-Fidelity Important for Human-like Virtual Avatars in Human Computer Interactions? International Journal of Network Dynamics and Intelligence, 2(1), 15–23. https://doi.org/10.53941/ijndi0201008
RIS
BibTex
Copyright & License
Copyright (c) 2023 by the authors.
This work is licensed under a This work is licensed under a Creative Commons Attribution 4.0 International License.