On the evening of December 31, 2020, the 2021 New Year's Eve Party of Zhejiang Satellite TV was brilliantly staged. NetEase Mutual Entertainment AI Lab and Can Media jointly created the digital girl "Wang Yi", in the evening for the first time amazing appearance, its vivid and delicate expression and clever natural expression change, let the eyes shine. Replacing real people with digital figures for show performance, combining cool virtual idol technology with bold content creativity, and increasingly mature AI production technology have brought a brand new presentation mode for the immediacy and strong sense of scene live evening parties.
In the four preview VCRS interspersed in the New Year's Eve party of Zhejiang Satellite TV, Wang Yi became a virtual assistant in the AR world and acted a short story between four characters on New Year's Eve. It's a digital person, but it looks like a real person. In terms of expression, this young and lovely girl is highly realistic. Her eyes, lip movements, eyebrows, and even the patterns of her facial expressions are all detailed. When she speaks, she matches every word with the exact shape of her mouth. In different contexts, a variety of emotional changes, such as innocence, alertness, pout anger, surprise and curiosity, can be vividly portrayed and accurately expressed. And all this is inseparable from the strong support of AIxFace facial dynamic capture technology of NetEase Interactive Entertainment AI Lab.
AIxFace is NetEase Interactive AI The lightweight facial motion capture system developed by Lab is different from the traditional facial motion capture technology. AIxFace does not need complex professional motion capture sites and equipment, nor does it need to set any facial markers. It only needs a monocular camera, which can transfer the actor's facial expression to the digital girl's face in real time, and realize the expression details such as tongue, line of sight and mouth shape Accurate tracking of change. The facial expression of virtual idols has always been an important standard for fans to measure their performance. However, in order to achieve a high level of expression change, the technical cost is usually relatively high, which is one of the key thresholds of the virtual idol industry. With the aid of AIxFace, the final effect can be displayed in real time in the movement capture stage, which can greatly save the production time and cost, reserve more space for post rendering and other aspects, and finally achieve the animation effect presented in the evening party.
In addition to animation,AIxFace is also able to help virtual idols truly "break dimensional walls" with its time-driven features. In the past, even though traditional virtual idols could perform offline through holographic technology, it was difficult for them to interact with fans on a live basis due to technological limitations. Through AIxFace, virtual idols can realize real-time interaction with audiences in live broadcasting and other scenes, and ensure accurate and vivid expression changes while narrowing the distance with fans, enhancing the emotional appeal and sense of reality. Before the project,AIxFace has successfully helped a number of virtual characters live interact, such as the virtual detective anchor of the final game of Bilibili Bald Cup game of the Fifth Personality, whose debut exceeded 600,000 popularity.
Founded in 2017, NetEase Interactive Entertainment AI Lab is a leading AI lab in the game industry. In addition to the research on AI landing in the game scene, NetEase Interactive Entertainment AI Lab also provides artificial intelligence technical support related to virtual idols, such as facial expression dynamic capture, video dynamic capture and music automatic generation of dance animation. NetEase Interactive AI Lab hopes to continuously improve the intelligence level of virtual idol content generation through leading artificial intelligence technology, further enhance the precision, reality and interactive ability of virtual idol, so as to promote the gradual diversification of its business model. In the future, virtual idols can not only be active in the currently very common fields of film and television, games, and live broadcasting, but also can gradually expand to finance, education, medical and other broader application scenarios.
