In the second installment of her K-pop series, Faye Bradley discovers how Hong Kong’s tech wizards and metaverse architects are changing the ways in which K-pop is consumed. 

Earlier this year, a four-member all-girl K-pop group claimed the spotlight. Launched by Metaverse Entertainment in January, Mave had racked up over 20 million views for its hit single Pandora in less than a month. At first glance, Mave comes across as a regular quartet of teenage performers. Only that its members are not human. The figures were created using machine learning, deep fake and real-time 3D rendering technologies. Though they look almost real, a closer scrutiny reveals that the even skin tones and flawless figures of these virtual performers are too good to be true.

Mave is just one among a number of virtual K-pop groups to have emerged during the most restrictive phases of the COVID-19 pandemic. Eternity and its hit single I’m Real debuted in 2021. Launched in 2020, Aespa is a hybrid of the real and virtual. The band’s human artists sometimes share the stage with their digital counterparts, hologrammed into the concert arena. Kakao Entertainment launched Feverse this past May. Thirty human contestants chosen from existing K-pop bands competed in a survival virtual reality show under the guise of their digital avatars for a place in the five-member virtual girl band.

South Korean virtual boy band Plave attracted over 250,000 YouTube subscribers soon after its debut in 2023. Behind each of the band’s five animated 3D figures is a human artist, performing the vocals and dance movements, generated with the help of real-time motion capture technology.

South Korean virtual boy band Plave. (PHOTO PROVIDED TO CHINA DAILY)

Closer home

Experiments in using artificial intelligence to make new K-pop music are not limited to the genre’s home country, nor to the music industry alone. At the Hong Kong University of Science and Technology’s Department of Computer Science and Engineering, Andrew Horner’s students have made a number of such attempts. The results include a mash-up music video featuring major hits by two of the hottest K-pop acts — Fire, a song recorded by the boy band BTS for its compilation album in 2016; and the girl band Blackpink’s song Playing with Fire, released the same year. Students used AI tools to separate the sounds produced by each instrument in both pieces before recombining them.

“Like a good K-pop artist, AI is good at separating musical components as well as recombining them in fresh and interesting ways,” Horner says.

His student, PhD candidate Chris Law Man-hei, sounds hopeful about the prospects of AI-generated K-pop. He says he believes the rapid advancement of AI technology as well as the variety of choices available to K-pop fans will inspire greater openness toward appreciating AI-generated music.

Xu Xinyang, also pursuing a PhD in computer music, says the application of AI can reduce the cost and speed of making K-pop videos available in multiple languages. She hastens to add that being the provider of the main idea and structure of the piece, the role of the human music composer is crucial when it comes to music created with AI input.

However, music generated via machine learning, more often than not, sounds highly similar to the training data. “It’s like a hobbyist trying to sound like Ed Sheeran,” Xu says. “People hate such copycat behavior.”

Horner says that the key to making intelligent use of AI in creating new K-pop tunes “is to keep enough of the iconic hooks so that the audience knows that it is K-pop, while varying the music in ways that are emotionally and musically satisfying.”

HyunA. (PHOTO PROVIDED TO CHINA DAILY)

Metaverse music

AI is useful when it comes to customizing one’s listening experience. It can help broaden the scope of K-pop and consequently widen its fan base.

“As AI technologies evolve, we can anticipate their growing influence on the K-pop landscape, paving the way for exciting developments and opportunities for both artists and their fans,” says Roger Proeis, co-founder and president of the Hong Kong-based extended reality studio and production facility Votion Studios.

Such developments include the chance to experience K-pop in the metaverse — a virtual-reality space where users can have their digital avatars interact with each other in a computer-generated environment. Hong Kong’s gaming, blockchain and AI-related service provider Animoca Brands and South Korean music production company Cube Entertainment have joined forces to create music meant to be enjoyed in the metaverse.

“The goal of the joint venture is to usher the entire K-pop industry into this new paradigm of digital property rights,” says Animoca co-founder and Executive Chairman Yat Siu.

The music metaverse gaming platform Pixelynx, a subsidiary of Animoca, is developing a personal AI music bot that users — be they professional musicians or hobbyists — can employ to play, remix and create music.

“I am certain that the impact of machine-driven and machine-assisted creativity and innovation will be incredibly significant, especially for those artists that begin leveraging it early,” Siu says.

Aespa. (PHOTO PROVIDED TO CHINA DAILY)

To each their own

With the advancement of AI and growth of Web3 marketing platforms, music can now be experienced in a whole new range of formats. But are music fans ready to buy into the big changes?

“I had absolutely no idea that some of the dancers and vocals were AI-generated when I began watching Black Mamba by Aespa,” says research student and avid K-pop listener Syan Hu, referring to a music video that has registered over 250 million views since its release in November 2020. “However, in the later scenes, I did notice some jerky movements, giving away the fact that the performers were not human.”

However, she says, she believes AI is going to get better at resolving such oddities. “AI has the potential to revolutionize the music and performance industry, including K-pop,” she says. “I believe this is the future.”

Occasional K-pop listener William Pau is willing to give it to AI that it “can help raise the standard of K-pop videos to a level that human performers or even computer-generated imaging cannot, except perhaps in ways that are extremely expensive”. However, he does not sound too ecstatic about AI-made vocals. “AI-generated music is not as nice to listen to as human singers,” he declares.

K-pop fan Stephanie Mui, who manages the social media account of a luxury lifestyle magazine, has a similar view. “I don’t think AI-generated K-pop idols can replace real-life ones, mainly because K-pop idols and their fans have a connection that I don’t think is replaceable by AI-generated performers.”

But then, like it or not, AI music is here to stay and steadily evolving.

“The virtual and real worlds are merging, setting an important direction for AI music,” says Xue Wei, assistant professor at HKUST’s Division of Emerging Interdisciplinary Areas. The crucial task for musicians using AI, he points out, is to enable the machine to separate music from noise. The next step would be to teach machines to create with curiosity, achieve thematic cohesion and produce music that is new as opposed to being a rehash of what already exists. 

As more artists, and amateurs, start using tools like Midjourney, ChatGPT and Stable Diffusion to create music, concerns regarding the lack of copyright-related regulations are also on the rise. At the end of the day, “there is still a quality gap between the art created by AI and humans,” says Henry Chu, new media artist and creative director of Hong Kong-based web and interactive design studio, pill & pillow.

“In the end, it’s up to us whether we want to appreciate artificial creations,” he adds. “If the process of making music involves very little human input, will you still like it? Fast food can feed you, but it takes fine dining to satisfy your soul.”