Editor’s Note: Jeffrey Shaw shines like a guiding light on Hong Kong’s growing new media art scene. The Yeung Kin Man Chair Professor of Media Art at City University of Hong Kong’s School of Creative Media has taught and/or collaborated with some of the city’s most exciting new media artists. His own creations cover an astounding range, and almost always include a scope for the audience to participate in and exercise agency.
A co-curator of the ongoing Atlas of Maritime Buddhism exhibition, and a co-creator (with Sarah Kenderdine) of the digital interfaces specially developed for the show, Shaw was extraordinarily generous with his time in explaining the nature of the innovation involved in putting the show together, and how the application of new technologies helped to make the experience more meaningful for the audience. In a freewheeling interview with China Daily, Shaw also shared his thoughts on some of his recent works, the fraught nature of the artist-technology dynamic and Hong Kong’s prospects as a center of experimental new media art.
Excerpts from the interview:
Jeffrey Shaw has been making significant contributions to Hong Kong’s new media art scene as artist, academic and curator since he joined CityU School of Creative Media as its dean, 11 years ago. (CALVIN NG / CHINA DAILY)
The Atlas of Maritime Buddhism exhibition presents ancient heritage through the lens of modern technology. I believe the project team has developed certain digital interfaces specifically for this show. Would you like to give any examples of the ways in which visitor experience has been enhanced in this exhibition?
Many of the visualization techniques that we are using in this exhibition grew out of my own art practice. I believe contemporary art can take new interactive forms, allowing the viewer to not just be a passive spectator, but to be pro-actively engaged with the artwork and become its co-creator. I am also interested in creating spaces of representation that are very immersive, giving audiences a sense of being an embodied presence in the virtual world. Panoramic paintings are the precursors, and today we can create these panoramic experiences with projection systems.
Professor Sarah Kenderdine (professor of Digital Museology at the École polytechnique fédérale de Lausanne, Switzerland) and I have worked on many projects related to tangible and intangible cultural heritage — the Dunhuang caves in China, and at the Hampi heritage site in India for example. We have also worked with Hong Kong kung fu masters to showcase the heritage of Chinese martial arts. In these projects we have developed new visualization paradigms that create engaging interactive experiences for the public, so that they become explorers and discoverers in cultural information spaces.
Would you like to tell us about some of the specific technologies used in this exhibition, photogrammetry, for instance… I believe you visited many of the sites in Cambodia, India and Sri Lanka where the Buddhist monuments are and personally took hundreds of thousands of photos used in this project. It would be great if you could share a bit about putting it all together for this show.
There are two aspects to the new media art components of this exhibition. One of these are our strategies for digital documentation. Most of the images in this exhibition are presented in the form of panoramic experiences — the viewer is not simply looking at an image but also have the experience of being immersed in its image space. To achieve this, we shot panoramic stereoscopic photographs at over a thousand locations across Asia using two side by side medium-format film cameras that are motorized to rotate 360-degrees. Another technology we deployed is panoramic videography — these are spherical video recording which immersively document prayer gatherings, ceremonies and ritual events in the Buddhist calendar.
Photogrammetry is a photographic technique that enables the making of digital 3D models of Buddhist sculptures. On location we employ an SLR camera to take hundreds of photographs of these sculptures from every possible point of view, and then use software tools to integrate these photos to create high-resolution virtual 3D models. These digital replicas are presented in this exhibition on vertical LCD screens where they are animated to slowly rotate as noble figures that can be viewed from all sides. Buddhist sculptures in museums are fixed objects in space, but via these digital forms they can be given extra dimensions that amplify their aesthetic and spiritual qualities.
Shaw, with Sarah Kenderdine, developed a number of visualization techniques to be used in the Atlas of Maritime Buddhism exhibition, including photogrammetry which enables the making of digital 3D models of Buddhist sculptures. (CALVIN NG / CHINA DAILY)
Were any of the tools and technologies, including the animation, employed to create this show produced or developed in Hong Kong?
Not too long ago, that spherical video camera that we are now using was not yet on the market. In my early works featuring spherical videography — such as PLACE-Ruhr (2000) — I had to custom build that cameras myself, and it was cumbersome and difficult to work with. But popular interest in panoramic photography and videography has become ubiquitous over the last few years. This has a lot to do with the immersion settings of video games, as well as sports-selfie culture, so now there are numerous consumer cameras that can do 360 degree videography. We used the Insta Titan, a high-end 11K camera made in Shenzhen. It’s interesting to see so many emerging manufacturers in China making innovative photographic and cinematographic systems.
For photogrammetry you can use an off-the-shelf camera, even a mobile phone. The important element is the AI software which takes hundreds of pictures and joins them together to make a 3D model. This software has been evolving over the years and today is very capable. Once you have achieved this digital replica, you can animate and light it to give it whatever appearance and behavior you want.
While this exhibition has a lot to do with these special documentation technologies, another set of technologies is employed for presentation and audience engagement. Many of these are unique, having been developed over the years by Professor Kenderdine and myself.
One example is the Linear Navigator, an LED-screen monitor that travels on rails along the wall for about nine meters. I invented this mobile screen system in 1997 and since then we have created many variations and applications for it. Basically, the Linear Navigator allows one to virtually configure a database of information (photographs, videos, graphic imagery or texts), and then use its mobile screen to let the viewer travel and explore this information in space and/or time. In the Atlas exhibition it enables you to travel from India to China over its nine-meter trajectory, visiting seven countries and seeing the panoramas of seventy famous Buddhist sites along the way.
In this exhibition we are also showing the Panoramic Navigator – another system that I developed back in the 90s. It’s similar in principle to the Linear Navigator, but here the visitor rotates a projected viewing window inside a cylindrical screen. The information is all around you and when rotating the projector, the viewer chooses the countries they want to visit to explore all the panoramic images that were shot at Buddhist locations there.
Another visualization system in the exhibition is the iDome. Developed by Paul Bourke in Australia, it’s a hemispherical projection environment that is a perfect way to show 360-degree immersive photographs and videos. Viewers find themselves standing in the middle of a virtual pictorial sphere and have a very embodied sense of being present in that space. This is because one stands exactly where the camera was when the photograph or video was taken and thus experiences the full surround of what the camera saw and recorded.
The Atlas of Maritime Buddhism exhibition, co-curated by Shaw, also features the iDome — a hemispherical projection environment to show 360-degree immersive photographs and videos. (CALVIN NG / CHINA DAILY)
Viewers seem to have a degree of control on what they see in all of these visualization features, as they get to pick and edit the images by maneuvering the tools provided to them. The idea ties in with your core philosophy of giving agency to the audience…
The viewer has the opportunity to rotate the pictorial space — a Buddhist cave in India for example — as if you are standing there and simply looking around. Then by just pressing a button you can transport yourself from one location to another, constituting an archive of profound Buddhist sites that become available to the viewer to virtually visit “in person.”
This show has been running for two months now. What has the audience response been like so far?
We are very gratified to see so many people enjoying it. There are visitors who have visited many of the locations we are showing in the exhibition who gain the pleasure of a re-enactment of being there. And for all the others, being able to visit these evocative Buddhist sites across seven countries by means of this exhibition is a veritable luxury of discovery.
Our curatorial strategy aims to enable people of all ages and interests to connect with the content. The installations that are immersive and interactive appeal to everyone — new media seduces their interest and creates spectacles that draw them in — the joy of looking becomes a path towards fuller understanding and appreciation.
The exhibition contains a lot of information even just with its seventy panoramic locations and over fifty different experiences in the iDomes. Yet we designed the exhibition in such a way that in about an hour you can have a comprehensive and satisfying experience. That design also allows one to spend more time there, or to come back many times and always discover new things.
This show could be a bit overwhelming for someone who is unaware of the historical context, especially given the sheer volume of images, texts and multimedia experiences on offer. What would you like the non-scholarly audience to take away from an exhibition that began as and remains an academic research project?
The project was first conceived by Professors Kenderdine and Lewis Lancaster (Department of East Asian Languages, University of California, Berkeley). Initial funding came from the Australian Research Council, with the intention to build a comprehensive documentation of the spread of Buddhism across Asia through maritime routes, which could then manifest itself in different exhibition forms.
The project aims for modularity, in the many ways this exhibition can adapt to different contexts. For instance, showing it in maritime museums would provide the opportunity to make a more in-depth exposition about maritime trade. In the upcoming Chengdu Biennale (opening November 6, 2021) we will just show our 360-degree panoramic projections inside a vast cylindrical screen. This immersive interactive display of the panoramic photographs that we shot at Buddhist sites across seven countries speaks for itself as a synoptic artwork. These exhibitions also serve as a research base, so the ways in which we present the Atlas’s materials in different contexts can be tailored to specific research interests.
Among the other unique visualization technologies invented by Shaw and featured at The Atlas of Maritime Buddhism exhibition is the Linear Navigator, a mobile database of images and texts. (CALVIN NG / CHINA DAILY)
So the Taiwan arm of the exhibition is not a replica of what one gets to see here in Hong Kong?
The exhibition in Hong Kong again demonstrates its modular capacity. Here we had the opportunity to bring in original Buddhist sculptures and artefacts that were generously loaned by the University of Hong Kong, Chinese University of Hong Kong and private collectors like Rossi & Rossi. Our exhibition co-curator Dr. Marnie Feneley selected these sculptures and orchestrated their relationships with our virtual photogrammetric sculptures. In this way the conjunction of new media components with the physicality of these real objects becomes a signature experience of this exhibition — a dialogue between actual objects and mediated representations. On the other hand, the exhibition at Fo Guang Shan in Taiwan is intentionally a purely “new media art” experience, entirely informed by its spectacular interactives and digital documentations of Buddhist temples, sculptures and happenings.
Moving on to your recent works as a new media artist, the Fall Again, Fall Better interactive video projection installation you created with Sarah Kenderdine in 2012 saw its third iteration recently in Bad Rothenfelde, Germany, in the form of a 250-meter video projection installation. Would you like to tell us how this work has evolved over the years and why you kept re-visiting it?
Yes, this work resonates with many people and is being often shown in various contexts. In my art practice I have always been interested in developing interfaces that create experiences of embodiment and engagement for the viewer. For the original installation of Fall Again, Fall Better at the Shanghai Biennale in 2012, I wanted to invert the normal use of the handles inside an MTR train that people reach out for to stop themselves from falling. But in Fall Again, Fall Better, pulling on this same handle causes the fall of its virtual group of human figures. In this way I call on viewers’ kinesthetic memory of their MTR use to amplify the narrative scope of the artwork.
But I didn’t foresee that this interface would turn out to be too playful, because this work needs to function in such a way as to make the viewer sensitive to its horror as well as its comedy. So in its following iteration I exchanged the handle for a mat on the floor that the viewer has to step on. This is both kinesthetically and conceptually more effective as the viewer uses their own body weight to make those projected figures fall, thereby creating a stronger physiological engagement and personal sense of agency.
Recently I was invited to show the work in Germany at the Lichtsicht 7 projection-triennale in Bad Rothenfelde (re-opening on October 22, 2021). It is shown on a projection surface that is an extraordinary 350 meters long and 10 meters high screen. At that urban scale it no longer meaningful for any one person to step on a mat and trigger its action. So I gave the falling figures an automated “domino effect” — over the length of the wall there are twenty groups of figures and the fall of one group causes the successive fall of the next groups. Once they have all fallen, all rise together again and the sequence repeats itself.
Most importantly in this artwork these figures never fall the same way twice because a computational algorithm controls their movements and ensures that the manner in which they fall is always different — always providing different configurations of the mass of their fallen bodies.
This piece can bring to mind Francisco Goya’s painting, The Third of May, 1808 (where Spanish rebels are being shot at by Napoleon’s soldiers as part of an imperialist mission to conquer Madrid), except that in Fall Again Fall Better the victims are made to fall an infinite number of times, and its viewers are the causal agency. On the other hand, Fall Again Fall Better also resonates with the absurd misadventures of a Buster Keaton movie where we become the instigators of something funnily slapstick. It’s at this tragicomic intersection that this artwork poses its underlying narrative.
One of Shaw’s most-recent COVID-19-themed creations, called Safe House, explores the ideas of anxiety, vulnerability, obsessive-compulsive behavior and voyeurism triggered by the pandemic. (CALVIN NG / CHINA DAILY)
Many of your works — including Fall Again, The Golden Calf, Legible City, Eavesdrop, which were all part of your last retrospective show What You See Is What You Get — are highly interactive, in fact, it’s up to the viewer to activate them. But I wonder if it’s also about the illusion of exercising agency. For instance, in Legible City, the viewer gets a simulation of biking through Amsterdam when actually he has been pedaling only an exercise bike and been treated to a pre-designed scenography. So there seems to be a hierarchical tension between artist and audience here…
There is a great range to what an interactive artwork can be. For example, the interaction can be symbolic or gestural. If you are merely stepping on a mat to switch a button, what does that say about the complexity of the idea of interaction? Yet in Fall Again, Fall Better pushing that button is the conceptual and operational key to its experience — it’s all you need to do to engage with its narrative. In some of my other works the viewer’s interactivity is given a bigger range — for example the freedom to explore a large space of representation in Legible City (1989-91). This artwork’s virtual world covers five square kilometers in Central Amsterdam, and the viewer is bicycling there at 1:1 scale, going wherever they want. While a painting invites your gaze to travels across its surface, such interactive works give physical expression to the viewer’s capacity to virtually travel within vast spaces of representation.
While in most of my works the virtual scene is predetermined and the interactors cannot change it, The Golden Calf does to some extent allow itself to be modified by the viewer’s presence. As you look at this virtual sculpture, video cameras are recording your image and transferring it as a reflection onto the skin of the golden calf.
There can be other types of interactive work that are even more so in the hands of the viewer to shape — like building blocks to use in any way they please, although even in such cases, it’s usually the artist who is designing these blocks. This form of interactive art can be expanded by AI and machine learning, thereby providing algorithmic frameworks that further empower the creativity of the interactor.
Ultimately it is the artwork’s conceptual framing that dictates the type of interactivity that is chosen. The design and functionality of the interface is an aesthetic and formal decision taken by the artist.
I would like to bring up the two pieces of yours featured in the recent Art Machines show, hosted by City University’s School of Creative Media. Let’s begin with the COVID-19-themed Safe House, where the viewer can use an app to view an elderly figure inside each compartment of a locker cabinet, spending time in isolation by exercizing. Does the title of this piece imply a degree of irony?
Safe House is a piece that came together quite easily — almost as if the work already existed and was just waiting/needing to be discovered. It is an assemblage of found objects, in the same way that Picasso’s Bull’s Head (1942) is just made from a bicycle’s seat and handlebars. The elements I put together were a locker cabinet, a microscope image of the coronavirus that I found on Google, two elderly human figures that I sourced from a 3D model library on the internet, and a set of human figure movements from a motion capture library, also found on the internet.
The microscope image of the virus looks a bit like a fungus. I wanted to relay the impression that the locker cabinet itself was carrying the Covid infection, because it was generally believed you can contract Covid by touching metal surfaces that carry its residue. The motion capture library of human figure movements that I sourced was mainly intended for video game makers, but in the context of the pandemic lockdown, these same movements can become exercize. Finding a representative elderly male and female figure was quite difficult because 3D model libraries are almost exclusively populated by younger stereotypes.
Safe House is an artwork informed by Covid sensibility and Covid anxiety. The elderly are the most vulnerable, and so the piece also talks about my own vulnerability as well. The figures in the lockers exercise with video game movements, seemingly picked up from an increased exposure to video games while being stuck at home or in quarantine. And the repetition of these movements in Safe House is disturbingly obsessive-compulsive, like animals in a cage — the oscillation between exercise regimens and sedentary Netflix watching that has become a symptom of the pandemic’s infection of our existential space.
The title Safe House has some irony of course. It’s a space where you expect to be protected. Yet anyone who has spent time quarantining knows the claustrophobia, the feeling of being locked up in a box. There is also the voyeurism — allowing the viewer to peek seemingly unobserved into each locker, and watch what the person inside is doing there, like the home-bound protagonist in Alfred Hitchcock’s film Rear Window, but now his telephoto camera has become an iPad with which to pry open doors and stare into other people’s “cubicles” and lives.
The title just came out of nowhere, but it felt immediately appropriate because the idea of a “safe house” links to other dimensions, and in the jargon of law enforcement and intelligence agencies it’s a secret and secure location, suitable for hiding. It also links to the idea of somebody…
Shaw’s interactive digital installation, Fall Again, Fall Better, originally made for the Shanghai Biennale in 2012, has seen many iterations, the latest being at the Lichtsicht 7 projection-triennale in Bad Rothenfelde, Germany. (PHOTO PROVIDED TO CHINA DAILY)
…staying in a space where the arms of the law cannot reach them?
Yes, exactly, and to other questions such as, how really safe am I in my safe house?
Would you agree that your other piece in the Art Machines show, the interactive kinetic sculpture called Spatial Pendulum II, is a bit like an ode to the low-tech, somewhat-eccentric contraptions that the English cartoonist William Heath Robinson (1872—1944) — who is invoked in the exhibition — had imagined?
The making of that work began in simplicity and developed into complexity. If I may use a painting analogy, I might say it went from being Kazimir Malevich’s Black Square to Hieronymous Bosch’s Garden of Earthly Delights. The original Spatial Pendulum that was conceived for the Centre for Mathematics in Amsterdam was simply a steel ball flying gently around inside its five-floor-high central atrium, hung from almost invisible wires to the motors that were controlling its movements. I wanted the flight path of this ball to represent mathematical formulas, and for the piece to be an interactive work controlled by the mathematicians who worked in the building. I designed an interface for them so that they could type in their computer code and then see its visualization in this kinetic sculpture.
The technical challenges of this project were more difficult than I had foreseen. At a certain point it became evident that the engineering solution I had come up with would not work. As I didn’t have the funds to develop a better solution I had to abandon the project half way, and just memorialize it on my website.
The recent Art Machines exhibition at CityU that I co-curated with Professor Richard Allen presented a context and opportunity to build a new version of Spatial Pendulum. Also because I was able to collaborate with Joseph Chan, a CityU School of Creative Media alumnus and extraordinarily gifted artist/engineer. He came up with an engineering solution that was quite different from my inoperative original one. This new Spatial Pendulum now has a long rotational arm with a motorized horizontal track that end in a circular device which controls the ball’s up-and-down movement.
Instead of mathematicians having to write algorithms to control the flight of the ball, the new Spatial Pendulum can be operated by any visitor to the exhibition using a simple set of small levers to determine the parameters of its trajectory.
While Joseph Chan was digitally modeling its construction he used different colors to help identify each of the components. Though the original Spatial Pendulum was intended to be all metallic, these colors looked so good that I decided to keep them in the manufacture, giving this complex machine a more Technicolor aesthetic that is mindful of the paintings of Fernand Léger (1881-1955).
… also there’s an element of exposing what’s typically inside a machine to the viewer, inviting them to engage with the intricate processes involved in making the machine do its work, and that, I think, adds an extra layer of meaning to one’s experience of the show.
Background stories to an artwork are particularly important. Media art works can be easily misunderstood if you ae not cognizant of the entire process by which the work came into being. Given the state of the technologies involved, media art is always struggling between constraints and capabilities. For instance, my earliest interactive computer graphic installation Points of View (1983), utilized a first-generation Apple II computer. Its images were monochrome, the resolution was very low and one could only draw straight lines because curves were too computationally expensive in 3D. So I made a work with just a hundred straight lines as if I was writing an epic poem that could only be expressed as a haiku. The resulting work is a transfiguration of those constraints — a struggle with them, a victory over them. And having an understanding of the technological conditions of this work is essential to its aesthetic appreciation.
The virtual world of Shaw’s interactive video installation Legible City covers five square kilometers in Central Amsterdam. The viewer gets a simulated experience of bicycling there at 1:1 scale, and can choose where they want to go. (PHOTO PROVIDED TO CHINA DAILY)
In a recently-held webinar you said an artist does not necessarily have to love the technology they are using, drawing attention to the complex, sometimes even problematic, dynamic shared between artists and the technologies used by them…
It is a love-hate relationship. You love the opportunities technology gives you but the constraints that can be extremely frustrating. Another aspect of media art is that its technologies are changing all the time. As an artist you often get caught up in the technological conundrum that machines may allow you to realize a fantasy but only up to a certain point. So you learn to adapt, or shift, your fantasy in such a way that it becomes compatible with the available technology, and will remain successful within this given framework. But sometimes artists look at the available technology and say it is not yet good enough, so they wait until the next year, or the year after. My experience is that if you put it off to the future you could be putting it off forever. At the same it is quite true that an artwork can be crippled by the unresolved tension between its artistic ambitions and the constraints imposed on it by the technologies it is made from.
How do you see the local government’s pledge to develop Art Tech in Hong Kong?
It’s a good thing for artists who work with technology. Throughout the history of media art there have been many instances of top-down initiatives that helped it to thrive. I experienced this in the mid-1980s in Germany where there was the political vision and financial will to synergize the creativity of artists with German industrial prowess. This led to the creation of the ZKM Center for Art and Media in Karlsruhe — one of the largest and most successful cultural institutions of its kind in the world. This has also become today’s vision because the success of capitalist society is predicated on the need to keep innovating, and innovation is tied to creativity — so “let’s get the artists on board.”
Yet I do believe media artists have a lot to offer, not just in terms of their technological ingenuity but also because artists demonstrate how these technologies can become culturally meaningful. With technology increasingly infiltrating our lives and transforming our culture, we need technological strategies to nurture cultural health — we need artists to signpost how these technologies can be made existentially relevant and aesthetically profound. So I’m grateful for the circumstances where there is an institutional embrace of art and tech, even though the instrumental vision may not exactly be the same as that of the artist’s. But as long as the two are not in conflict, it can be a very fortuitous confluence.
Even as the viewer looks at Shaw’s virtual sculpture, The Golden Calf, video cameras record their image and transfer these as a reflection onto the body of the piece. (PHOTO PROVIDED TO CHINA DAILY)
Do you see Hong Kong artists taking advantage of the push to promote Art Tech by creating meaningful pieces that explore the inter-relationship between art and technology in ways that go beyond creating spectacles?
One of the reasons why I came to Hong Kong 11 years ago as the dean of CityU School of Creative Media was because at that time Hong Kong had already established itself as a city where new media was thriving and there was already a community of artists making very interesting media art. Also, the institutions and festivals promoting media art were already in place, so I felt I could connect and make a meaningful contribution to that ecology.
Hong Kong is one of those wonderful crossroads places where people from everywhere come together. It’s a truly a space of cultural and artistic convergence — a vibrant melting pot — and this is evidenced for instance by the many internationally renowned arts teachers at Hong Kong universities. All the artworks I’m making here are a product of Hong Kong. They are tied to the local context and the narratives of this city, albeit seen from a sāi yàn (Western person’s) point of view.
I think the academic institutions here are very strong. And new cultural institutions like Tai Kwun and M+, and galleries such Osage are vibrant spaces for media art, Vibrancy is important because it enables quality. There has to be a density of experimentation before quality emerges. A lot of people are doing a lot of things, testing the ground through trial and error and learning by looking at the works of others. Even though there is a singularity to what each individual artist makes, in my opinion that’s only a smaller part of their work. The larger part is the way she/he connects with all other artists and with their social, cosmo-technical and cultural contexts. While seemingly diacritical, I think art is better understood as being a “joint venture.”
Do you think the recent changes in the political climate of Hong Kong might affect your work as a curator and academic in terms of freedom of creative expression or control over the material you wish to work with?
There is no doubt that the circumstances have changed over the last year and there is a lot of uncertainty, especially in a community of artists where freedom of creative expression is paramount. What can I say? I love Hong Kong and I want to be as optimistic as I can be. So I look forward to continue to live and work here and contribute to an optimistic future for Hong Kong.
(The interview has been lightly edited for the sake of length and clarity.)
Interviewed by Chitralekha Basu
If you go
Atlas of Maritime Buddhism
Dates: Until October 14, 2021
Venue: Indra and Harry Banga Gallery, 18th Floor, Lau Ming Wai Academic Building, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon Tong
www.cityu.edu.hk/bg/exhibitions/atlas-maritime-buddhism
