I am often asked about the role of technology in my art.
I always begin with an idea — often one I have no idea how to achieve, yet one I can’t get out of my head. At the point I realise the idea is with me to stay, I go looking for ways to bring the artwork into reality. This is a process that involves experimenting with every method I can find — often in unconventional ways that splice various technologies — in service of the art.
I was living in Brazil when I was given the opportunity to return to New Zealand as Artist in Residence at AUT in 2019. Having moved to Brazil in 2015, I had become fixated on the Amazon rainforest. I was captivated by its size, abundant wildlife, unimaginable biodiversity and richness of indigenous cultures. The trees themselves were most captivating to me.
It is no surprise that when I began my Residency at AUT, my first project was to explore the majestic and culturally sacred trees right on our doorstep — beginning with New Zealand’s Waipoua Forest. I went in search of methods to visually capture these trees in 3D.
My initial focus was on scanning Tāne Mahuta, a 45 metre, 2,000 year old kauri tree. Known also by its name of ‘God of the Forest,’ Tāne Mahuta is sacred to Māori. It was an honour to be given the permission by Te Roroa iwi to ascend Tāne in the name of raising awareness of this magnificent tree, whose life is threatened by kauri dieback.
Around this time, in early 2020, I was also given the opportunity to travel to the Amazon to scan the trees there. But as the pandemic unfurled across the globe, my plans were put on hold. I expanded the scope of the Tāne Mahuta project to encompass other New Zealand trees. With the freedom of travel within New Zealand for most of 2020, I was able to continue to collect lidar and photogrammetric data for additional trees including the millennium tree, Taketakerau, a 2000 year old Pūriri tree in Ōpōtiki, and Ratanui, a 1500 year old northern Rātā near Whanganui. This expansion gave me the opportunity to undertake pivotal R&D, as we worked through kinks and perceived limitations in the scanning process.
Our data collection journey began with lidar, a remote sensing method that produces 3D data by emitting pulsed light waves into the sensor’s environment to determine the shape of surrounding objects. As I learnt how to use this technology, I was guided by Nathan Cossar at Envivo who was a real inspiration. But we reached a roadblock. Lidar operates on the basis of determining a finite set of knowable information about the object it is capturing. A tree is anything but finite and knowable — it presents infinite variations because of the way the wind moves. It is also growing and changing form over time, meaning a tree is not the same from one week to the next. As a result, in the lidar scans branches weren’t where they were supposed to be and pieces of the sky were overlaid onto foliage, creating chromatic aberrations that made these images frustrating from a technical perspective, and unusable from an artistic one.
I began experimenting with alternative approaches, including other forms of remote sensing such as multispectral and ultraviolet, and drone-captured photogrammetry. Drones were becoming more compact and easily manoeuvrable, making them significantly better in sensitive environments than they had been previously. This airborne dexterity gave us the ability to capture a combination of video, stills, and spheres of imagery.
Despite now having a high volume of images from every angle, I still did not trust photogrammetry to achieve the high fidelity and photorealism we were aiming for. In the world of photogrammetry, the forest is one of the most difficult things to brick together. It wasn’t until I was introduced to RealityCapture that I saw the possibility for a software to stitch together these images in a way that did justice to the organic visual qualities of light and movement. We would feed the software tens of thousands of unordered photographs and the system would sort these, like a rapid 3D puzzle without seams. We’ve found this is the only software that could match the requirement of how much data we needed to process and the complexity, and after a few years of perfecting this approach, we are starting to get some amazing results.
When I realised I wasn’t going to be able to travel to the Amazon in person, I began working out how I could produce tree scans from the rainforest remotely. Before long, I was introduced to a production company in Brazil called Amazon Film, and was remotely sending a crew into the Amazon to start gathering this library of trees. This team includes a producer, a boat pilot, a drone operator, a director of photography, a technician, a paramedic, logistics and behind the scenes documentary crew, and an ex-military “jungle guide” who doubled as the chef. There is also a data-wrangler, charged with getting the huge amounts of data onto hard drives which are then sent to me in New Zealand.
I have come to love this process of learning, innovating and problem-solving. But it’s important to stress that these methods are all in service of the idea we are trying to achieve. The reason this method has stuck is that it serves the artwork. As a photographer, I am keen to capture the spirit and mood of these magnificent trees, the fact they are living and breathing, and the way they are responsive to air, light and other trees. For example, if I wanted to capture the foliage in the soft morning, I could decide to take the photos in the morning. In this way, I could bring the technology into the artistic realm.
As the technology continues to mature, myriad pathways of possibility are unfolding. While some of these pathways lead to ambiguity around the kind of world we are creating, others present real historical and cultural value around the digital twinning and archiving of nature. Against the sobering backdrop of the climate crisis, the enforced stillness of the past year has revealed the true extent of the growing disharmony between humans and the natural world. As we begin to recognise the pivotal role of indigenous wisdom and stewardship of nature, I am beginning to experiment with visualising elements of the forest that we can’t see. Developing this unique scanning method has raised the level for me and my practice, bringing me to a place where I am butting up against the true frontier of what we’re capable of creating in the digital realm.
The early stages of creating Amazon — Raised Up Sky using these technologies are captured in our short documentary, directed by Jen Raoult.