Stu Favilla. Juried Artist
The Edge, State Library of Queensland
The Edge, State Library of Queensland
Artist Statement
Multisensory science books leverage the lived experience of a legally blind artist, musician/interaction designer, an industrial designer and a laboratory of scientists, to communicate cutting-edge biomedicine discovery through exhibitions for blind, low vision and diverse needs audiences. Multisensory science books are a portable exhibition format enabling inclusion, through visual and tactile design, novel technologies, interactions and experiences, music, audio design, and sonification.
The advent of COVID lockdowns and exhibition cancellations led to the creation of the unique format. The books were created in A3 size format with laminated pages on a foam core base and include tac-tile artworks, large print, braille supplements, software tracking optical fiducials (reacTivision, Jordà et.al, 2007) narration, music and sonification.
Tactile artworks throughout the books include ‘braille’ inspired molecular structures, and tactile depictions of immune cells, viruses, bacteria, and the digestive system. A combination of handmade cell sculptures appears throughout with audio text to accompany each topic and associated artwork.
Sensory science has shifted from multimodal representation to multisensory representation engaging the sensory memory and perceptual understanding that all of us carry from childhood through to our current experiences of daily living. Experience has shown that 3D printing is not the only way to communicate tactile information. Using food, found objects and a range of other materials allows for the sense memory of participants to be accessed and deeply engaged.
The message is in the material! This may be a simple metaphor such as covering a kidney sculpture with dried kidney beans or using burnt coffee grounds to represent carbon atoms in a braille-inspired molecule model. Plant stems, buds, rhizomes, feathers, bark, pasta shapes, and grains can all depict complex structures and interactions, or present well-established sensory metaphors for function that people understand immediately. 3D printing is amazing, but it lacks a sensory fidelity that can be represented by found objects.
A major component of the books was the use of interactive sonification of tactile artworks and electronic music. A sonification method was developed to display 2D art and images into sound utilizing MaxMSP/Jitter. The books were designed to sit on individual inclined (30-degree) stands. The book stands also included headphones and Mac mini computers driving the software interactions. The project generated around seven hours of original music, illustrating and representing protein and cellular interactions and themes. The protein and immune-system-inspired music was composed using modular electronic synthesisers.
The books have sought to bring blindness, low vision and diverse needs groups recognition and understanding through designing specifically for their needs. What the participating designers, scientists and researchers have all discovered through their involvement, is that multisensory design also en-gages sighted people and those living without disabilities extremely well. During exhibitions, understanding and empathy are also shared. As the barriers that exist between vision-based information and blind and low-vision people are removed, so too are the social barriers and fears that exist across society.
About the artists
Dr Erica Tandori is a legally blind artist, researcher, creative academic and artist in residence at the Rossjohn Lab, Biomedicine Discovery Institute, Monash University. There she has been directing the Sensory Science outreach program for blind, low vision and diverse needs audiences. This unique program communicates cutting-edge scientific discovery through co-creation workshops with scientists and also through a range of innovative exhibitions. She holds a PhD in Ophthalmology and Art describing the en-toptic effects of macular dystrophy. With lived experience of blindness, Erica also works as an inclusive technologies designer with the Monash Inclusive Technology lab developing accessibility frameworks for blind and low vision audiences. Her research and practice, explore new ways to heighten accessibility for visual and cultural heritage for diverse needs people. Erica’s multisensory science exhibitions have garnered national and global recognition including the Eureka Prize (finalist) and the Berlin Falling Walls Breakthroughs in Science, where she was a finalist in four separate categories.
Audio/interaction designer Dr Stu Favilla has been recognised internationally for three decades, contributing to computer music, interaction design and digital musical instrument design. Winner of notable international prizes including the Karl Szucka Preis and finalist for the Bourges Electroacoustique Prize, Stu Favilla has developed digital music instruments and interactions rich in creative possibilities and skill-developing affordances. His work has been profiled by the Qantas in-flight magazine, International and Australian Television, featured by the DK Books’ Cool Stuff series and in numerous academic books including the recent publication “Body as Instrument” by Mary Mainsbridge (Bloomsbury, Academic). He has per-formed complete concert programs for the European main stage including Festival deBase (Basel Art Fair, Switzerland), Festival Resonances (Pompidou, Paris), AudioArt (Krackow) and twice at Earzoom (Ljubljana). A specialist in software development and audio and video signal processing, Stu was a silver medalist at the 2020 International Design Awards for his efforts in developing data-driven and generative-life video wall projections and public exhibitions. An innovator in the area of data sonification, spatial sound, and immersive audio, Stu currently heads the Swinburne Sonic Research group.
Dr. James George Marshall is a professional designer, academic and software developer specialising in emotion design. James has over 30 years of commercial experience in the design and development of physical products, games, and interactive systems. His company Education Network Group develops sci-ence-based educational products. These include Whizkid Games, which helps autistic children to develop independent living skills and Science Island which promotes STEM learning for children globally. James is a senior lecturer of design at Swinburne University of Technology where he teaches into the Master of Design and Architecture programs. He has received an Australian Learning and Teaching Council (ALTC) Citation for Outstanding Contributions to Student Learning and three Swinburne Vice Chancellors’ awards for teaching excellence and community engagement. Marshall’s research incorporates state-of-the-art concepts from design, artificial intelligence, and neuroscience to model, create and evaluate systems involving the interaction of people and technology. This research is applied in his Principle-led Design Toolkit, which facilitates the creation of Wonderful Design – from Primary to PhD. His design work is internationally recognised and awarded. In 2010 he won the overall best commercial design, across all categories in the Biennial Victorian Premiers Design Awards. The same year The Age newspaper, named James George Marshall in their list of Melbourne’s “Top 100 influential people”.
Credits: Dr Erica Tandori, Dr Stu Favilla, Dr James Marshall