NEURAL MATERIALS

A love letter made from the materials of a post industrial city

A study of the rhythm and noise of neural synthesis via the sonic language of cotton mill machinery, urban noise, and canal network waters

A system for sound sculpture, modular electronics, and machine learning

NEURAL MATERIALS is a new live AV show created by SONAMB, in collaboration with Sean Clarke on visuals. The piece debuted at Modern Art Oxford in spring 2024 and has been subsequently performed at the Creative Machines symposium at the Royal Holloway University. The piece is the result of a two year R&D commission by Cyborg Soloists, a UKRI funded project at Royal Holloway University led by Dr Zubin Kanga. Cyborg Soloists is a 4 year programme partnering artists and digital technologies to create new work. I worked with Bela technologies who created the Bela board, an ultra low latency sonic microcontroller for musicians and sound artists to create interactive artworks. I used their Pepper module as part of my Modular set up, along with their Trill sensors for gestural interaction.

Video: Live set at Modern Art Oxford 25/04/24

BACKGROUND & PROPOSED SYSTEM  

The project is a further exploration of Machine Learning and Musique Concreté, building upon my 2021 residency with NOVARS, University of Manchester (in collaboration with PRiSM, RNCM) where I developed methodologies for building Musique Concreté training datasets for neural synthesis training. 

For NEURAL MATERIALS I created a new performance system for AI, modular electronics and sound sculpture; a hybrid setup, with Ableton linking to the modular. With the Bela technology I used their PEPPER module for sample manipulation, controlled via their gestural sensors called TRILL. The aim was to be more hardware focused for live, and explore the ML materials more tangibly through gesture and feedback. To develop sound content for the project I undertook a series of studio experiments and workflows. 

Image: Proposed Performance System

SONIC DATASET & RECORDING

My sonic dataset for this project is ‘post industrial’ with each data class containing field recordings that represent a different material that tells the story of Manchester’s industrial past and present. Beginning on the outskirts of the city with cotton mill machinery at Quarry Bank Mill and journeying inwards towards the centre through the canal networks, we finish in the (property booming) city of luxury mill apartments and the shiny tower blocks of modern development and gentrification. This journey is represented by sonic data classes of cotton, water and noise.

Image: Data classes for Post Industrial dataset

SONIC EXPERIMENT EXAMPLE ONE: COTTON MILL DATA CLASS RECORDINGS & RHYTHM

Recording_ For the ‘Cotton’ class of field recordings, I was thrilled to record the machinery at Quarry Bank Mill. Built in 1784, it is one of the best preserved textile factories of the Industrial Revolution, now owned by the National Trust. I was privileged to gain access to the Cotton Processing Room and the Weaving Shed, to record the unique sounds of the machinery using a mixture of directional and stereo mics to record the looped rhythms, unique timbres and drones, and the power up and down of these amazing historical artefacts including the Draw Frame, the Slub Roving machine, the Carding Machine, the Ring Spinning Frame and the Lancashire Loom. My Dad trained as an apprentice electrician here in the 70s, cycling from Hulme to Styal everyday, so it’s a special place for me.

Pattern Generating_ The dataset was used to train the PrismSampleRNN model. Post training I was interested in what ‘future rhythms’ the ML model would generate. Through feature extraction would rhythms collide and be sonically interesting or would they replicate the staccato analogue loops in the dataset? In the Ableton still below, I am analysing and comparing rhythmic patterns from the original machine field recordings with the exported ML audio post training. You can see the audio is classified by each epoch, meaning the amount of times the model has trained on the full set of data. You can also see the ‘temperature’ setting which is a hyper-parameter of ML used to control the randomness of the predictions in latent space.

I enjoyed listening into the sounds seeing if I could identify particular machines from the dataset. Using a midi-drum conversion was the clearest way to listen back to the rhythmic material, which was eclectic and brilliant non-sense that a human drummer could not play, you can see this from the velocity hits in the roll at the bottom. Sending these pulses as CV triggers to the modular and looping the ML sections to locate areas I wanted to work with. In this audio clip you can hear the different generated machine rhythms falling in and out of each other.

SONIC EXPERIMENT EXAMPLE TWO: GEOMETRIC SCULPTURE & TONAL ASPECTS

I’m working with a new metal sound sculpture, another in my series of AURA MACHINE icons ( a visual symbolic language i’ve developed to represent the sound object in latent space.) I’m interested in the acoustic potential of these metal sculptures and how these transmutational objects can be amplified, activated and fed back into the system for live performance. I mapped the frequency response of the physical sculpture at different points across it’s geometry using contact microphones positioned on each plane. There was a clear fundamental on each plane but also evident are harmonics and partials that recur throughout the object. Here is the 3D model with the mapped frequencies.

SONIC EXPERIMENT EXAMPLE THREE: MODULAR DIY_ BELA PEPPER & EMF FEEDBACK

Using the Bela PEPPER module, we wrote a PureData patch that would control the playback and manipulation of the AI cotton mill samples via touch. We used two TRILL sensors to do this, the square one controlled the volume and speed on an XY axis, and the circle sensor controlling a granulated loop through touch. The sensors were fitted to my sculpture and used for live improvisation throughout the set. Another DIY aspect of the modular setup included working with my EMF circuit, which sampled and amplified the live electricity of the modular and fed it back into the system. We designed and laser cut three bespoke units;  a breakout board for the TRILL sensors, a CV controlled light reactive unit and a holster for the EMF circuit. Special thanks to Chris Ball and Luke Dobbin for helping me with the coding on this!

AUDIO REACTIVE VISUALS WITH SEAN CLARKE

I collaborated with digital artist Sean Clarke on this project, who expertly took my visual source materials and crafted them into a new audio reactive system for live performance using Touch Designer. I love working and performing with Sean, his artwork has so much feel and energy and he conveyed the sense of the materials in the dataset and the moods and changes I was trying to evoke.

NEXT STEPS

I am still experimenting with this new performance system, the shows so far were the first outings for the piece and I plan to keep experimenting with the sculpture (especially acoustically) and develop the improvisational aspects for any new NEURAL MATERIALS shows. In 2025 I plan to develop my practice into the realm of spatial installation drawing on my methodologies and research from the past 5 years in sonic AI. More to come!

THANK YOU TO MY COLLABORATORS AND SUPPORTERS!

Cyborg Soloists, University of Holloway: Zubin Kanga, Caitlin Rowley, Mark Dyer, Jonathan Packham

Sean Clarke: Live Visuals

Chris Ball & Luke Dobbin: PureData coding support

Nik Void : Music Mentor

Quarry Bank Mill: Suzanne Kellett and team!

PRiSM: RNCM for neural network training

Flux Metal & Neon Creations: Sculptural Fabrication

Dan Hulme, EMPRES, University of Oxford

NEURAL MATERIALS is dedicated to my inspirational Dad, Vince Clarke who passed away July 2024.

NEURAL MATERIALS_ CYBORG SOLOISTS R&D COMMISSION

I am currently undertaking R&D for ‘Neural Materials’ a new commission for Cyborg Soloists

NEURAL MATERIALS explores the materiality of neural synthesis through a performance system for sound sculpture, electronics and machine learning. My research will explore potentials for gestural and expressive acoustic interactions with metal AI sculpture and neural synthesis. I’m working with Bela sensor technology from the Augmented Instruments Lab, building a ‘Pepper’ unit to integrate with modular electronics. I’ve been creating a post industrial musique concrete dataset of field recordings (cotton / water / noise)from Quarry Bank Mill machinery, Manchester Canals and City centre noise of Ancoats and Deansgate ‘square neighbourhoods. for ML training.

Quarry Bank Mill, cotton machinery

Cyborg Soloists is a UKRI-funded Future Leaders Fellowship project led by Director and Principal Investigator Dr Zubin Kanga. Hosted at Royal Holloway, University of London, it explores interdisciplinary interactions between music, the other arts and new digital technologies.

Follow my instagram for sounds, tech experiments and research progress.

My research will culminate in a live AV performance this September at Testcard.

Ancoats, noise recordings EMF
Castlefield canals,hydrophone recordings

SLEEPSTATES.NET

>>>ENTER SLEEPSTATES.NET >>>

SLEEPSTATES.NET is a browser-based work where the user navigates to different states of algorithmic slumber, a sleep-cycle of audiovisual interactions that takeover their screen. Exploring themes of machine addiction and autonomy, sleep sanctity, the networked condition and how citizens are subjected to emotion extraction via platform capitalism, these dream transmissions journey through the techno-emotional states of slumber we experience between humans and machines late at night. This piece accompanies SLEEPSTATES, the debut album from SONAMB.

Users experience AV screen-takeovers, interactive pop-up provocations and monitor their slumber status and machine addiction levels. They are asked to join SONAMB in defending sleep by ‘transmitting their frequency’ and breathing into the machine via an audio reactive noise-field, contributing to a collective citizen-white-noise-sleep-aid.

SLEEPSTATES.NET began in lockdown through late night internet explorations learning web coding and visual machine learning whilst experiencing sleep deprivation and questioning my mental health in relation to the machine. Documenting techno-emotional states of anxiety, addiction and inertia through audiovisual vignettes and code sketches, I sought to create a free space to commune collectively with the machine, an alternative to interfaces of power, data extraction and capitalism.
The piece takes inspiration from 24/7, Jonathan Crary (sleep=the last defence against capitalism) and MIT’s Dream Lab( sleep sound implantation). As a child of the 80s/90s, I grew up surrounded by computers as my Dad was a programmer. I was too young to take part in early utopian net communities but I do remember the aesthetic like a dream. 

SUPPORT:The piece received R&D support via a remote residency for Manchester International Festival during lockdown, followed by creative web design mentoring with Studio Treble supported by Arts Council England. 

PUBLIC SHOWINGS: The original SLEEPSTATES music video appeared at MUTEK Distant Arcades exhibition and online for Amplify festival at Somerset House. The artwork was shown at the 2023 NFT Biennale as part of the Sound Obsessed New City archive.

SLEEPSTATES.NET PART OF SOUND OBSESSED SONIC INNOVATION ARCHIVE AND NFT BIENNIALE 2023

I’m thrilled to be one of the artists in the Sound Obsessed Sonic Innovation Archive, in collaboration with Refraction Festival and New Art City. We are an international collective of artists, musicians, scientists and creative technologists, and this evolving archive celebrates the journey, milestones, and challenges of sonic innovation.

SLEEPSTATES.NET is showcased as part of this curated online space exhibiting AV works by artists working across generative, creative AI and mixed media. Exploring web3.0 with amazing sonic innovators, coders and scientists. Head over to https://www.soundobsessed.com/ to view works.

The exhibition is touring to the ‘worlds first art’ NFT Biennial in January and February 2023.

Special thanks to PortraitXO for inviting me to be involved ❤️💙❤️💙

BBC 6 MUSIC FEATURE_ AFRODEUTSCHE_THE PEOPLE’S PARTY

On Friday 2nd December 2022, I was thrilled to feature on Afrodeutsche’s The People’s Party show on BBC 6 Music showcasing my debut album SLEEPSTATES and taking on the unmixable challenge (Foals & Daniel Avery.) The feature included a first radio play of album track, Sync Whole and a behind the scenes chat, big thanks to Afrodeutsche for giving my work this platform.

recording:

SLEEPSTATES ALBUM REVIEWS_ BOOMKAT & ELECTRONIC SOUND MAGAZINE

BOOMKAT review

ELECTRONIC SOUND MAGAZINE review

“The debut release from Manchester sound artist and Oram Award Winner Vicky Clarke (aka SONAMB) takes the listener through the dream states that are a product of our interaction with analogue, algorithmic and digital technologies. Drawing on random frequency snatches (‘Sleepstates’), disquieting textures (‘Sync Whole’) and skittering somnambulist pulses (‘Hypnabyte’), the album skirts around the edges of lucid dreams that characterise REM sleep- shaped, manipulated and interrupted by what Clarke describes as the noise of “machine addicition”. A stunning collection.”
MJ

AURA MACHINE RESEARCH SITE_ MACHINE LEARNING & MUSIQUE CONCRETE RESIDENCY_NOVARS_UNI OF MANCHESTER

Homepage AURA MACHINE research website

The AURA MACHINE research website is now live. The platform collates the research and creative outputs from my artist residency over the past two years with NOVARS, centre for innovation in sound at the University of Manchester, where I have been exploring the collision of musique concrete and machine learning. The residency was supported by European Arts Science Technology Network for Digital Creativity and in collaboration with PRiSM Lab (Practice Research in Science and Music) at the Royal Northern College of Music and was kindly supported by Arts Council England.

Framed by my research question “how can concrete materials and neural networks project future sonic realities?” , the site sets out the critical rationale for exploration, could machine learning, specifically sample based neural synthesis, be a tool for contemporary musique concrete?

Throughout the research I developed skills in building sonic concrete datasets, technical knowledge of neural synthesis and methodologies for listening, composition and performance when working with these materials. The musical output was in the form of my piece AURA MACHINE, whose first iteration was a 10 minute AV piece showcased at FutureMusic#3 event at RNCM.

AURA MACHINE, AV piece FutureMusic#3 RNCM

‘What happens to the sound object when proessed by a neural network?
‘A sound object has an AURA, can a machine generate an AURA?

AURA MACHINE

The piece took the listener on a journey through training a neural network, starting with the raw concrete materials (representing the sonic dataset), through to transmutation (model training) to the purely generated AI materials. Conceptually the piece played with both Pierre Schaffer’s idea on the sound objects and Walter Benjamin’s ‘The work of art in the age of mechanical reproduction’ with the idea of a piece of art being authentic to a time and place’. This AV piece was further developed as a live piece into a 20 minute live performance with sound sculpture, audio reactive visuals (collab with Sean Clarke) and performances at Manchester Science Museum and a lead feature for PatchNotes, Factmag.

Patch Notes, FACTmag, live performance AURA MACHINE

The residency was also supported by Arts Council England, this meant I could also develop new systems for algorithmic sound sculpture and circuit designs combining the creation of an ancient alchemical dataset (I found many parallels between alchemy and modern day ML) for visual generation and contemporary prototyping tools including 3d geometric design with Blender and 3D printed models before professional fabrication.

AURA#2 NEON glass sculpture
AURA#1 STEEL sound sculpture

One aim of the residency was to demystify machine learning, break out of the hidden black box and through examining both the inherent barriers and the current accessible tools, share insights and approaches for other artists to create with sonic AI. I created ‘DREAMING WITH MACHINES’ an educational project for young women with Brighter Sound to support others in using these tools and techniques. We looked at the technical history of AI, inherent bias in datasets, the positive aspects of the turn of creative AI and its potential to imagine future societies and realities. We discussed our relationship with machines and the participants created their own datasets and audio visual pieces which premiered online as ‘dream transmissions’ and were showcased at the Unsupervised#3 event at University of Manchester.

SLEEPSTATES DEBUT ALBUM_NOV 2022

>>dream transmissions from the electrical imaginary>>

SLEEPSTATES is the debut release by SONAMB, the new music project from Manchester sound artist Vicky Clarke, released November 2022. Exploring machine addiction, broken radio transmissions and algorithmic sleep territories, the record takes the listener through a cycle of dream transmissions from the electrical imaginary; the techno – emotional states of slumber we experience between humans and machines late at night.  

‘a glitchy experimental techno jerker’

Boomkat review
SLEEPSTATES album artwork

Created over the past three years via networked locations, sonic states combine transmission recordings between Manchester, Berlin and St Petersburg and lockdown internet noise experiments, bringing together her processes of sound creation with DIY electronics, sound sculpture, musique concrete and machine learning. The record explores the sonic materiality of our technologies, the sanctity of sleep and ideas of autonomy and control as we meld into our machines, blurring conscious boundaries of our waking and networked selves.

Reception: The album was described as a ‘glitchy experimental techno jerker’ by Boomkat and ‘a stunning collection‘ receiving a full page feature in Electronic Sound magazine >>Read both reviews here >> Sleepstates was featured on Afrodeutsche’s The People’s Party show on BBC 6 Music, as SONAMB took on the unmixable challenge and played album track Sync Whole.>>Listen to feature>>. Album tracks also received airplay on NTS, Resonance FM, Camp Radio and Repeater radio.

SLEEPSTATES.NET: The release is accompanied by SLEEPSTATES.NET, a net-art piece featuring music from the record. 

Credits: All tracks are written, produced and mixed by SONAMB, except track 7, mixed by Francine Perry and tracks 2 & 9, co-mixed by SONAMB and Francine Perry. SLEEPSTATES is mastered by Katie Tavini, Weird Jungle.

Listen: The album is self-released on Control Data Records, and is available on tape cassette and digital download direct from Bandcamp and Boomkat.

Circular Sferics_ East St Arts residency

For the East St Arts / Convention House residency I am undertaking research for my project ‘Circular Sferics’. This will explore the duality of our technologies being both of the earth and used as detection devices for measuring, through creating a circular technological approach using sound and sculptural process. I’m using DIY environmental sensing technologies to record sounds of the natural environment using self built radio antennae, and my AURA MACHINE VLF and EMF detector circuits. 

The recordings will focus on sferics (atmospherics), the natural electrical phenomena detected by radio antennae, which will be analysed as data to develop a methodology of visualising these as 3D geometries. Working at Convention House I will learn new skills in 3D ceramic printing and 3D scanning to create new sculptural objects  – made out of clay – completing the circle as new material forms. 

I am looking forward to working with clay, the original technological material and representing unseen sferic phenomena as physical matter. The cyclical format of the residency explores themes around the circular economy, deep time and technological e-waste through sonic materiality. I will also be running an electronics workshop, developing my knowledge around environmental sculpture through mentoring and I look forward to getting involved and contributing to the Convention House community.

BACKGROUND

The work is inspired by my research and reading into natural radio and environmental sculpture. In lockdown I undertook the Radio x Rectangle course, learning to build short wave radios, crystal oscillators and antennae, in tandem I got interested in listening to natural electricity through reading Earth Sound Earth Signal by Douglas Kahn, A Geology of Media by Jussi Parikka and was inspired by the great Alvin Lucier’s experiments in the 80s recording sferics.

PHASE ONE: FABRICATION | ELECTRONICS | RECORDINGS

For the first stage I have been collaborating with the wonderful Daniel Simpkins (and fellow Rogue Studios artist) at KUNSTRUCT, who worked with me to fabricate and realise my designs. We began with the material, opting for a dark grey Valcrohmat with yellow flecks, an ethically sourced wooden composite that when varnished takes on a look of stone. Dan meticulously hand crafted each block for the sculpture, the forms of which were algorithmically generated (this sculpture is AURA#3 Wood, the third in the series of my aura machine sculptures developed via my methodology of neurally generating sculptural works) and then further designed by myself. The sculpture needed to be functional also with in-built speaker chamber, a detachable loop antenna and as an outdoor piece it needed to be mobile for recording in fields.

The electronics and interface for the piece encompass a VLF (very low frequency) circuit, a patchable mixer where you can opt to plug in the EMF detector and stereo amplification through a bass speaker and two mids. There is a detachable loop antenna and for grounding a large earth spike, I worked with long term collaborator Creative Technologist Chris Ball who helped me with circuit amplification and testing at Yorkshire Sculpture Park for our first recordings, which attracted a fair few cows! You can listen to these first test recordings below.

recording at Yorkshire Sculpture Park, August 2022

Thanks to Yorkshire Sculpture Park and Invisible Flock for hosting me for these first recordings, and to Shazia and Matt at East St Arts for their ongoing support.

In phase two I will be translating these sounds into 3D geometries for new sculptural ideas and working with the clay 3D printer in the Convention House maker space.

This work has been supported by East Street Arts and explored at Convention House in Leeds.

PATCH NOTES_ FACTmag performance

FACT magazine invited me to present a live performance of AURA MACHINE for their legendary PATCH NOTES series. Directed by Pedro Kuster and filmed at 180 Studios, The Strand, this special film is the latest AURA MACHINE collaboration with Sean Clarke. This performance represents the final piece in my musique concrete and machine learning residency and experience with NOVARS, University of Manchester. It was wonderful to showcase the visuals we made with Sean on a fantastic LED screen and the film really captures the materiality and mood of the sonic machine learning textures and movements throughout the piece. Huge thank you to Pedro and the whole team at FACT, a huge privilege.

You can watch the entire episode of Patch Notes and read the full article on FACTmag.com

What happens to the sound object when processed by a neural network? The AURA MACHINE piece takes the listener on a journey through training a neural network, starting with the concrete training data – through to transmutation and training the model and ends with the purely generated AI output.

Here are some clips from the 20 min set.