AI as Catalyst:
Transformative Impacts on Digital Performance, Computer Music, and Cultural Creativity.
FCT Grant 2024.09158.CEECIND | 2026–2029 | INET-md | University of Aveiro
Principal Investigator: Hugo Paquete, PhD
AI as Catalyst is a CEEC‑FCT artistic research project exploring artificial intelligence as a creative co‑agent capable of generating emergent behaviour in digital performance and computer music. Developed at INET‑md, University of Aveiro, the project investigates the technological unconscious — the hidden infrastructures and algorithmic systems shaping contemporary experience.
At its core is the concept of the AI‑Chimera, a hybrid entity formed through the interaction of humans, algorithms, and real‑time data. Through three case studies — Hyperobject Soundscapes, AI Remix Collective, and Glitch Ecology — the project builds sonic ecologies where authorship, control, and intention are redistributed across human and non‑human agents.
AEROSONIC SONIFIER: The Atmosphere as a Dreamlike Archive
AEROSONIC SONIFIER is an unsupervised AI system that transforms real‑time meteorological data into musical behaviour. The atmosphere becomes a dreamlike archive — read, metabolized, and manifested through digital resonance, meta‑listening, and hyperobjectal composition.
The system operates through a 1:N transduction paradigm: a single data-event radiates across multiple sonic layers — harmonic tension, rhythmic density, granular texture, spatial diffusion, and affective state.
● Active — Prototyping → PerformanceThe AI-Chimera Framework
The system is grounded in the AI-Chimera, an ontogenetic computational entity defined as:
X = {B, M, A, E, Be}
- Body (B): The operational materiality — data streams, transformation processes, surfaces of manifestation
- Metabolism (M): The interpretative transduction layer that converts processed data into internal behavioral states
- Environment (A): The ontological condition of existence — external data, periodicities, constitutive noise
- States (E): Internal configurations of tension and transitional potential
- Emergent Behavior (Be): The dynamic manifestation resulting from the interaction of all previous components
Intelligence, within this framework, is stripped of anthropomorphic cognition and defined operationally as differential sensitivity to the environment:
I = ∂Be / ∂A
System Architecture
The system architecture operates across three integrated layers:
- Data Intake Layer: WeatherClient aggregates data from Open-Meteo, NOAA/NCDC with 7 normalized variables (temperature, wind speed, precipitation, pressure, humidity, and derived features) and location-based sonification (POLAR, TROPICAL, SUBTROPICAL, TEMPERATE)
- AI Intelligence Core: Isolation Forest (unsupervised anomaly detection), DreamingAI (affective computation), QuantumState (probabilistic collapse), DataMutator (stochastic transformation)
- Sonic Generation Layer: HarmonicField (12 modes), DroneManager (5 voices), RhythmVoice (Euclidean rhythms), RainVoice (granular synthesis), CCBank (8 MIDI controllers)
Key Modules
- Isolation Forest: Learns "normal climate" for a location; triggers glitch state on anomaly detection (score > 0.3)
- DreamingAI: Maintains 6-dimensional emotional vector (Longing, Curiosity, Ecstasy, Melancholy, Fear, Serenity); enters 13 dream archetypes (Crystalline, Amethyst, Obsidian, Inferno, Gravity, Nebula, Thunderstorm, Aurora, Volcano, Mirage, Abyss, Whirlwind, Stardust)
- QuantumState: Simulates quantum particle with configurable collapse probability (up to 15%); triggers arpeggiated patterns on collapse
- DataMutator: 6 transformation types (warp, scramble, gravity, glitch, fractal, chaos) driven by logistic map where r is modulated by the global chaos coefficient
Hardware — The Ontogenetic Body
The AEROSONIC SONIFIER hardware is a 16-key atmospheric interface functioning as the ontogenetic body of the system. Each icon represents a perturbation vector — a force capable of reorganizing the internal states of the computational organism.
The device does not control the system — it co‑participates in its emergence. It does not translate the weather — it dialogues with it. It does not execute functions — it provokes states.
Technical Documentation
- Lines of Code: 30,000+
- Modules: WeatherClient, AnomalyDetector, DreamingAI, QuantumState, DataMutator, HarmonicField, DroneManager, RhythmVoice, RainVoice, CCBank, MidiProxy, Visualizer
- Data Mapping: 1:N Transduction
- Operation: Resonance
- Ontology: X = {B, M, A, E, Be}
- Funding: FCT 2024.09158.CEECIND
- Status: Active — Prototyping → Performance
Code Example — WeatherClient Module
# Conceptual extract — AEROSONIC SONIFIER class WeatherClient: def __init__(self, location_type='temperate'): self.location_type = location_type self.baseline = self.establish_baseline() def get_forecast(self): # Not just fetching—interpreting raw_data = self.fetch() return self.contextualize(raw_data) # Simplified conceptual logic — DreamingAI class DreamingAI: def __init__(self): self.emotions = {'longing': 0.5, 'curiosity': 0.5, ...} def evolve(self, weather_data): for emotion in self.emotions: # Differential equation with decay self.emotions[emotion] += self.forcing(weather_data) - self.decay() if self.threshold_exceeded(): self.trigger_dream_archetype()
Theoretical Overview — AI as Catalyst
This video presents a theoretical introduction to the core concepts of the AI as Catalyst project, generated using Notebook LM. It explains the AI-Chimera framework, 1:N transduction, and the philosophical foundations of sonic infrastructuralism in an accessible format.
Audio explanation generated using Notebook LM — theoretical overview of the AI-Chimera, 1:N transduction, and sonic infrastructuralism.
HELIOSONIC SONIFIER: Listening to the Sun
HELIOSONIC SONIFIER extends the AEROSONIC framework to solar and heliospheric data — sonifying solar wind, sunspot activity, and coronal mass ejections into musical behaviour. The system explores planetary‑scale phenomena through expanded audiovisual intelligence, rendering audible the invisible forces that shape our solar system.
Using real‑time data from NOAA/SWPC (Bz, solar wind speed, proton density, G/S/R alerts), the HELIOSONIC Sonifier translates the dynamics of space weather into expressive sonic behaviour. Each solar event — from a gentle solar wind fluctuation to a violent coronal mass ejection — becomes compositional material, generating emergent responses across harmonic tension, rhythmic density, and spatial diffusion.
Data Sources
- NOAA/SWPC — Real‑time solar wind data (Bz, speed, density, temperature)
- GOES-R — X‑ray flux (solar flare intensity)
- Kp-index — Planetary geomagnetic activity
Technical Specifications
- Data Mapping: 1:N Transduction — each solar event radiates across harmonic, rhythmic, and spatial layers
- Latency Compensation: Predictive models compensate for satellite data delays (5‑15 minutes)
- Hardware: Extension of the AEROSONIC 16‑key interface with solar‑specific iconography
- Status: In Development — Prototype expected Q1 2027
The Solar-Chimera
The HELIOSONIC system extends the AI-Chimera framework to operate at planetary scale. The Environment (A) becomes the entire heliosphere — the vast magnetic bubble that encloses our solar system. The system listens to the sun, metabolizes its emissions, and manifests them as sound.
This is not a translation of solar data into sound. It is a resonance — a computational coupling where the system's internal states oscillate at frequencies determined by solar forcing. The sun becomes a performer, the system becomes a listener, and the audience becomes a witness to the invisible forces of our star.
Cyber Attack Sonifier: Glitch Ecology V01 — AI Router Edition
● Active — Prototype
Cyber Attack Sonifier: Glitch Ecology V01 extends the AI as Catalyst research into the domain of cybersecurity infrastructures, revealing the hidden flows of global digital conflict as sonic and visual ecologies. The system transforms real‑time threat‑intelligence data into dynamic audiovisual behaviour, exposing the rhythms, tensions, and emergent patterns of the contemporary networked world.
Developed as an integrated AI‑driven environment, the Sonifier combines neural pattern generation, genetic rhythm evolution, probabilistic harmonic fields, and real‑time visualization. Cyberattacks — scans, exploits, botnets, ransomware, command‑and‑control signals — become compositional material, producing emergent behaviour across rhythm, harmony, density, and microtonal gesture.
Operating as a post‑digital ecology, the system redistributes agency between human and non‑human actors. Each attack modifies the harmonic field, alters rhythmic probability, and reshapes the audiovisual landscape, generating a continuously evolving environment of tension, drift, and algorithmic memory.
Data Sources and Collaborations
A special acknowledgement is given to Shodan, which collaborated in providing access to its premium API. Shodan's global search engine for vulnerable devices enables the system to incorporate high‑value data on exposed infrastructures, enhancing the accuracy and depth of the sonification.
Conceptual Framework
Within AI as Catalyst, the Cyber Attack Sonifier expands the project's core concepts:
- AI‑Chimera — hybrid entities formed through human–algorithmic interaction
- Algorithmic Listening — distributed listening across systems, data, and machine inference
- Meta‑Production — emergent creation through real‑time computational processes
- Techno‑Animism — treating digital infrastructures as living, reactive agents
- Hyperobject Soundscapes — sonic manifestations of large‑scale, non‑human phenomena
- Technological Unconscious — revealing hidden algorithmic and infrastructural flows
Artistic and Research Impact
The system functions simultaneously as:
- a performance instrument
- a real‑time data‑driven composition engine
- a critical visualization of global cyber ecologies
- an experimental platform for AI‑mediated music
- a research tool for exploring algorithmic infrastructures
By converting digital conflict into sound, the Sonifier positions cybersecurity as a sonic hyperobject — a vast, distributed phenomenon rendered audible through computational ecologies.
Keywords
AI‑Chimera · Algorithmic Listening · Meta‑Production · Hyperobject · Techno‑Animism · Generative Music · Data‑Sound · Technological Unconscious · 4D Spatialization · 1:N Transduction · Sonic Infrastructuralism · AEROSONIC · HELIOSONIC
Research Journal: AI as Catalyst — Living Archive
Documenting experiments, prototypes, failures, and discoveries in real time
Listen to the prototypes in action
Performances and sound examples will be released throughout 2026–2027.
Follow the research journal for updates.