Hybrid Ghosts: An Album of AI-Generated Folk Instrument Crossbreeds

Hybrid Ghosts is a speculative project that aims to create a new acoustic vocabulary by digitally crossbreeding folk instruments from around the world. The core idea is simple: take recordings of a sitar and a didgeridoo, let an AI learn their sonic essences, and generate a hybrid that sounds like neither but retains the soul of both. The project would begin by amassing a diverse collection of samples—at least 1,000 recordings spanning every continent, from the mbira of Zimbabwe to the hardanger fiddle of Norway. These samples would be preprocessed (pitch-normalized, trimmed, labeled) and fed into a variational autoencoder inspired by Google's NSynth. The model learns a latent space where each instrument is a point; by interpolating between points, we can generate new, hybrid audio clips. After generating hundreds of hybrids, we would select 30 that are both musically usable and emotionally evocative. Then, a small ensemble of musicians would compose and record an album of 10 tracks using only these hybrid instruments, performed via MIDI controllers mapped to the generated sounds. The result would be a celebration of global diversity—joyful, weird, and deeply respectful of the traditions that birthed the source instruments. The absurdity lies in the combinations: imagine a bagpipe-banjo or a koto-accordion. The project never got past the sample-collection phase, but the vision remains a tantalizing what-if.

Folk instruments are disappearing, and cross-cultural fusion in music often relies on superficial mashups rather than deep sonic integration. Existing sample libraries are Western-centric, and generative AI models rarely celebrate specific cultural timbres. Hybrid Ghosts scratches the itch of wanting to hear genuinely new acoustic possibilities that honor their roots while creating something entirely fresh.

• Collect and catalog 1,000+ folk instrument samples from at least 100 cultures
• Train a variational autoencoder to generate hybrid instrument sounds via latent space interpolation
• Generate 100+ candidate hybrids, then select 30 that are playable and evocative
• Compose and record an album of 10 original songs using only these hybrid instruments
• Release the album with extensive liner notes documenting each hybrid’s heritage
• Open-source the sample library and model for others to create their own hybrids

• NOT aiming for perfect acoustic realism—hybrids may sound synthetic or otherworldly
• NOT a commercial product; the album will be free or pay-what-you-want
• NOT trying to replace traditional instruments or discourage their use
• NOT a machine learning research paper; the focus is on musical results
• NOT a live performance project initially (though that could be a future phase)

• Audio Samples: Field recordings, institutional archives (e.g., Smithsonian Folkways), collaborations with ethnomusicologists. Capture at 44.1kHz/16-bit WAV.
• Preprocessing: Python (librosa, numpy) for pitch detection, normalization, and segmentation into single-note samples (~2s each).
• Generative Model: TensorFlow + Magenta’s NSynth implementation. Variational autoencoder with 512 latent dimensions, trained on all samples.
• Instrument Generation: Interpolate between two source instrument latent vectors, decode to audio, then map to a MIDI note range for playability.
• Composition/Recording: Ableton Live with Max for Live patches to trigger generated samples via MIDI. Use a Push controller for expressive performance.
• Mixing/Mastering: Standard DAW tools with care to retain hybrid timbres.

The project has three main technical layers: (1) Sample Pipeline – a Python script that ingests raw recordings, extracts single notes, normalizes pitch to A440, and creates a dataset of spectrograms. (2) Generative Model – a VAE trained on the spectrograms; each instrument becomes a point in latent space. A hybrid is created by linearly interpolating between two points, then decoding the resulting spectrogram back to audio using Griffin-Lim. (3) Performance System – the decoded audio clips are sliced into per-note samples and loaded into a sampler in Ableton. A Max patch maps incoming MIDI notes to the appropriate sample, allowing a musician to play the hybrid instrument chromatically. The composition process involves standard songwriting, but all sounds originate from these custom instruments.

• Sample Quality: Field recordings may have background noise, making training difficult. Mitigation: use clean isolated recordings where possible, and preprocess with noise reduction.
• Model Artifacts: Generated hybrids may sound glitchy or unnatural. Mitigation: we embrace artifacts as part of the aesthetic, but set a threshold for musical usability.
• Cultural Sensitivity: Blending sacred or culturally significant instruments might be seen as appropriation. Mitigation: work with cultural consultants, only use instruments with permission, and clearly credit source communities.
• Musician Frustration: Players may find the hybrid instruments hard to control (e.g., inconsistent dynamics). Mitigation: provide velocity maps and allow tweaking of ADSR envelopes.

• How do we ensure that the generated hybrids sound like more than just sum of parts? Should we incorporate musical context (scales, playing techniques) into the model?
• What is the best way to interpolate in latent space? Linear, spherical, or something else?
• Should we allow users to choose which instruments to blend via a web interface? That could be a later phase.
• How do we handle instruments with microtonal tunings? The model normalizes to equal temperament, but that may lose nuance.

The project stalled after the first milestone—we collected about 200 samples from local musicians and online archives, but the lead developer got a job in a different field and the momentum fizzled. The idea remains a cherished 'what-if' that could be revived with the right team.

The album title 'Hybrid Ghosts' refers to the spectral quality of the generated sounds—they feel like echoes of real instruments. The tracklist would include names like 'Sitar-Digeridoo Dream', 'Koto-Cajón Sunset', and 'Mbira-Xylophone Groove'. The absurdity is intentional: the kazoo might sneak in as a surprise element in one track.