Project Siren: The Silent DJ

Project Siren is a desktop application that turns your personal music collection into a living, breathing ecosystem of recommendations—but with a radical twist: it never phones home. Unlike every streaming service that hoards your listening habits, Siren runs entirely offline. It builds an acoustic fingerprint of every track you own using on-device feature extraction (chroma, MFCC, tempo, key, etc.), stores those fingerprints in a local vector database, and uses a lightweight neural network to find subtle connections between songs you love. The result is a DJ that knows you better than any cloud algorithm could, because it lives on your computer and sees your full library, not just a slice. Voice control runs on a local wake-word engine (Porcupine) and a tiny NLP model, so even your commands stay private. Siren integrates with popular local music players like Foobar2000 and VLC via plugins, or can play audio directly. It was born from the frustration of building recommendation systems that required massive data collection—this is the antidote: a system that respects your privacy by design, not by policy.

Every music recommendation service today—Spotify, Apple Music, Pandora—collects vast amounts of listening data to train their algorithms. They know not just what you listen to, but when, where, and for how long. Even offline players like VLC offer no intelligent recommendations. Music lovers who value privacy are forced to choose between isolation and surveillance. There is no mainstream, privacy-respecting tool that provides rich, context-aware recommendations from a local library. Project Siren scratches that itch: a recommendation engine that works without a single byte of data ever leaving the user's machine.

• Build a fully offline audio feature extraction pipeline that runs in real-time on consumer hardware.
• Create a local vector database (using Spotify's annoy or FAISS) to store and query audio fingerprints.
• Train a small neural network on public datasets (e.g., FMA) to learn musical similarity, then fine-tune on the user's library without sending data anywhere.
• Implement a voice control interface using Porcupine (wake word) and a tiny transformer model running locally for natural language commands.
• Integrate with major local music players via plugin architecture (Foobar2000, VLC, MusicBee).
• Achieve playlist generation in under 2 seconds for a library of 10,000 tracks.
• All code open-source under MIT license.

• Will not collect any usage telemetry, crash reports, or user behavior data.
• Will not support cloud sync or online streaming services that require authentication.
• Will not include any user accounts or login system.
• Will not perform any kind of digital rights management (DRM) or restrict playback.
• Will not use GPU for inference (to keep it accessible on laptops).

• Audio feature extraction: librosa, essentia
• Vector storage: FAISS (Facebook AI Similarity Search) or Spotify's Annoy
• Machine learning: PyTorch (lightweight models), ONNX runtime for deployment
• Voice control: Porcupine wake word engine + tiny-LLM (e.g., GPT-4-All's smallest model) for intent parsing
• Backend: Python 3.10+ with asyncio, SQLite for library metadata
• Frontend: Electron with React (or Tauri for lighter weight)
• Integration: Plugin APIs for Foobar2000 (C++), VLC (libvlc), and MusicBee (C#) – or direct audio playback via PyAudio

Siren's architecture is a modular pipeline with four layers: (1) Ingestion Layer: Watches file system changes and extracts audio features using librosa, storing vectors in FAISS and metadata in SQLite. (2) Similarity Layer: A lightweight neural net (e.g., a three-layer MLP on reduced features) computes pairwise similarities, building a graph of tracks. Queries take a seed track and return top-k neighbors from the graph. (3) Control Layer: The voice module runs Porcupine + a tiny local LLM that parses commands like “play something like this” or “make a quiet jazz playlist”. (4) Playback Layer: Either spawns a local player process (VLC executable) or communicates with plugins via named pipes or HTTP. All data stays in the user’s home directory; no network sockets are opened for data transmission.

• Performance: On-device feature extraction for thousands of tracks may be slow on older machines. Mitigation: Use incremental extraction and offer a batch mode.
• Accuracy: Small local models may not match cloud-scale recommendations. Mitigation: Leverage pre-trained embeddings from public datasets and allow user feedback (purely local) to refine.
• Voice control privacy: Even local voice models could be exploited if not properly sandboxed. Mitigation: Use open-source, audited models and run them in a subprocess with minimal permissions.
• Adoption: Users may not trust open-source claims of privacy without third-party audits. Mitigation: Publish a thorough security audit and reproducible builds.

• Should the similarity model be collaborative (based on user listening patterns within the local library) or purely content-based? A hybrid could be more powerful but adds complexity.
• How to handle dynamic playlists that react to user mood without any external input? Possibly use time-of-day heuristics.
• What is the best way to integrate with existing players without requiring the user to switch? Plugin support vs. acting as a standalone player.
• Should we include a limited set of public domain music to seed recommendations for new users with small libraries?

Project Siren remains a plan because life got busy during the feature extraction research phase. I had a working prototype that could fingerprint 500 songs, but the fine-tuning of the similarity model required more time than I had. I intended to spend a sabbatical on it, but then I changed jobs and the momentum faded. It's still a project I'd love to build, but for now it lives as a detailed notebook and a set of half-written Python scripts.

The idea for Siren came from a frustration with Spotify's data collection. I wanted to prove that a good recommendation engine doesn't need to spy on you. The name is a nod to both the mythological creature (luring listeners) and the privacy watchdog (alarm against surveillance). Initial tests on the Free Music Archive showed that a simple content-based model can achieve a 70% similarity agreement with user ratings—enough to be useful. The voice control part was never started; it was always a stretch goal.