EchoLens — Private, Local Music Discovery

EchoLens is a desktop music player designed for people with sizable local music collections who want intelligent, serendipitous discovery without sacrificing privacy. Unlike streaming services that mine every click, EchoLens runs entirely on-device. It uses lightweight neural networks to extract audio features (timbre, rhythm, mood, energy) from each track, then learns your preferences from your listening history—stored only in a local SQLite database. The result is a recommendation engine that surfaces forgotten gems, creates mood-based playlists, and even suggests transitions between songs. The app also includes a revolutionary sharing feature: you can generate a compact QR code that encodes a small set of model parameters representing your current musical taste. When a friend scans it, their local EchoLens can reconstruct a similar recommendation profile—without ever exchanging actual listening data. The vision is a future where music discovery is intimate, private, and owned entirely by the listener.

Current music streaming services collect vast amounts of personal data to train their algorithms, often prioritizing commercial interests over genuine discovery. People with large local libraries have few tools to explore them intelligently; most players offer only rudimentary shuffle or manual playlists. EchoLens scratches the itch of wanting a smart, adaptive player that respects your privacy and works entirely offline. It's for anyone who values their listening history as personal data and doesn't want to trade it for convenience.

• Achieve state-of-the-art on-device genre/mood classification without external data or cloud calls.
• Implement a playlist generation algorithm based on audio similarity and local listening history.
• Develop a cross-platform desktop app (macOS, Windows, Linux) with a reactive, beautiful UI.
• Enable playlist sharing via encrypted, self-contained codes (QR or file) that contain only model parameters, not raw audio.
• Ensure zero network requests by default; all processing is local and auditable.

• Not a streaming service; only plays local files.
• Not a music store or downloader.
• Does not collect any analytics, crash reports, or usage data.
• No social network, user accounts, or cloud sync.
• Not a replacement for library management or tagging—it's strictly for listening.

Backend (feature extraction & ML): Python with librosa, essentia, and TensorFlow Lite for on-device inference. Pre-trained models: VGGish embeddings, Mood classifier (valence-arousal). Database: SQLite for listening history and song metadata; FAISS for local vector similarity search. Desktop UI: Electron or Tauri with React frontend. WebAssembly for some ML inference in the renderer. Sharing: QR code generation using a custom tool that serializes a small neural network (few layers, few thousand parameters) into a binary string, then base64 compressed. Audio playback: PortAudio via a native module or Web Audio API in Electron. Build system: CMake + npm; Docker for reproducible builds.

EchoLens is structured as a client-server architecture running entirely on one machine. The backend is a Python service that starts on app launch. It scans the user's music directory (or specified folders) and extracts audio features using a pipeline: (1) decode file to PCM via ffmpeg, (2) compute mel-spectrogram, (3) run through pre-trained deep learning models to get embeddings (e.g., 128-dim vector per 3-second window, then aggregated by mean/max). These embeddings are stored in a local FAISS index along with track metadata. A separate module tracks listening sessions (play counts, skips, complete listens) in SQLite.

The recommendation engine works in two modes: (a) Explore – given a seed song, find nearest neighbors in embedding space, weighted by recent listening to avoid repeats; (b) Mood – let user select arousal/valence sliders, then retrieve tracks whose predicted mood falls in that region, shuffled for surprise. Playlists are generated on-the-fly and cached.

The frontend (Electron) communicates with the backend over a local HTTP/REST API (localhost only, no network exposure). It shows a custom UI: album art, a 'now playing' view with a visualizer, and playlist views. The sharing feature works by exporting the current recommendation model's weights (a small neural network for mood prediction) into a QR code. The receiving app decodes and loads those weights into a local model, effectively replicating your mood-prediction taste without ever seeing your library.

All network interfaces are blocked by default. The app has a firewall-like setting: 'No Network' mode that drops any outbound requests; developer tools confirm zero connections.

• On-device ML performance may be slow on older hardware, especially during initial scan (could take hours for 50,000 tracks).
• Audio feature quality may be insufficient to capture nuanced taste, leading to mediocre recommendations.
• Share QR codes could become large (target < 10KB), but model compression might reduce accuracy.
• User adoption may be low because people expect cloud convenience (sync across devices).
• Legal risk: some music files have DRM; app must clearly state it only plays DRM-free local files.

• What is the optimal trade-off between model size (for sharing) and recommendation quality?
• Should we pre-train a general audio model on public datasets (e.g., FMA, Million Song Dataset) or let users train from scratch on their own library? (Privacy vs. out-of-box experience)
• How to handle lossy audio formats and variable bitrates that affect feature extraction?
• Should we include a 'public' mode where users can opt-in to share anonymous aggregate listening patterns? (Probably not, to stay pure privacy-first.)
• What is the best way to visualize musical taste? A 'taste graph' showing clusters of genres/moods?

The project was started by a solo audio engineer/developer who built the ML pipeline and proof-of-concept during weekends over six months. After achieving a working prototype with decent recommendations, the developer faced burnout and a cross-country move. The UI remained unfinished (ugly Electron shell), and integrating all platform-specific audio backends proved daunting. The idea was shelved as a 'someday' project, but never revived. It's a classic case of an ambitious plan outlasting the energy of its creator.

The sharing feature is the most innovative part: it transfers taste, not data. Could be patented? Also consider a 'jukebox' mode for parties where guests can add songs to a queue via QR code scan. If revived, would use Tauri instead of Electron for smaller binary size and better native API access.