The Unscreen: Ambient Life OS

The Unscreen is a bold vision for a post-smartphone world where digital interactions happen through ambient interfaces—voice, gestures, projected displays, and subtle haptics—rather than a glowing screen in your pocket. It is a central ‘brain’ that orchestrates your digital life across all ambient devices: smart speakers, earpieces, smart glasses, and home projectors. Unlike today's siloed assistants (Alexa, Siri, Google Assistant), The Unscreen learns your routines, anticipates your needs, and provides a unified, proactive experience. It runs as a Progressive Web App (PWA) on a local hub (e.g., a Raspberry Pi) inside your home, ensuring privacy and low latency. The default interface is voice-driven, but a web dashboard allows for configuration and complex tasks. The Unscreen uses on-device machine learning to predict when you need reminders, directions, or information without you asking. For example, upon waking, it might softly announce your day's schedule; when you pick up your keys, it might display a map with traffic to your first meeting on your smart glasses. It’s designed to be open and extensible: third-party developers can create ‘modules’ (skills) using a simple API. The Unscreen is not a device; it’s the software that brings ambient computing to life.

The smartphone has become an attention vortex—a constant source of distraction, pulling us into a cycle of endless scrolling and notification checking. In an ambient-computing future, we need a digital companion that respects our attention, provides information proactively and contextually, and fades into the background when not needed. Current smart assistants are too reactive, siloed, and lack deep personalization. They don't understand your routines across devices. The Unscreen aims to solve this by being the consistent, intelligent layer that connects all ambient touchpoints, making technology assist without demanding focus.

• Create a unified ambient interface that works across diverse devices (speakers, glasses, earpieces, projectors) without a central screen.
• Use on-device machine learning to predict user needs (e.g., automatic commute alerts, medication reminders, proactive note suggestions).
• Support multiple input modes: voice commands, hand gestures, eye tracking, and discrete haptic feedback.
• Provide an open API and SDK for third-party 'modules' (e.g., weather, calendar, smart home control).
• Ensure privacy-first design: all voice and sensor processing happens locally on the hub; cloud sync is optional and encrypted.
• Deliver a configuration web app that lets users set up routines, view logs, and manage modules without needing a smartphone.

• Not a smartphone replacement: it won't run traditional mobile apps or support a touchscreen interface as primary.
• Not a social network or messaging platform: communication features are limited to basic notifications (calls, messages from existing services).
• Not a gaming console or entertainment hub: it won't play music or video by itself (delegates to existing services).
• Not a subscription service: the software is free; users only pay for the hub hardware (e.g., Raspberry Pi) or buy a pre-built hub.

• Frontend (Web Dashboard): React, WebXR for potential AR overlays, Web Speech API for voice configuration, Web Bluetooth for wearable pairing.
• Backend (Hub): Node.js with Express, MongoDB for user profiles and historical data, Redis for real-time state and session management.
• Machine Learning: TensorFlow.js for on-device models (activity recognition, intent parsing), Rasa NLU for natural language understanding.
• Communication: MQTT for device-to-hub messaging, WebSocket for real-time dashboard updates.
• Hub Hardware: Raspberry Pi 4 or similar with Bluetooth 5.0, Wi-Fi, and optional microphone array.
• Security: WebAuthn for passwordless login to dashboard, local-first data storage, end-to-end encryption for cloud sync.

The Unscreen architecture is hub-and-spoke. At the center is the Unscreen Hub, a low-power Linux computer (Raspberry Pi) that runs all core services. The hub listens continuously for voice commands via a far-field microphone, processes them locally using TensorFlow.js and Rasa, then executes actions or announces information via connected speakers (e.g., via AirPlay or Bluetooth). Peripheral ambient devices (smart glasses, earpieces, smart displays) connect to the hub via a local network using MQTT or WebSocket. Each peripheral runs a lightweight client that sends sensor data (location, accelerometer, eye tracking) and receives rendering instructions. For example, a pair of smart glasses can receive a 'visual note' overlay. The Prediction Engine runs on the hub: it uses a recurrent neural network trained on the user's historical patterns (e.g., when they leave for work, take medication) to surface proactive suggestions. The Web Dashboard is a PWA that runs in any browser on the same network, allowing for deep configuration (routines, module settings), viewing the hub's logs, and manual overrides. Cloud sync is optional and implemented via a REST API with end-to-end encryption; it allows syncing profiles across multiple homes (e.g., work and home hubs).

• User trust: an always-listening device that processes all audio locally may still raise privacy concerns; community audits and open-source code are essential.
• Prediction accuracy: false positives (e.g., wrong reminders) could annoy users and degrade trust. Models need continuous feedback loops.
• Interoperability: competing with established ecosystems (Apple HomeKit, Google Home, Alexa Skills) is difficult; many users already have devices from these ecosystems.
• Hardware fragmentation: supporting a wide range of ambient devices (different smart glasses, earpieces) adds complexity and testing burden.
• Latency: local processing of speech and ML models on a Raspberry Pi may be too slow for real-time interaction; need optimization or more powerful hardware.

• How to distinguish multiple users in a household using voice biometrics? Is on-device voice recognition accurate enough?
• Should we support visual projections (e.g., onto a wall) as an output modality? That would require a pico projector and add cost.
• What is the business model? One-time hub sale, or a marketplace for premium modules?
• How to handle complex tasks that require a screen (e.g., editing a spreadsheet) in an ambient manner? Possibly through AR glasses or a short-lived projected interface.

The plan was drafted in early 2023, but the team got excited by the generative AI wave and pivoted to a chatbot startup. By the time they wanted to revisit, ambient computing hardware hadn't reached critical mass, and the idea felt ahead of its time. It remains a fascinating what-if that could be revived as smart glasses and local LLMs improve.

This project assumes a world where ambient interfaces are mainstream, but it could be prototyped today using existing devices like Google Home, Apple AirPods, and a Raspberry Pi. The key differentiator is the unified prediction engine and local-first approach.