SignScape: A Decentralized Visual-First Social Network for Sign Language Communities

SignScape is a reimagining of social networking for the Deaf and Hard of Hearing. Instead of text posts, the primary unit is the signed video—a 'sign story'—along with a visual-first interface that uses icons, gestures, and spatial cues. Users can record short signed clips, stitch them into narratives, and share them with followers.

A core innovation is the AI bridge: trained on large sign language corpora (e.g., WLASL for ASL, RWTH-PHOENIX for German Sign Language), the platform can translate signed content into text captions for hearing users who join the network. Conversely, text comments can be converted into animations of a signing avatar, making the platform truly bilingual. The AI runs optionally on-device (via WebAssembly) or on the instance server, respecting privacy.

SignScape is federated using an extended ActivityPub protocol that treats signed videos as first-class objects. Each user's 'signspace' is hosted on an instance, and videos are stored on IPFS for content-addressed, decentralized distribution. WebRTC enables real-time signed video calls. The goal is to create a digital home where sign language is the norm, not an afterthought.

Mainstream social media platforms are built around text and hearing-centric experiences. Deaf users often struggle with poor automatic captioning, video compression that degrades sign language clarity, and interfaces that require reading text. They exist on the margins—tolerated but not prioritized. SignScape scratches the itch for a platform where sign language is the native language, not a translated afterthought. It also addresses the lack of a dedicated, decentralized network that respects the visual and spatial nature of sign languages.

• Design a fully visual interface with minimal text, using icons and intuitive gestures.
• Enable high-quality signed video recording and playback with low latency.
• Develop AI sign-to-text translation with accuracy above 90% for common sign languages.
• Implement text-to-sign rendering via a realistic signing avatar.
• Build a federated architecture (ActivityPub extension) for decentralized instance hosting.
• Integrate IPFS for video storage and peer-to-peer sharing.
• Support end-to-end encryption for private signed video messages.
• Conduct usability studies with Deaf communities to iterate on design.

• Not a replacement for Video Relay Services (VRS) or emergency communication.
• Not a sign language learning tool for hearing people.
• Not a general-purpose video platform (e.g., YouTube).
• Not aiming to translate every signed video perfectly; AI is a bridge, not a requirement.

• WebCodecs API for efficient video encoding and decoding in the browser.
• WebRTC for peer-to-peer video calls and real-time signing.
• Custom CNN + LSTM models for sign language recognition (trained on ASL, BSL, etc.).
• Three.js with procedural animations for signing avatars.
• ActivityPub protocol extended with Video objects and IPFS hashes.
• IPFS (InterPlanetary File System) for decentralized content storage.
• SQLite for lightweight instance metadata.
• WebAssembly for client-side AI inference to preserve privacy.

SignScape uses a federated model: each instance (like a Mastodon server) hosts multiple users. Users create 'signspaces' with written bios (optional) and a feed of signed videos. When a user records a video, it's encoded using WebCodecs, uploaded to IPFS, and the resulting CID is added to an ActivityPub 'Create' activity with a special 'VideoNote' type. The activity is distributed to followers via ActivityPub's Inbox.

For translation, an optional service on the instance runs sign language models (loaded via TensorFlow.js or PyTorch WebAssembly). The model processes the video frames and outputs text captions, which are stored alongside the video metadata. Hearing users see a 'show captions' button. For text-to-sign, the avatar engine (Three.js) parses text into a sequence of glosses and animates the avatar signing them. The avatar is rendered client-side to reduce server load.

Real-time calls use WebRTC with adaptive bitrate to prioritize signing clarity. Federation ensures that no single entity controls the network; instances can be run by Deaf organizations, schools, or individuals.

• AI translation accuracy for sign language is still around 60-80% on diverse datasets, risking frustration.
• Decentralization complicates moderation (e.g., hate speech in signed videos).
• IPFS latency for video playback may be poor without a reliable gateway.
• Adoption barrier: requiring users to install specialized apps or browsers.
• Sign language variation (regional dialects, home signs) may not be covered.

• Which sign language to prioritize first? (ASL due to available data, but community feedback may favor BSL or others.)
• Should the avatar be stylized or realistic? (Realistic may be uncanny, stylized may be more accepted.)
• How to handle reported signed video content in a decentralized way? (Instance-based moderation vs. shared blocklists.)
• Can peer-to-peer video storage be reliable enough for daily use?

The sheer scope of combining decentralized infrastructure, state-of-the-art AI, and a dedicated community platform was beyond what a single developer could achieve in evenings and weekends. Interest was real, but life moved on, and the idea remains a carefully planned what-if.

This project would require collaboration with Deaf community members from day one to ensure authenticity. The name 'SignScape' evokes a landscape of signs. Patents may be considered for the avatar system, but the federated protocol should be open. Funding could come from grants for accessibility technology.