The Algorithmic Harm Audit Collective

This project flips the script on AI bias research. Instead of waiting for academics locked in ivory towers to publish papers on algorithmic harm, it empowers the communities most affected to design and conduct their own audits. Using an open-source toolbox of methods—ethnographic interviews, data scraping, participatory observation—the collective gathers evidence of harm from the ground up. The research is not peer-reviewed in the traditional sense but validated through community consensus, transparent methodology, and a living database accessible to journalists, regulators, and activists. The vision is a growing, federated network of local nodes, each focused on a specific algorithmic domain (hiring, policing, credit scoring), contributing to a shared public record. This is not a study of AI bias; it is a movement to democratize the production of knowledge about AI’s real-world effects.

Current AI bias research is dominated by academics who are often disconnected from the communities most impacted. Research questions are framed by funding bodies, not by those experiencing harm. This leads to a gap between what is studied and what is felt. Moreover, traditional peer review can be slow, gatekept, and dismissive of lived experience as evidence. This project scratches the itch for a more democratic, bottom-up research approach that treats affected people as experts, not subjects.

• Develop and distribute an open-source toolkit for community-led algorithmic auditing
• Train 10 community groups in under-resourced areas to conduct their own audits
• Produce a public, searchable database of at least 50 documented harm cases
• Publish a community-authored manifesto on ethical research methods
• Influence at least one local policy change regarding AI transparency

• This project will not seek traditional academic journal publication
• It will not collect personally identifiable information without explicit consent and anonymization
• It will not engage in adversarial hacking or illegal data extraction
• It will not aim for statistical representativeness; qualitative depth is prioritized

Open-source tools: Python for ethical web scraping (with strict guidelines), Obsidian for knowledge management, Signal for secure communication, Git for version-controlled evidence logs, and a shared Notion baseline for coordination. Paper notebooks and audio recorders for offline ethnography. The database is hosted on GitHub Pages with a static site generator.

The research is structured as a federation of local nodes. Each node is a community group that selects a focus area (e.g., hiring algorithms, policing AI). They follow a shared protocol for evidence collection: daily diaries, weekly debriefs, monthly cross-node synthesis. A rotating council of community researchers reviews new entries for quality and consistency. The database is a public GitHub repository with a web interface built by volunteers. The architecture is intentionally decentralized—no central university or corporate entity owns the data.

• Burnout of community researchers without compensation
• Legal threats from companies whose harms are documented
• Inconsistent data quality across nodes
• Difficulty in getting policymakers to take non-traditional research seriously
• Internal conflicts over governance and data ownership

• How to handle cases where harm is not yet legally defined?
• Should the collective accept funding from foundations? Potential for co-option.
• What is the best way to anonymize data while preserving its evidentiary power?
• How to balance speed of documentation with thorough verification?

The project was scoped out in a series of community workshops but never secured sustainable funding. The core organizers got overwhelmed by day jobs and union organizing. The idea remains alive in a Slack channel, waiting for the right moment and resources to ignite again.

This plan originally included a decentralized autonomous organization (DAO) for budget governance, but that was dropped as too complex for the initial phase. The ethical guidelines were adapted from existing participatory action research frameworks.