Comprehensive Online Agreement Transparency (COAT) Toolkit
Coach Name
Jordi Bosch i Garcia
EU Organization
Sangiorgi SRL (Italy)
Members
- Alessandro Sangiorgi
- Mirko Dimartino
- Massimo Gollo
- Fabio Esposito
US Organization
Saint Louis University (USA)
Members
- Flavio Esposito
- Lorenzo Pappone
Project Overview
COAT addresses one of the most pervasive problems in today’s digital ecosystem:
online users are routinely asked to accept privacy policies and terms of service that are thousands of words long, opaque, and nearly impossible to understand. More than 90% of users consent without reading, resulting in a “consent paradox” where legal consent exists but meaningful informed consent does not.
COAT introduces an AI-powered transparency layer that automatically analyzes online agreements and delivers:
- Privacy Scores – a quick, quantitative assessment of risk
- Alerts – flags for clauses that may affect user rights
- Plain-language summaries – concise, accessible explanations
Delivered through a mobile app, users can submit an online agreement and receive a human-readable analysis within seconds. COAT enables informed decision-making, helps organizations audit their vendors, and aligns with modern data-protection principles promoting transparency and accountability.
Methods and approaches
AI-driven legal text analysis using LLMs + MCDM scoring
COAT developed and fine-tuned a specialized LLM (based on Qwen3) using a labelled dataset of 250+ privacy policies. The team combined Multiple Criteria Decision Making (MCDM) with LLM outputs to generate explainable, auditable privacy scores across categories such as data collection, user control, third-party sharing, retention, and security.
User-centred mobile application with real-time evaluation
A mobile MVP (Android) was designed based on UX research with 50+ users, producing an accessible interface that displays scores, alerts, and summaries. The app retrieves agreements, sends them to the backend for processing, and returns structured results instantly.
Key Achievements
Annotated dataset of 269 privacy policies, evaluated across multiple LLM models and published openly.
Fine-tuned LLM (Qwen3-30B-A3B) achieving 83% validation accuracy in rubric-based analysis.
Mobile MVP including Privacy Score, Alerts, and Summary views, validated by real users.
Proof-of-Concept video demonstrating end-to-end app functionality.
System architecture and technical specifications published openly.
Peer-reviewed paper accepted at IEEE WETICE 2025 detailing architecture, methodology, and evaluation.
Extensive dissemination with over 39,000 impressions across Instagram, Telegram, and Reddit.
Open-source contributions including dataset, model weights, training scripts, and adapters on HuggingFace.
Impact & Results
Scientific Impact
COAT advances research on AI-driven legal document analysis, providing:
- a large open annotated dataset,
-a hybrid LLM + MCDM transparent scoring method,
- empirical validation of fine-tuned LLMs for privacy policy assessment.
Societal Impact
COAT combats “digital resignation” by giving users clear, rapid insight into how their data is used—reducing exposure to hidden practices and enabling genuine informed consent.
Economic Impact
Organizations can use COAT to assess third-party agreements, reduce compliance risk, and understand how policy changes affect user trust.
One new job and two contractor roles were created directly through the project.
EU–US Collaboration
The partnership with Saint Louis University enabled joint research, academic outreach, a seminar and pitch event on-site, and long-term collaboration on privacy-tech research.
Publications and Open-Source Contributions
- Dataset: https://github.com/maczg/coat-dataset
- Fine-tuned LLMs, LoRA adapters, safetensors, GGUF formats, and training notebooks on HuggingFace.
- Conference paper: Quantifying Privacy Risk in Online Agreements with COAT – An LLM Approach (IEEE WETICE 2025).
- Proof-of-Concept video: LinkedIn (public link in report).
- Project website: https://coat.sangiorgisrl.com
- Active LinkedIn presence with monthly posts over the 9-month programme.
Future directions