On-device approaches have a contested historical past, and any deployment should be genuinely privacy-preserving by design. However know-how firms should weigh the objection in opposition to the harms that proceed to happen. A security by design strategy is required.
On-device security measures have been demonstrated at scale with Apple’s on-device nudity detection for photos despatched or acquired by way of Messages, AirDrop and FaceTime. A 2025 research demonstrated high-accuracy grooming detection utilizing Meta’s AI mannequin designed particularly for on-device deployment on cell phones.
Not too long ago, each Apple and Google have began to take measures in direction of app store-based age verification in some jurisdictions.
The best-profile real-world deployment of those is Apple enabling device-level privacy-preserving age verification within the UK.
Social media and personal messaging firms, together with working system distributors (Microsoft, Apple, and Google), all have a task to play in making certain dangerous content material is detected, whether or not or not end-to-end encryption is used. Progress has been sluggish. However we, as a neighborhood, have to demand extra from these firms.
Joel Scanlan is Adjunct Affiliate Professor on the Faculty of Regulation and Educational Co-Lead at CSAM Deterrence Centre, College of Tasmania. This text first appeared in The Dialog.
