Gregory's ServerThe EU’s “chat control” legislation is the most alarming proposal I’ve ever read. Taken in context, it is essentially a design for the most powerful text and image-based mass surveillance system the free world has ever seen.
|
Matthew Green
16 comments
Matthew Green
The legislation is vague about how this will be accomplished, but the “impact assessment” it cites is not. The assessment makes clear that mandatory scanning of images & text, especially in encrypted data, is the only solution the Commission will consider.
Matthew Green
The calls for detecting “grooming behavior”. If you wonder what that means, here is a brief description. Roughly it means developing new AI tools that can understand the content of textual conversations and can automatically report you to the police based on them.
Matthew Green
You might ask how the EU, famous for its focus on privacy, justifies the development of automated text-analysis tools that scan your private chats. The Impact Assessment has an analysis. To say that this analysis is deficient is really much too kind.
Matthew Green
As a technologist I have to point out that the technological solutions to do this *safely* don’t exist. They are at best at the research stage. ML textual analysis schemes do exist, and often misfire. These systems will need to accomplish this task perfectly and also privately.
Matthew Green
The idea that we can deploy AI systems to read your private conversations and report crimes is frankly dystopian. Even if such systems existed, no reasonable democracy would vote for this. But this is what the EU is proposing to mandate and *build* in the next couple of years.
Matthew Green
If you take comfort from the fact that these systems are aimed at “awful crimes” or “will be fully transparent”, please don’t. The nature of these proposals is that they will be easy to reprogram, either by law or by technical accident.  
IoT is the grey goo
...and all of it based on the unspoken assumption that adolescents ought to be out playing by themselves on the info superhighway. If they consider the potential harm to children serious enough to extinguish private communication and create a ubiquitous Big Brother, then the #CarAnalogy extends to this issue as well... Your children do not belong "behind the wheel" or "playing on the highway".
Alexander The 1st
@tasket @matthew_d_green I don't agree with this; with the way the internet ends up being applied to a lot of Internet of Things connected items, and the wealth of knowledge accessible online, this ends up being the equivalent of not allowing anyone to drive a car below the age of 21...in a rural town more than 100 km away from the nearest post secondary education institutions.
Alexander The 1st
@tasket @matthew_d_green (And while post secondary education is potentially optional, consider the same conditions for the local secondary education, or elementary, or pre-school education institutions.)
ChiefBongo
@tasket @matthew_d_green exactly! But I guess there are many parents who do not care or look what their Kids are doing - the logical thing would be to legally force device manufacturers to install a child-protection feature and legally compelling parents to activate it. NCMEC are a bunch of zealots and have blown the stats completely out of proportion - several German NGO's have taken it apart and left only with a fraction of what is claimed. 
LisPi
@matthew_d_green A little meme about this that @mullvadnet posted and I think fits. source: https://mastodon.online/@mullvadnet/109965262142928181
Skyler (Star Group)
@matthew_d_green if this were implemented and enforced in my country i would probably just stop talking to minors online at all and i don't know what else, i'm sure that wouldn't protect me from every sort of accusation
plokta
@matthew_d_green Even if the worst parts of the proposal can still be blocked by memberstates: the proposal also tries to establish mandatory age verification for the web, that is, slipping in ID/passport verification through the backdoor. This part is easily overlooked in analyses and needs to get more attention in public debates. @khaleesicodes |
This legislation, which is initially targeted at child abuse applications, creates the infrastructure to build in mandatory automated scanning tools that will search for *known* media, *unknown* media matching certain descriptions, and textual conversations.