Email or username:

Password:

Forgot your password?
jonny

More fun publisher surveillance:
Elsevier embeds a hash in the PDF metadata that is *unique for each time a PDF is downloaded*, this is a diff between metadata from two of the same paper. Combined with access timestamps, they can uniquely identify the source of any shared PDFs.

50 comments
серафими многоꙮчитїи

@jonny they look kind of meaningful. Not base64. Any ideas what could be in there?

jonny

@derwinmcgeary
yeah, I thought so too but don't know where to start reverse engineering it :/

jonny

@derwinmcgeary
it decodes with base85, but it's not Unicode. not sure if that's meaningful

Old Tom

@jonny I do not have any IT skills, but if I did I’d love to write a script to remove metadata from PDFs. Adobe has them wrapped up pretty well.

jonny

You can see for yourself using exiftool.
To remove all of the top-level metadata, you can use exiftool and qpdf:

exiftool -all:all= <path.pdf> -o <output1.pdf>
qpdf --linearize <output1.pdf> <output2.pdf>

To remove *all* metadata, you can use dangerzone or mat2

jonny

Also present in the metadata are NISO tags for document status indicating the "final published version" (VoR), and limits on what domains it should be present on. Elsevier scans for PDFs with this metadata, so good idea to strip it any time you're sharing a copy.

jonny

Links:
exiftool: exiftool.org/
qpdf: qpdf.sourceforge.io/
dangerzone (GUI, render PDF as images, then re-OCR everything): dangerzone.rocks/
mat2 (render PDF as images, don't OCR): 0xacab.org/jvoisin/mat2

jonny

here's a shell script that recursively removes metadata from pdfs in a provided (or current) directory as described above. For mac/*nix-like computers, and you need to have qpdf and exiftool installed:
gist.github.com/sneakers-the-r

jonny

The metadata appears to be preserved on papers from sci-hub. since it works by using harvested academic credentials to download papers, this would allow publishers to identify which accounts need to be closed/secured
twitter.com/json_dirs/status/1

jonny

for any security researchers out there, here are a few more "hashes" that a few have noted do not appear to be random and might be decodable. exiftool apparently squashed the whitespace so there is a bit more structure to them than in the OP:
gist.github.com/sneakers-the-r

jonny

this is the way to get the correct tags:
(on mac i needed to install gnu grep with homebrew `brew install grep` and then use `ggrep` )
will follow up with dataset tomorrow.
twitter.com/horsemankukka/stat

jonny

of course there's smarter watermarking, the metadata is notable because you could scan billions of pdfs fast. this comment on HN got me thinking about this PDF /OpenAction I couldn't make sense of earlier, on open, access metadata, so something with sizes and layout...

jonny replied to jonny

updated the above gist with correctly extracted tags, and included python code to extract your own, feel free to add them in the comments. since we don't know what they contain yet not adding other metadata. definitely patterned, not a hash, but idk yet.
twitter.com/json_dirs/status/1

jonny replied to jonny

you go to school to study "the brain" and then the next thing you know you're learning how to debug surveillance in PDF rendering to understand how publishers have so contorted the practice of science for profit. how can there be "normal science" when this is normal?

jonny replied to jonny

follow-up: there does not appear to be any further watermarking: taking two files with different identifying tags, stripping metadata, and relinearizing with qpdf's --deterministic-id flag yields PDFs identical with a diff, ie. no differentiating watermark (but plz check my work)

jonny replied to jonny

which is surprising to me, so I'm a little hesitant to make that as a general claim

Nick Astley replied to jonny

@jonny

It's a couple things:

a) Elsevier's vendor's tool only has to be good enough to impress Elsevier

b) Deterrence being more efficient than prevention

shusha replied to jonny

@jonny for the normativity of science see the discourse of STS (science and technology studien), great field!

jonny replied to shusha

@shusha
yes definitely, love it and spend basically all my time reading it nowadays ❤️

robryk

@jonny I wonder whether uploading every paper to sci-hub twice would be feasible (i.e. would we still have enough people do that). (If we did so, then it would allow sci-hub to verify with reasonable certainty that whatever watermark-removal method they would use still works.)

jonny

@robryk
I think it may be easier to scrub it server side, like to have admins clean the PDFs they have. I don't know of any crowdsourced sci-hub-like projects. scrubbing metadata does seem to render the PDFs identical

robryk

@jonny And then obviously the watermarking techniques will adapt. Asking for two copies is a way to ensure that whatever we are doing still manages to scrub the watermark (they should be identical after scrubbing).

jonny

@robryk
yes, definitely. all of the above. fix what you have now, adapt to changes, making double grabs part of the protocol makes sense :)

Advanced Persistent Teapot

@jonny hmmm, alternatively start inserting copies of that metadata into blank or template PDFs. Send 'em chasing wild geese and make them look at Lorem Ipsum 50 times a day 😈

Thai Thien

@http_error_418 @jonny Very well.
Do you have code to insert those code ? I would like to help.
Btw, you can use mathgen thatsmathematics.com/mathgen/ to make meaningless paper to upload on scribd

:garfield:‍fuchsiaaaaaaaaaaaaaaaaa

@jonny word of caution is that while removing exif is good, knowing publishers there's a bunch of other ways they'd directly include such trackers into the file, in a less human/machine readable spot than EXIF. so be careful

Stewart Russell

@jonny they're almost getting to the level of ISO standards for metadata f'wittery.

For a while, many ISO standards that you bought (for $$$$) looked like a bad photocopy. If you zoomed in really close to the marks on the page, they were made up of a pattern of punctuation characters. Totally screwed up any screen reading, though

Orca🌻 | 🏴🏳️‍⚧️

@jonny@social.coop seems like some countermeasures against scihub, libgen and other shadow libraries that provide those PDFs for free 🤨

KawaiiPunk

@jonny this is the same technique that was being used in the OS designed in North Korea called Red Star OS. It was in the Chaos Congress talk about it.

Gord

@jonny they really are a right bunch of bastards, aren’t they?

🦇Lyle Solla-Yates🦇

@jonny this makes me think some horrible things are going to happen to students because of this but I can’t quickly think of an example

Carl Mathias Kobel 🧬🦠🧫👩‍💻

@jonny But what meaningful data can they attach to that unique ID? The IP adress? Assume a user is not logged in, has cleared tracking cookies and is using a VPN.
Wait a sec. That is why we need open access.

jonny

@cmkobel
Browser fingerprinting is p robust.
amiunique.org/fingerprint
And even those mitigations wont be taken by 99.999% of visitors.

OddOpinions5

@jonny

note: I am a tech illiterate

I went to the exif tool website and clearly not somethng I would find easy to use

so I googled
tool to remove pdf metadata

and it seems like there are lots of nice easy to use programs - google chrome has one built in ?

eg

my fav software seems to have a metadata removal/sanitize tool
pdf-xchange.com/search?query=r

and also

tools.pdf24.org/en/remove-pdf-

pdfgear.com/pdf-editor-reader/

@jonny

note: I am a tech illiterate

I went to the exif tool website and clearly not somethng I would find easy to use

so I googled
tool to remove pdf metadata

and it seems like there are lots of nice easy to use programs - google chrome has one built in ?

eg

my fav software seems to have a metadata removal/sanitize tool
pdf-xchange.com/search?query=r

Sertonix

@jonny uBlock is able to modify the response of http requests. Maybe somebody can create a filter to strip the hash from the file.

Serge from Babka

@jonny

This is for sure embedded watermarking but I'm not seeing active surveillance in this (unlike, say Kindle or the myraiad of phone-home technologies).

While I would strongly prefer Free Culture media where there is permission granted for distribution, modification, attribution mandates, and redistribution, if the media is DRM-free but does not permit widespread distribution, what mechanisms do you see as appropriate for the copyright holder to use to identify unauthorized distributed copies?

Gerard Ritsema van Eck

@serge @jonny
They could easily use this to track who is uploading papers to shadow libraries and slap them or their institutions with huge lawsuits.

For-profit publishing has no place in modern #academia.

Serge from Babka

@Gerard

I agree with you that for-profit publishing and academia are not good bedfellows, and I generally think copyright is broken in more ways than one. The issue for me is given the current legal environment, this seemed a simple way to address unauthorized duplication that doesn't involve DRM or other surveillance tech.

@jonny

KawaiiPunk

@jonny this is the same tracking technique used in the operating system from North Korea called Red Star OS.

http :verified:

@jonny If this "hash" is the only difference, then it can easily be removed or replaced, as it seems to be related to a user id. The question is if there are further differences, but looking at your later posts it seems that no.

Gilgwath

@jonny Aaah yes, DMCA against random citizens, enforced by the same govs who original funded the majority of the papers in the first place. So, the tax payer gets to pay for the education, the research, the access fee AND the cops and legal system who have to go after the right minded citizens who publish the stuff they already payed for at least twice. This is fine. 🔥 The market will regualte itself.

Rue Mohr

@jonny Why else would they require a free account and login to download it...

Jacques Chester

@jonny @SwiftOnSecurity those look sequential rather than fully random hashes

Go Up