When you buy a physical book, you don't sign an agreement dictating the conditions which you must satisfy to read it. That would be crazy, right? You expect to be able to read it anywhere you like. You can read it on the bus, you can read it in the park. Heck, you can even take it to the shower and read it, damaging the pages, but it is your book, you paid money for it and you can do whatever you want.
If you buy an ebook from a major store (Amazon, Kobo), chances are you are getting something extra with your book.
If you have a supported platform, everything magically works, and you don't think about buying the next book much. If you, like me, have GNU/Linux as your only OS, then your purchased book basically turns into a digital paperweight. Instead of owning the book, you merely own a license to read the book and only a set of approved readers will allow you to actually read it. The class of systems that impose these restrictions is called DRM, after Digital Rights Management. And you have very little rights there.
There are several well-known DRM systems in existence:
- Amazon Kindle (originally Mobipocket, it has a number of different formats now)
- Apple iBooks, uses FairPlay DRM.
- Adobe Digital Editions Protection Technology (ADEPT), implemented with Adobe RMSDK and licensed to various vendors, such as Kobo, Aldiko, PocketBook, etc.
DRM is Fundamentally Broken
As a copy-protection mechanism, DRM is fundamentally broken. In order to read an encrypted book, you need to decrypt the data. In order to decrypt the data, you need a key. So the whole DRM magic works only because you, as the end user, 1) don't care that your books are limited in functionality and 2) don't know where the key is stored, aka "security through obscurity".
But fear not, the companies who sell you DRM software know that, and they have a powerful law against anybody who figures out the obscure parts.
No person shall circumvent a technological measure that effectively controls access to a work protected under this title.
‘(b) ADDITIONAL VIOLATIONS .—(1) No person shall manufacture, import, offer to the public, provide, or otherwise traffic in any technology, product, service, device, component, or part thereof, that—
‘‘(A) is primarily designed or produced for the purpose of circumventing protection afforded by a technological measure that effectively protects a right of a copyright owner under this title in a work or a portion thereof;
‘(A) to ‘circumvent a technological measure’ means to descramble a scrambled work, to decrypt an encrypted work, or otherwise to avoid, bypass, remove, deactivate, or impair a technological measure, without the authority of the copyright owner; and
‘‘(B) a technological measure ‘effectively controls access to a work’ if the measure, in the ordinary course of its operation, requires the application of information, or a process or a treatment, with the authority of the copyright owner, to gain access to the work.
And in §1204:
§ 1204. Criminal offenses and penalties
‘‘(a) IN GENERAL.—Any person who violates section 1201 or 1202 willfully and for purposes of commercial advantage or private financial gain—
‘‘(1) shall be fined not more than $500,000 or imprisoned for not more than 5 years, or both, for the first offense; and
‘‘(2) shall be fined not more than $1,000,000 or imprisoned for not more than 10 years, or both, for any subsequent offense.
So when you figure out how to work around the DRM and (based on my reading) create tools to remove it, you'll be prosecuted, or will be forced to stop any research.
Back to technology
But casual users don't care about DMCA, and they buy an ebook, and everything works.
Until it does not.
It stopped working
I have a PocketBook Pro 912, released in 2011, and no firmware updates since 2012. It has Adobe Reader Mobile with ADE support, and having already paid for the hardware, I expected I will be able to use it to read the ePub files protected by whatever technology Adobe Digital Editions employed.
A while ago I bought an ebook, Android Programming - The Big Nerd Ranch Guide. I had some free time lately and I decided to resume reading about these Android Fragments (basically, one of the first chapters). It used to be on my reader, but at some point I may have deleted it. Google Play allows the book to be read in the browser, on the Android devices, and you can always download the file to read it on Adobe Digital Editions-compatible device:
Well, in my case the downloaded file did not turn into a book:
When you download a book served by the Adobe Content Server, you get an .acsm file which looks like this:
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <fulfillmentToken fulfillmentType="buy" xmlns="http://ns.adobe.com/adept"> <distributor>urn:uuid:9c47a658-956c-4e89-928f-9980443ae131</distributor> <operatorURL>http://example.com/fulfillment</operatorURL> <expiration>2016-02-30T00:00:00-00:00</expiration> <transaction>transaction-id</transaction> <userId>your-user-id</userId> <resourceItemInfo> <resource>urn:uuid:9a041f36-d50b-4f20-9593-aa661660aeb6</resource> <resourceItem>1</resourceItem> <src>https://example.com/9a041f36-d50b-4f20-9593-aa661660aeb6.epub</src> <downloadType>simple</downloadType> </resourceItemInfo> <hmac>c2VudCB2ZXJiYXRpbQo=</hmac> </fulfillmentToken>
It is expected that your reader will read this file, contact operatorURL, get a decryption key for the book, download the book from the src URL, update the downloaded epub with the data received from the fulfillment endpoint and then will finally decrypt the needed files and open the book. Updating epub is a crucial step to ensure that the book will be available offline, when there is no network connection to the activation servers (thanks, Adobe).
Salvaging my book
A naive approach of changing https to http for resourceItemInfo/src to avoid possible SSL connection issue caused an error (hmac mismatch?):
Feeling cheated by the authors for allowing DRM to exist, Addison-Wesley Professional for putting DRM on the book, and Google for changing something that used to work, I started looking into the communication between the device and the fulfillment service:
Having collected enough packets (in plaintext, no MITM required) from Google's ACS Fulfilment service, I decided to...
No, that would have been too straighforward.
I decided to do a stupid thing and try the only legal way to make it work: activate the book by planting the necessary DRM data into the book.
Giving DRM a helping hand
I used a backup of an activated DRM-ed book from BooksOnBoard as a reference (BooksOnBoard stopped operation in 2013, bringing their activation servers with them.)
The book difference was contained in a single file called rights.xml.
This is the file that gets planted by the RMSDK into the downloaded book (spec). It has the information about the user who obtained the book, the rights assigned, and more importantly, the encryption key used to decrypt the book.
<?xml version="1.0"?> <rights xmlns="http://ns.adobe.com/adept"> <licenseToken> <user>urn:uuid:570387b2-b87a-4b2d-8918-14281286752f</user> <resource>urn:uuid:75ed1798-4531-4d56-9808-84d32945f9a0</resource> <resourceItemType>application/epub+zip</resourceItemType> <voucher>no-idea</voucher> <licenseURL>https://nasigningservice.adobe.com/licensesign</licenseURL> <operatorURL>http://example.com/fulfillment</operatorURL> <encryptedKey>book-key-rsa-encrypted-with-your-private-key</encryptedKey> <permissions> <display/> </permissions> <signature>base64-encoded signature</signature> </licenseToken> <licenseServiceInfo> <licenseURL>https://nasigningservice.adobe.com/licensesign</licenseURL> <certificate>base64-encoded-certificate</certificate> </licenseServiceInfo> </rights>
The file consists of a licenseToken and licenseServiceInfo. According to my packet captures, the licenseToken is taken verbatim from the fullfilmentResult:
<envelope xmlns="http://ns.adobe.com/adept"> <fulfillmentResult> <fulfillment>ba8432ed-4475-446a-b60c-408e2055bb86-0000024d</fulfillment> <returnable>false</returnable> <initial>true</initial> ... <resourceItemInfo> ... <licenseToken> <user>urn:uuid:570387b2-b87a-4b2d-8918-14281286752f</user> <resource>urn:uuid:75ed1798-4531-4d56-9808-84d32945f9a0</resource> <resourceItemType>application/epub+zip</resourceItemType> <voucher>no-idea</voucher> <licenseURL>https://nasigningservice.adobe.com/licensesign</licenseURL> <operatorURL>http://example.com/fulfillment</operatorURL> <encryptedKey>book-key-rsa-encrypted-with-your-private-key</encryptedKey> <permissions> <display/> </permissions> <signature>base64-encoded signature</signature> </licenseToken> </resourceItemInfo> </fulfillmentResult> </envelope>
The licenseServiceInfo was apparently coming from somewhere else.
The standard location for Adobe DRM data is in the root of the e-reader under a folder called '.adobe-digital-editions'. I have not investigated the way the files 'device.xml' and 'activation.xml' appear on the device during provisioning, but there are some publicly available protocol dumps showing this process.
My assumption was correct, and the missing licenseServiceInfo was found in activation.xml:
<activationInfo> ... <adept:licenseServices xmlns:adept="http://ns.adobe.com/adept"> <adept:licenseServiceInfo> <adept:licenseURL>https://nasigningservice.adobe.com/licensesign</adept:licenseURL> <adept:certificate>MIIEvjCCA6...Vu5</adept:certificate> </adept:licenseServiceInfo> </adept:licenseServices> </activationInfo>
Zipping it up
Having constructed the seemingly valid rights.xml file, I downloaded the book from the Google Play src URL (overriding the user-agent - using wget triggered a 401 Unauthorized response) and patched it:
$ mkdir META-INF $ xmllint -format rights.xml > META-INF/rights.xml $ zip Android_Programming.epub META-INF/rights.xml adding: META-INF/rights.xml (deflated 36%)
And I was rewarded for my guesswork:
It worked, but it sent a very important signal:
Even if you have a license to read a DRM-protected ebook, you will find it unreadable at some point.
The software gets abandoned, the hardware stops receiving updates, activation servers disappear. Adobe has recently introduced a new DRM scheme requiring an always-on connection to really make sure you are not reading the book the way publisher did not intend you to. And when that server breaks, you'll be left with a digital paperweight.
Now, this method only works with ePub. Even though I downloaded the PDF from the Google Play Books as well, it is encrypted and I haven't yet figured out how to embed the relevant bits into the PDF structures to make it work with my reader.
There are multiple pain points here, but the worst is the existence of DRM. If you are an author, allowing the publisher to "protect" your work with a digital lock turns your readers into potential criminals.
Not a single user is thrilled to have the books they bought rendered unavailable just because the publisher believes piracy needs to be fought with a nuclear weapon, if needed.
Collateral damage? Pfft...
I, for one, can't even legally read the DRM'd books on my Linux laptop, and linux powers all these Kindles, Kobos, and... well, whatever other e-ink devices are available. What do I do then? I buy less books.
It does not help that Adobe knows about every book I download to my device as the fulfillment service needs to talk to Adobe to get the encrypted book key.
I really want to buy the books and own them.
And I can do that! There are publishers that decided to forego DRM completely - O'Reilly, Tor books, there is a Defective by Design web site dedicated to DRM with a list of places you can get DRM-free books from. There is a prominent Sci-Fi author, Cory Doctorow, who is a very vocal DRM opponent (Apollo 1201, and the talk at DEF CON 23 — it is really chilling).
The technical aspect of ePub DRM is a joke. The activation requests are sent over plain HTTP (probably to avoid root SSL certificate expiration), so can be trivially intercepted. The pages in the book are encrypted with AES128-CBC - it says so in META-INF/encryption.xml:
<EncryptedData xmlns="http://www.w3.org/2001/04/xmlenc#"> <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc"></EncryptionMethod> <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#"> <resource xmlns="http://ns.adobe.com/adept">urn:uuid:49a5ad8c-be33-873e-792b-fc62b58d075e</resource> </KeyInfo> <CipherData> <CipherReference URI="OEBPS/Front/cover.jpg"></CipherReference> </CipherData> </EncryptedData>
The decryption key can be found in rights.xml as encryptedKey. That key, in turn is RSA-encrypted with your privateLicenseKey in device's activation.xml. This is common knowledge, and there is software that does the decryption for you, provided that you bought the book. If you pirate a book, it will be already decrypted, so you will not have any problem reading it on any device:
Of course, I could just forego all that and read the book from the glowing screen of an Android tablet, I could probably read it directly from Google Play Books. But I want to read it on the bus and in the park.
Why can't I?