Folder Sets and Plausible Deniability

[tzugo]

Dear Tao Effects team,

I'd like to know more about the "plausible deniability" which is provided through folder sets.

If I understand correctly, the idea goes like this: If someone pushes me to unlock my secret data, I enter the Espionage 3 master password of a folder set which does only contain some "unimportant" encrypted folders, i.e. a folder set which I set up only for this very purpose of disclosing it in case I am forced to disclose "something". The stuff which I really want to hide will then go in one or more additional folder sets.

But how are folder sets stored internally? How do you make sure that the "oppressor" is not able to find out that more than one folder set exists?

Thank you,
tzugo

PS: IMO the fact that the encrypted sparsebundles themselves can be easily found (e.g. with "find / -name *.sparsebundle" in a Terminal.app window) is already quite a big threat to the "plausibility of the deniability" so to speak. I encourage you to proceed into this direction with a future version of Espionage, think about hiding encrypted images inside of 1080p movie files etc.  8)

[greg]

But how are folder sets stored internally? How do you make sure that the "oppressor" is not able to find out that more than one folder set exists?

It's certainly possible for a highly skilled adversary (e.g. some evil-geek at the NSA) to find out that there is more than one folder set, but that doesn't mean much. Folder Sets, once created, cannot be deleted, and are stored forever in Espionage's database. If you forget the password to a folder set, then you'll never be able to get it back, and you'll forever lose access to all of the folders that were stored in it (unless you copied their passwords manually and saved them somewhere).

For example, I honestly don't know how many Folder Sets I have in my database. I haven't kept track. I've used them temporarily for experimentation and testing Espionage. I've forgotten the passwords to most of them.

Just because someone discovers that you have multiple Folder Sets, doesn't mean they contain anything useful, or even that you know what's in them or remember their password. :)

PS: IMO the fact that the encrypted sparsebundles themselves can be easily found (e.g. with "find / -name *.sparsebundle" in a Terminal.app window) is already quite a big threat to the "plausibility of the deniability" so to speak. I encourage you to proceed into this direction with a future version of Espionage, think about hiding encrypted images inside of 1080p movie files etc.  8)

Yes, that's a valid point, however Espionage allows you to store the sparsebundles anywhere you want. You don't have to store them on your computer, that's up to you. If you want, you can store some of them on a USB stick, or someplace else.

The idea of disguising them as other types of files is interesting, however it comes with its own issues. Sparsebundles have files within them that have a sequence of bits that say, "I'm a sparsebundle!". So, in our example of the "evil-NSA geek", if they really wanted to get you, they'd be able to search the entire drive looking for that exact pattern of bits, and they'd find them.

We could do some trickery like encrypting the sparsebundle header files themselves, but I think this is already getting to an area that's a bit too involved and doesn't have much applicability to most users. If you are that paranoid, there are other things you can do that are basically just as effective, and not as involved and convoluted.

We have to prioritize what features we implement, and disguising sparsebundles as movie files, while a very interesting idea, would be difficult to get right, and simply doesn't increase the value of Espionage enough to justify the effort we'd have to put into it. Simply move the sparsebundles onto a thumb drive and breathe easy if that's a concern to you.

If you really want this feature, I think you could actually already do something like it with Espionage's Folder Actions. You, or someone else, could write a script that is run prior to unlocking a folder that extracts a sparsebundle out of some other file (be it a 1080p video or something else), and places it where Espionage expects it. Voila! :)

[tzugo]

We have to prioritize what features we implement, and disguising sparsebundles as movie files, while a very interesting idea, would be difficult to get right, and simply doesn't increase the value of Espionage enough to justify the effort we'd have to put into it.

Well, it is of course up to you to decide in which direction you want to bring Espionage forward. I liked it when version 3 introduced support for the important concept of "plausible deniability" and — in the assumption that you feel strongly about it, too — wanted to point out that there is room for improvement, should you wish to go this route. If your plans for Espionage are merely to deliver "convenient encryption for the masses", than that is fine for me, too. I know that I can, and perhaps should, continue "to roll my own" regarding strong encryption and steganography; I come from a Linux background and have been doing such kind of things on the command line for years. But I also very much enjoy tools like Espionage which have a nice UI and are pleasurable and convenient to use, and — not only now a few months after the uncovering of PRISM etc. — there should be a strong niche for programs like Espionage if they are committed to go some extra mile.

It's certainly possible for a highly skilled adversary (e.g. some evil-geek at the NSA) to find out that there is more than one folder set, but that doesn't mean much.

I'm afraid no "evil geek" is needed — we should assume the "relevant parties" to have a list of known software for the various operation systems and a little checklist about what to watch out for if e.g. the app Espionage is found on a suspect's system, and if there is evidence that the suspect is using multiple folder sets, they just might continue to torture him until all of them are revealed, right?

It should be easy to make the presence or absence of multiple folder sets completely undetectable: Just create a big file of, say, 1 megabyte, for Espionage to keep its settings which is initialized with random bytes. This file shall be separated in, say, 32 logical chunks — slots — of 32 kilobytes each. Whenever the user creates a folder set, he must select a slot number and a new master key. Espionage will then put its administrative data into the chunk of the file at offset 32kB*slot_number, encrypted with the new master key (which will cause this part of the file to still contain data that looks totally random as it only gains any meaning if the proper key is used to decrypt it). Whenever a key is entered to unlock Espionage, it will iterate thorugh all slots and try decrypt each with the key; if this succeeds for a slot, the config data (folder set information) to use is found.

The idea of disguising them as other types of files is interesting, however it comes with its own issues. Sparsebundles have files within them that have a sequence of bits that say, "I'm a sparsebundle!". So, in our example of the "evil-NSA geek", if they really wanted to get you, they'd be able to search the entire drive looking for that exact pattern of bits, and they'd find them.

Sure, but I am not suggesting a simplistic approach here. What e.g. if you used fixed-size encrypted disk images instead of sparsebundles, without any recognizable headers (i.e. a plain encrypted byte stream), and just put these in the middle of a 25 gigabyte Blu-Ray rip?¹ Sure, this movie would be damaged and no longer play correctly, but it should be hard to tell — for anyone who is lucky enough to detect it at all! — whether this "damage" originates from some hardware or software fault or whether this is important encrypted information, right?

Well, I do not want to persuade you into going somewhere you don't want to go with Espionage, but I think you have a little gem here with a lot of potential...

¹) This used to be easy to accomplish on Linux with the various kinds of loopback block devices, I assume there should be similar ways on OS X.

[greg]

I'm afraid no "evil geek" is needed — we should assume the "relevant parties" to have a list of known software for the various operation systems and a little checklist about what to watch out for if e.g. the app Espionage is found on a suspect's system, and if there is evidence that the suspect is using multiple folder sets, they just might continue to torture him until all of them are revealed, right?

Err, I'm not so sure. I'll expand on this next, but the basic idea would be to establish that it is common for Espionage 3 users to have extra folder sets that contain no info, outdated info, or are protected by forgotten passwords.

Whenever a key is entered to unlock Espionage, it will iterate thorugh all slots and try decrypt each with the key; if this succeeds for a slot, the config data (folder set information) to use is found.

This is a possibility, yes. We'll have to check whether doing this could cause Espionage to become too slow to unlock or not. If it doesn't noticeably affect the UX then we can change it to do it this way.

However, there's also an alternative approach that doesn't require changing the "database" (DB) format. We could simply fill DB with a random number of folder sets that contain a random number folders. Would that equivalently address your concern?

Sure, but I am not suggesting a simplistic approach here. What e.g. if you used fixed-size encrypted disk images instead of sparsebundles, without any recognizable headers (i.e. a plain encrypted byte stream), and just put these in the middle of a 25 gigabyte Blu-Ray rip?¹ Sure, this movie would be damaged and no longer play correctly, but it should be hard to tell — for anyone who is lucky enough to detect it at all! — whether this "damage" originates from some hardware or software fault or whether this is important encrypted information, right?

We removed support for sparseimages (fixed-sized encrypted disk images) in E3 because they're horrible for backup purposes. Most of the time, sparseimages are an inferior choice to sparsebundles, and therefore I have no plans on bringing them back without very good reason.

One of our goals for the E3-rewrite was to remove the number of choices users were faced with. Too many choices creates too much of an opportunity for bad decision making, and complicates the app. Each option is another UI widget or button or dialog the user must read, and for us, another potential bug or security risk.

As an example of what happens when encryption software gives users too many choices, I'll pick on GPG/PGP (which I use on a daily basis). One of its many flaws (IMO), is giving users the option of choosing various hashing algorithms for signing data. This is a terrible idea, and made all the worse by the fact that they appear to use SHA1 by default. Users shouldn't even know of the existence of these algorithms or concern themselves with what they're called. The software should gracefully and silently make the best decision on their part, so that users can get on with actually using the software. The average user hasn't even heard of GPG, and I'm fairly sure that's entirely on account of how terrible its UX is (and this includes all the software that I've seen that tries to put a GUI on it).

[greg]

I'd like to expand a little on the above:

We want E3 to be extremely simple for users to use, and that does mean limiting what bells and whistles it has to the essentials (and doing the best job we can at implementing them).

However, I am not opposed to accommodating the feature requests of advanced users such as yourself. In doing so, however, I want to make sure that we don't ruin E3 for a large segment of users who don't want a complicated app, and know very little about the details of data security.

One way of doing this would be to expand on the idea behind Folder Actions, and make it possible for third-party developers to write plugins for Espionage that expand its capabilities. We are very open ideas along those lines.

[tzugo]

However, there's also an alternative approach that doesn't require changing the "database" (DB) format. We could simply fill DB with a random number of folder sets that contain a random number folders. Would that equivalently address your concern?

Yes, that is in fact a good idea, an even better one than my own.

So only the problem of the too-easy-to-find and too-difficult-to-deny-plausibly encrypted sparsebundles is left. I agree that sparsebundles are a very nice and clever invention with a lot going for them, not only regarding backups. It's just that one would need to hide them better. I envision putting a sparsebundle directory in the middle of a big Aperture library, with its telltale files (Info.plist, Info.bckup) removed and the notorius file and directory names (.sparsebundle suffix, bands, tokens) obscured. Yes, this could be accomplished through folder actions.¹

I agree that PGP/GPG sucks big time because of its usage concept, I avoid it for that reason, too. I also agree that Espionage should continue to be very easy to use, but it should also make it easy for everyone to do more sophisticated stuff. Perhaps folder actions are really the proper way to go. Please just make sure that any script which is executed as part of a folder action is also encrypted along with the database part that defines the folder set. I.e. not only the list of mounts and actions should only be visible with the proper master key, also the "contents" of the action script(s) itself should be safely hidden. I guess this might bring Espionage a big step forward regarding advanced usage scenarios. Or does Espionage already work that way? (Haven't checked folder actions in detail yet.)


¹) Does OS X support the concept of "loopback mounts"? It would be cool to have all the .sparsebundle support files, i.e. everything except the contents of the bands directory, in a small fixed-size encrypted image which is mounted and then the bands directory (which bears some other name and is hidden somewhere between totally unrelated stuff) is mounted into it, yielding a complete sparsebundle again which can then be mounted in turn. Or perhaps a softlink instead of a loopback mount would already do?!

[greg]

Yes, this could be accomplished through folder actions.¹

Yes, we need to implement sending the path of the folder to any launched scripts, but that's easy to do.

the "contents" of the action script(s) itself should be safely hidden. I guess this might bring Espionage a big step forward regarding advanced usage scenarios. Or does Espionage already work that way? (Haven't checked folder actions in detail yet.)

At the moment it simply open any files or documents given to it (scripts can be easily turned into app bundles using Platypus). I'm not sure though why the script would need to be encrypted if it's operating on paths that are passed in to it (either via arguments or environment variables). Would that be OK or is there some scenario I'm not thinking of?

¹) Does OS X support the concept of "loopback mounts"? It would be cool to have all the .sparsebundle support files, i.e. everything except the contents of the bands directory, in a small fixed-size encrypted image which is mounted and then the bands directory (which bears some other name and is hidden somewhere between totally unrelated stuff) is mounted into it, yielding a complete sparsebundle again which can then be mounted in turn. Or perhaps a softlink instead of a loopback mount would already do?!

Is that the same thing as a "union mount"? I think it might have at one point... but I'm not sure whether it still does.

Still, I'm not sure how you'd obscure the bands themselves. They're fairly easy to spot on their own, and I honestly have no idea how you'd hide them without at least renaming them. Renaming them would be a problem of course, because that would likely mess up backups (you'd most likely be forced to backup the entire sparsebundle each time).

[greg]

Hey tzugo, just an fyi, this feature is coming in the next update! :)

Thank you for the inspiration!

[tzugo]

Hey, that's great news. Way to go! I also just read today's blog entry of yours about this topic.  :)

And thank you for pointing out the union mounts to me — I didn't know the Mac had them, too (they do still exist). Very useful.

Keep up the good work,
tzugo

PS: Are you going to encrypt the Folder Action scripts along with the Folder Sets database in the upcoming or a future version of Espionage? If one uses a Folder Action script to further obfuscate one's sparsebundles that code needs protection, too, and the best and safest place I can think of should be the encrypted Folder Set definition...

[greg]

Hey, that's great news. Way to go! I also just read today's blog entry of yours about this topic.  :)

And thank you for pointing out the union mounts to me — I didn't know the Mac had them, too (they do still exist). Very useful.

Keep up the good work,
tzugo

Glad to hear of your approval! :)

PS: Are you going to encrypt the Folder Action scripts along with the Folder Sets database in the upcoming or a future version of Espionage? If one uses a Folder Action script to further obfuscate one's sparsebundles that code needs protection, too, and the best and safest place I can think of should be the encrypted Folder Set definition...

Folder Actions are already encrypted along with the rest of the information about the folders. You could store a script that's run within one of your other encrypted folders...

[greg]

This feature has been implemented! :)

Released today in Espionage 3.6:

https://www.taoeffect.com/blog/2014/07/major-advancements-in-deniable-encryption-arrive-in-espionage-3-6/

[Murraci]

PS: IMO the fact that the encrypted sparsebundles themselves can be easily found (e.g. with "find / -name *.sparsebundle" in a Terminal.app window) is already quite a big threat to the "plausibility of the deniability" so to speak. I encourage you to proceed into this direction with a future version of Espionage, think about hiding encrypted images inside of 1080p movie files etc.  8)

I think this is a very important point. I know Greg acknowledged it but I'd like to explore it a little more. Before its mysterious demise, I used TrueCrypt. When creating a hidden volume with it, there is zero trace on my computer of this hidden volume or its contents. It seems irrelevant to me to allow the creation of multiple folder sets if the a sparsebundle of the encrypted contents of these folders is located in plain site on my computer for all to see. That's not plausible deniability. In fact quite the opposite, there's zero deniability to the existence of encrypted contents in my Espionage folders. Or am I missing a trick?

[greg]

I think this is a very important point. I know Greg acknowledged it but I'd like to explore it a little more. Before its mysterious demise, I used TrueCrypt. When creating a hidden volume with it, there is zero trace on my computer of this hidden volume or its contents. It seems irrelevant to me to allow the creation of multiple folder sets if the a sparsebundle of the encrypted contents of these folders is located in plain site on my computer for all to see. That's not plausible deniability. In fact quite the opposite, there's zero deniability to the existence of encrypted contents in my Espionage folders. Or am I missing a trick?

It sounds like there's misunderstanding going on.

You have very strong Plausible Deniability with Espionage: better than the PD you get from TrueCrypt.

[Murraci]

Apologies. I forgot about the creation of fake data.

[zsolt]

I'm glad it looks better to you now. If you have any further suggestions, just let us know.