# NSA-proof your e-mail in 2 hours

25 June 2013 Published in: Code 140 comments

You may be concerned that the NSA is reading your e-mail. Is there really anything you can do about it though? After all, you don’t really want to move off of GMail / Google Apps. And no place you would host is any better.

Except, you know, hosting it yourself. The way that e-mail was originally designed to work. We’ve all just forgotten because, you know, webapps-n-stuff. It’s a lot of work, mkay, and I’m a lazy software developer.

Today we kill your excuses. Because I’m going to show you exactly how to do it, it’s going to take about two hours to set up, and it’s a “set it and forget it” kind of setup. Not only that, but it is actually going to be better than GMail, from a purely features perspective. It might surprise you to learn that people continue to develop email server software in a post-Google-apps world, and that the state of self-hosted is much better than you remember.

Now fair warning: it took me about two days to figure the stuff out you’re going to see in this blogpost, starting from knowing basically nothing about modern e-mail servers. But now that I’ve figured it out, if you don’t ask too many questions you can implement it from these notes in just two hours. So take this not just as a guide for setting up an e-mail server, but as two days of free consulting, that just happens to produce a complete recipe for a modern, fully-featured, fast email server at the end. You’re really going to turn down free consulting? Come on, buckle down and do this.

So bookmark this blog post, block off a Saturday next month, and get it done. Seriously. If you are still using GMail (or Yahoo, or arbitrary US-based email company) in August, your right to complain about the NSA spying on you is revoked. If you’re complaining about government spying on the Internet, or in a gathering of programmers, and you won’t take basic steps to do anything about it, then you’re a hypocrite, full-stop. I will personally come to your terminal and demand the return of your complain license. Pick a weekend, get it done. Or just admit that you’re okay with it. Either way, just be consistent.

Edit Some people are complaining that the NSA is pulling all the e-mail over the wire anyway, so encrypting your own server is stupid. However, these people are not aware of just how much e-mail goes over TLS. For example, if you follow this guide, every message to or from a GMail / Google Apps account goes over TLS. I measured it, and that’s a third of my ham volume. And Google isn’t a special snowflake; any reasonably-configured mail will be encrypted over the wire. It’s 2013; people have been getting TLS into wide adoption for awhile now.

In the next two hours, we’re going to fix this.  You’re going to build a modern, 2013, e-mail stack. From scratch. Or something.

• You’re going to host your own mail.
• It’s going to be encrypted on the server, locked-on-boot, SSH on reboots to unlock
• While we’re at it, let’s fix some things that annoy me about GMail:
• Better SPAM detection.  Yes, you can beat the Big G.
• Push support that actually works.  Little known fact: Google discontinued push for iOS earlier this year.  Also little known fact: push has never worked in OSX Mail.app.  We’re gonna fix it.  Lightning fast push on all the devices.
• Full-text search. Believe it or not, mail search on my iPhone is pretty bad under GMail. Three out of four Google Suggest results for “iPhone mail search” are “doesn’t work”, “not working”, and “slow”. Our version is going to be working, working, and fast. Take that, Google.

I’m going to assume:

• You’re running Debian Wheezy
• You know your way around Linux more or less. Experience running Apache or Lighttpd or Nginx, etc.
• You have root access
• Nobody really cares if you mess anything up.
• You’re not stupid and you know how to google things

Hang on just a minute though. Doesn’t hosting your own mail require beefy hardware to deal with the spams? Not really. I get a lot of mail, and this is my graph:

Doesn’t it go down a lot though? Getting up in the middle of the night to do server reboots and that?

Again, not really. RFC 2821 requires that mail servers try for “at least 4-5 days” to deliver your mail. Let’s just say, that’s a lot more forgiving than running Apache.

# A crash course on e-mail

The people who have designed e-mail software are, on the whole, people with a lot more problems than you.  Unfortunately this means there are a lot of separate little projects that work together.

• Postfix – this is called an “MTA” in the email lingo.  Essentially it runs SMTP, and delivers incoming mail to Dovecot.
• Dovecot – this is the “LDA” in the email lingo.  Essentially it runs IMAP.
• EncFS – this is what we use to encrypt our email store
• opendkim – implements DKIM authentication
• spamd – antispam
• z-push – better push support
• mySQL – really we’re going to use this to hold information about users and domains, but nothing really sensitive.

Let’s get started!

# Security Foreward

I assume you kinda-sorta know how to configure a secure server on the Internet and have done so for your mail server. If you don’t know what I’m talking about, time to read my first 5 minutes on a server.

# DNS

We should probably start configuring your DNS, since that stuff takes awhile. If you have an existing domain, you might want to lower the TTL on your MX records to the smallest possible setting. That way you can switch over when you’re ready.

For this article, I’m going to set up my mailserver running on a subdomain – drew@awesomebox.sealedabstract.com. So I configure an MX record that points mail for the subdomain to my server.

# EncFS

$apt-get install encfs mkdir /encrypted-mail /decrypted-mail chgrp mail /decrypted-mail/ chmod -R g+rw /decrypted-mail/ gpasswd -a mail fuse chgrp fuse /dev/fuse; chmod g+rw /dev/fuse root@li212-205:~# encfs /encrypted-mail /decrypted-mail -o --public Creating new encrypted volume. Please choose from one of the following options: enter "x" for expert configuration mode, enter "p" for pre-configured paranoia mode, anything else, or an empty line will select standard mode. ?> p Paranoia configuration selected. Configuration finished. The filesystem to be created has the following properties: Filesystem cipher: "ssl/aes", version 3:0:2 Filename encoding: "nameio/block", version 3:0:1 Key Size: 256 bits Block Size: 1024 bytes, including 8 byte MAC header Each file contains 8 byte header with unique IV data. Filenames encoded using IV chaining mode. File data IV is chained to filename IV. File holes passed through to ciphertext. -------------------------- WARNING -------------------------- The external initialization-vector chaining option has been enabled. This option disables the use of hard links on the filesystem. Without hard links, some programs may not work. The programs 'mutt' and 'procmail' are known to fail. For more information, please see the encfs mailing list. If you would like to choose another configuration setting, please press CTRL-C now to abort and start over. Now you will need to enter a password for your filesystem. You will need to remember this password, as there is absolutely no recovery mechanism. However, the password can be changed later using encfsctl. New Encfs Password: Verify Encfs Password:  It’s that simple. /decrypted-mail is now a regular directory. /encrypted-mail is that same data, just encrypted. It’s important to use the --public option with EncFS. This is because by default, EncFS goes to great lengths to be a lot narrower than the standard UNIX permissions model. Meanwhile, we have a lot of UNIX users who expect standard UNIX behaviors. Any time we want to mount the /decrypted-mail dir, we run the same encfs command again, and this time it mounts the folder, instead of creating it. ## OpSec It’s important as a practical matter that you keep other people off your root. It goes without saying, but there are a lot of attack vectors (like physical access, or access through a VPS admin panel) that could potentially allow your host, acting under the direction of a government agent, to run commands as root, and take your mail. For security reasons, I’m not going to disclose the measures that I take to avoid others gaining root on my system. A good start might be changing your root password, or keeping your mail server under your pillow at night. # Postfix apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-mysql mysql-server dovecot-lmtpd  Postfix prompts you with a bunch of information–the one that you want is “Internet Site”. It’s also the default. Postfix prompts you for a “mail name” – I chose awesomebox.sealedabstract.com. About 60 seconds later, you’ve got a mail server. Now to configure it. First we need to derive our password. doveadm pw -s SHA512-CRYPT  This will give you a string like {SHA512-CRYPT}$6$gJ8hXjMn/lePALEt$JMX1jd... The part after “{SHA512-CRYPT}” is the hash for your password. It always starts with “$6$”.

mysqladmin -p create mailserver
mysql -p mailserver
mysql> GRANT SELECT ON mailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mailuserpass';
mysql> FLUSH PRIVILEGES;

mysql> CREATE TABLE virtual_domains (
id int(11) NOT NULL auto_increment,
name varchar(50) NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

mysql> CREATE TABLE virtual_users (
id int(11) NOT NULL auto_increment,
domain_id int(11) NOT NULL,
password varchar(106) NOT NULL,
email varchar(100) NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY email (email),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

mysql> CREATE TABLE virtual_aliases (
id int(11) NOT NULL auto_increment,
domain_id int(11) NOT NULL,
source varchar(100) NOT NULL,
destination varchar(100) NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

mysql> INSERT INTO mailserver.virtual_domains
(id ,name)
VALUES
('1', 'awesomebox.sealedabstract.com');


You could also configure additional domains here if desired.

mysql> INSERT INTO mailserver.virtual_users
(id, domain_id, password , email)
VALUES
('1', '1', '$6$YOURPASSWORDHASH', 'drew@awesomebox.sealedabstract.com');


Again, you could also configure multiple users here.

mysql> exit


Okay, now let’s configure postfix. Let’s back up the original file:

cp /etc/postfix/main.cf /etc/postfix/main.cf.orig
nano /etc/postfix/main.cf


Comment out the “TLS parameters”, and use these instead:

smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
smtp_tls_loglevel = 2


Below the TLS parameters, paste these:

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

smtpd_recipient_restrictions =
permit_sasl_authenticated,
permit_mynetworks,
reject_unauth_destination


There’s a line that sets “mydestination” to a wide variety of domains. Make sure it’s only set to localhost.

mydestination = localhost


At the very bottom of the file, paste this:

virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
local_recipient_maps = $virtual_mailbox_maps  This essentially points postfix at Dovecot and the MySQL tables. Okay, so now we’re going to create the three files we specified earlier. nano /etc/postfix/mysql-virtual-mailbox-domains.cf  . user = mailuser password = mailuserpass hosts = 127.0.0.1 dbname = mailserver query = SELECT 1 FROM virtual_domains WHERE name='%s'  Similarly, for /etc/postfix/mysql-virtual-mailbox-maps.cf: user = mailuser password = mailuserpass hosts = 127.0.0.1 dbname = mailserver query = SELECT 1 FROM virtual_users WHERE email='%s'  Finally, for /etc/postfix/mysql-virtual-alias-maps.cf: user = mailuser password = mailuserpass hosts = 127.0.0.1 dbname = mailserver query = SELECT destination FROM virtual_aliases WHERE source='%s'  So let’s restart postfix: service restart postfix  And verify that it works: $ postmap -q awesomebox.sealedabstract.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
1
$postmap -q drew@awesomebox.sealedabstract.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf 1  Looks good. # Dovecot So first, let’s backup all the config files: cp /etc/dovecot/dovecot.conf /etc/dovecot/dovecot.conf.orig cp /etc/dovecot/conf.d/10-mail.conf /etc/dovecot/conf.d/10-mail.conf.orig cp /etc/dovecot/conf.d/10-auth.conf /etc/dovecot/conf.d/10-auth.conf.orig cp /etc/dovecot/dovecot-sql.conf.ext /etc/dovecot/dovecot-sql.conf.ext.orig cp /etc/dovecot/conf.d/10-master.conf /etc/dovecot/conf.d/10-master.conf.orig cp /etc/dovecot/conf.d/10-ssl.conf /etc/dovecot/conf.d/10-ssl.conf.orig  And edit the main one: nano /etc/dovecot/dovecot.conf  Down at the bottom, we’re going to enable imap: protocols = imap  So far, so good. Now edit another config file: nano /etc/dovecot/conf.d/10-mail.conf  We patch the variables listed below: mail_location = maildir:/decrypted-mail/%d/%n mail_privileged_group = mail first_valid_uid = 0  Now we edit the auth file: nano /etc/dovecot/conf.d/10-auth.conf  Here are the values we patch: disable_plaintext_auth = yes auth_mechanisms = plain login #INSERT a hashtag in front of the following import. This separates your mail server's login from UNIX logins. #!include auth-system.conf.ext #REMOVE the hashtag in front of the following import. This points it at mysql for authentication. !include auth-sql.conf.ext  Now let’s configure that SQL in more detail: nano /etc/dovecot/conf.d/auth-sql.conf.ext  . passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = static args = uid=mail gid=mail home=/decrypted-mail/%d/%n }  Edit yet another config file nano /etc/dovecot/dovecot-sql.conf.ext  . driver = mysql connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailuserpass default_pass_scheme = SHA512-CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';  Now I know what you’re thinking. SHA512? Why not bcrypt? In theory dovecot supports the argument BLF-CRYPT here (e.g. bcrypt) but in practice only if your libc provides bcrypt. SUSE is the only Linux that has bcrpt in its libc. This topic comes up enough that Ulrich Drepper has an entire webpage on this topic. In practice you can do any of the following: A) use SHA512-CRYPT, B) Switch to SUSE, C) Switch to Real Unix, D) whine at Ulrich Drepper, E) whine at NIST. I’ve chosen choice A. If it makes you feel any better, it’s been vetted by HP, IBM, RH, and Sun. chown -R mail:dovecot /etc/dovecot chmod -R o-rwx /etc/dovecot  . nano /etc/dovecot/conf.d/10-master.conf  We’re setting our ports to zero. This essentially forces people to use “secure” sockets. You know, cause we’re paranoid. service imap-login { inet_listener imap { port = 0 } … service pop3-login { inet_listener pop3 { port = 0 } ...  For the service lmtp, we use as follows: service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 group = postfix user = postfix } # Create inet listener only if you can't use the above UNIX socket #inet_listener lmtp { # Avoid making LMTP visible for the entire internet #address = #port = #} user=mail }  We’ll replace the whole “service auth” and “service auth-worker” section with this one: service auth { # auth_socket_path points to this userdb socket by default. It's typically # used by dovecot-lda, doveadm, possibly imap process, etc. Its default # permissions make it readable only by root, but you may need to relax these # permissions. Users that have access to this socket are able to get a list # of all usernames and get results of everyone's userdb lookups. unix_listener /var/spool/postfix/private/auth { mode = 0666 user = postfix group = postfix } unix_listener auth-userdb { mode = 0600 user = mail #group = } # Postfix smtp-auth #unix_listener /var/spool/postfix/private/auth { # mode = 0666 #} # Auth process is run as this user. user = dovecot } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to #$default_internal_user.
user = mail
}


Generate those SSL certs:

openssl req -new -x509 -days 1000 -nodes -out "/etc/ssl/certs/dovecot.pem" -keyout "/etc/ssl/private/dovecot.pem"


And set them in the config file:

nano /etc/dovecot/conf.d/10-ssl.conf
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem
ssl = required


Let’s kick the server:

service dovecot restart


At this point, it should basically work. You should be able to send and receive mail. Go ahead and try it! You should be running IMAP on 993 (standard SSL port) and you should be able to log in with the e-mail and password you set.

Note that you can get debug information with

tail /var/log/mail.log


You can also test “over-the-wire TLS”. When you send a message, mail.log should show this:

Jul 15 19:20:52 li212-205 postfix/smtp[17453]: Untrusted TLS connection established to ASPMX.L.GOOGLE.com[2607:f8b0:4003:c02::1b]:25: TLSv1.2 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)


When you receive a message, this header will appear:

Received: from mail-ob0-f169.google.com (mail-ob0-f169.google.com [209.85.214.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mailserver.com (Postfix) with ESMTPS id 7F90E1CC71D for <drew@sealedabstract.com>; Mon, 15 Jul 2013 19:28:07 -0400 (EDT)


However, we should set up some more things.

# Getting through SPAM filters

If setting up an e-mail server was that easy, spammers would do it. Alas.

You should go read Jeff Atwood’s ‘so you’d like to send some e-mail’ for the details, but we’re going to set up SPF, OpenDKIM, and (maybe) PTR.

## OpenDKIM

apt-get install opendkim opendkim-tools
mkdir -pv /etc/opendkim/
chown -Rv opendkim:opendkim /etc/opendkim
chmod go-rwx /etc/opendkim/*
cd /etc/opendkim/
opendkim-genkey -r -h rsa-sha256 -d awesomebox.sealedabstract.com -s mail
mv -v mail.private mail
cat mail.txt


This will output our DKIM key to the terminal. Then, we install it on our DNS server. My ZONE file looks like this. Unfortunately it doesn’t wrap very well in this blog post.

mail._domainkey.awesomebox      300 TXT "v=DKIM1; h=rsa-sha256; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDbLRiNXX9zxAtdw45Vsd35d/1VQZUFG8ejfQu6aql74obJhTESaqANBbOkNaD0xb+2kuN/w+2725Gv2tTPAcHfBZogyazkwtNrlNQV2h3q4ub/UTkn0AHeq0P/RMtmhV+hKRzk0hcYWPlzWMoR5ZGWwMYdhbocTeBX4Mc2pWEYewIDAQAB"


Returning to our Wheezy box, let’s configure postfix to use that key:

nano /etc/opendkim/KeyTable
awesomebox.sealedabstract.com awesomebox.sealedabstract.com:mail:/etc/opendkim/mail


.

nano  /etc/opendkim/SigningTable
*@awesomebox.sealedabstract.com awesomebox.sealedabstract.com


.

nano /etc/opendkim/TrustedHosts
127.0.0.1


.

nano /etc/opendkim.conf
##
## opendkim.conf -- configuration file for OpenDKIM filter
##
Canonicalization        relaxed/relaxed
ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
LogWhy                  Yes
MinimumKeyBits          1024
Mode                    sv
PidFile                 /var/run/opendkim/opendkim.pid
SigningTable            refile:/etc/opendkim/SigningTable
Socket                  inet:8891@localhost
Syslog                  Yes
SyslogSuccess           Yes
TemporaryDirectory      /var/tmp
UserID                  opendkim:opendkim


Now we return to /etc/postfix/main.cf and add (to the bottom)

smtpd_milters           = inet:127.0.0.1:8891
non_smtpd_milters       = $smtpd_milters milter_default_action = accept  Allright, let’s check it out! service opendkim restart service postfix restart  ## SPF SPF is simpler. All you have to do is edit your zone: awesomebox 300 TXT "v=spf1 mx -all"  All I am saying here is “the mail server for awesomebox can send mail for awesomebox” You’d think it would be obvious, but recall at the start, the observation that this system was designed by people who have far more problems than you. At scale, you probably have more than one mail server. You can also debug your DKIM (and SPF) settings with e.g. this tool. ## Reverse PTR This is not as hard as Jeff leads you to believe. If you’re hosting with Linode (and you should) it’s right on the config panel. Linodes->[name]->Remote Access->Reverse DNS->Type in your hostname. Done. # Spam Let’s move on to everybody’s favorite: fighting spam. The inbound kind. People argue about what is the best antispam tool. Argument over: dspam. I’ve found it quite a bit better than e.g. SpamAssassin and the other traditional tools. It takes some time to train, so it doesn’t work “right out of the box”. But after a few days, my spam detection has been great. apt-get install dspam dovecot-antispam postfix-pcre dovecot-sieve  patch these in /etc/dspam/dspam.conf: Home /decrypted-mail/dspam TrustedDeliveryAgent "/usr/sbin/sendmail" UntrustedDeliveryAgent "/usr/lib/dovecot/deliver -d %u" Tokenizer osb IgnoreHeader X-Spam-Status IgnoreHeader X-Spam-Scanned IgnoreHeader X-Virus-Scanner-Result IgnoreHeader X-Virus-Scanned IgnoreHeader X-DKIM IgnoreHeader DKIM-Signature IgnoreHeader DomainKey-Signature IgnoreHeader X-Google-Dkim-Signature ParseToHeaders on ChangeModeOnParse off ChangeUserOnParse full ServerPID /var/run/dspam/dspam.pid ServerDomainSocketPath "/var/run/dspam/dspam.sock" ClientHost /var/run/dspam/dspam.sock  Of course, we have to create the directory in question mkdir /decrypted-mail/dspam chown dspam:dspam /decrypted-mail/dspam  As best as I can tell, the preferences in /etc/dpsam/dspam.conf are completely ignored. If you want to edit them, the right place is /etc/dspam/default.prefs. Totally logical, right? Anyway, patch these values: spamAction=deliver # { quarantine | tag | deliver } -> default:quarantine signatureLocation=headers # { message | headers } -> default:message showFactors=on  Okay, now we’re editing /etc/postfix/master.cf. These go at the end. dspam unix - n n - 10 pipe flags=Ru user=dspam argv=/usr/bin/dspam --deliver=innocent,spam --user$recipient -i -f $sender --$recipient
dovecot   unix  -       n       n       -       -       pipe
flags=DRhu user=mail:mail argv=/usr/lib/dovecot/deliver -f ${sender} -d${recipient}


And now:

nano /etc/postfix/dspam_filter_access
/./   FILTER dspam:unix:/run/dspam/dspam.sock


Now we’ll patch the end of /etc/postfix/main.cf

# new settings for dspam
dspam_destination_recipient_limit = 1 #only scan one mail at a time
smtpd_client_restrictions =
permit_sasl_authenticated #localhost doesn't get scanned
check_client_access pcre:/etc/postfix/dspam_filter_access #run dspam on everything else


Integrating dspam with imap:

nano /etc/dovecot/conf.d/20-imap.conf
mail_plugins = $mail_plugins antispam  Integrating dspam with lmtp: protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins =$mail_plugins sieve
}


Telling sieve to move spam into a Spam folder. Edit /decrypted-mail/awesomebox.sealedabstract.com/drew/.dovecot.sieve (e.g. for your username and domain)

require ["regex", "fileinto", "imap4flags"];

# Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox
if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$", not header :contains "X-DSPAM-Reclassified" "Innocent") { # Mark as read setflag "\\Seen"; # Move into the Junk folder fileinto "Spam"; # Stop processing here stop; }  And then we’ll configure /etc/dovecot/conf.d/90-plugin.conf. Note that, there’s just one plugin {} dict, and this stuff goes inside it. plugin { ... # Antispam (DSPAM) antispam_backend = dspam antispam_allow_append_to_spam = YES antispam_spam = Spam;Junk antispam_trash = trash;Trash antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_dspam_binary = /usr/bin/dspam antispam_dspam_args = --user;%u;--deliver=;--source=error antispam_dspam_spam = --class=spam antispam_dspam_notspam = --class=innocent antispam_dspam_result_header = X-DSPAM-Result }  Give postfix and dovecot a kick: service postfix restart service dovecot restart  And we’re good to go. Incoming messages should have headers like X-Dspam-Result: Innocent X-Dspam-Processed: Wed Jun 12 21:46:08 2013 X-Dspam-Confidence: 0.9899 X-Dspam-Probability: 0.0000 X-Dspam-Signature: 51b9246071121935811689 X-Dspam-Factors: 27, Received*12+Jun, 0.01000, Received*12+Jun, 0.01000, Received*Postfix+with, 0.01000, Received*with+#+id, 0.01000, Received*awesomebox.sealedabstract.com+#+12, 0.01000, Content-Type*text+plain, 0.01000, Received*Postfix+#+ESMTP, 0.01000, Received*li212-205.members.linode.com+Postfix, 0.01000, Received*drew+#+#+#+Jun, 0.01000, Received*Wed+#+Jun, 0.01000, Received*Wed+#+Jun, 0.01000, Received*li212-205.members.linode.com+#+with, 0.01000, Received*Wed+#+#+2013, 0.01000, Received*Wed+#+#+2013, 0.01000, Received*Postfix+#+#+id, 0.01000, Received*li212-205.members.linode.com+#+#+#+id, 0.01000, Received*ESMTP+id, 0.01000, Date*12+Jun, 0.01000, Received*for+#+#+#+12, 0.01000, Date*Jun+2013, 0.01000, Received*by+#+Postfix, 0.01000, Received*by+#+with, 0.01000, Received*awesomebox.sealedabstract.com+#+#+Jun, 0.01000, Received*by+#+#+#+ESMTP, 0.01000, Date*Wed+#+#+2013, 0.01000, Received*drew+#+#+12, 0.01000, Received*for+#+awesomebox.sealedabstract.com, 0.01000  To train the message as spam, move it to a folder called “Spam” or “Junk” on any device. Otherwise it’ll be trained as ham. # Push So long story short, push is complicated. On IMAP, people generally use “IMAP IDLE”, which is basically a fancy way of saying they leave the IMAP connection open all the time. Alas, it turns out to be bad for mobile devices in a lot of dimensions, although you can configure it to be better. Anyway, the iPhone doesn’t support it, ostensibly on battery life grounds. In theory Apple Mail supports IDLE on OSX, but in practice there is some debate about this. In my experience there is some incompatibility between GMail’s IDLE and Apple Mail’s IDLE, but Dovecot and Mail.app seem to get along just fine. I’m calling it now and blaming PRISM. Believe it or not, Apple actually uses Dovecot as part of OSX Server–which obviously seems to manage push mail with the whole Apple universe of platforms. As a result, Apple has an implementation of push-on-Dovecot that you can in fact replicate. However, mostly it shuffles data off to its closed-source tool. On the other hand, the closed-source tool mostly does a thing that’s fairly well-known involving the APN system, so it wouldn’t be that hard to clone. However, there’s an even simpler solution. There’s a tool called “z-push” (Debian rebrands it “d-push” on trademark grounds) that emulates Exchange ActiveSync, much like the old Google Sync did. In fact, I’m fairly certain you could use it to get the Google Sync behavior on GMail again if you wanted. apt-get install d-push  Now Exchange uses HTTPS. Which means you need an HTTP server. I happen to use lighttpd already, so that’s what I’m going to use. I’m told it’s “designed for” Apache, so it’s probably easy to configure with that server too. Patch these in /etc/d-push/config.php: define('IMAP_PORT', 993); define('IMAP_OPTIONS', '/ssl/novalidate-cert'); //only use this when IMAP_SERVER is 'localhost'! define('STATE_DIR', '/decrypted-mail/dpush-state/'); define('IMAP_SENTFOLDER', 'Sent Messages');  My lighttpd config rewrites all incoming URLs to wordpress by default. It’s how you get really great URLs like the ones on the top of this post. So I have to “opt out” the Exchange stuff from this system. If you don’t use mod_rewrite, you won’t have to do this. Editing /etc/lighttpd/lighttpd.conf: url.rewrite-once = ( "^/(Microsoft-Server-ActiveSync*)(\?.*)?$" => "$1$2",
…
)


Note that, if you’re not careful with the rewrite rules, lighttpd will slice the params off the URL before they get passed to d-push. Then you get errors like this:

[INFO] Version='2.0-1.1' method='GET' from='XXXX' cmd='' getUser='drew@awesomebox.sealedabstract.com' devId='' devType=''
[FATAL] FatalException: Requested the d-push URL without the required GET parameters - code: 0


Several people seem to be having this problem on the Internet.

If you don’t have it already, you need mod_alias listed under server.modules in your /etc/lighttpd/lighttpd.conf. I think it is there by default, but I’m not entirely sure.

Now you need to point lighttpd to dpush. Basically anywhere in your /etc/lighttpd/lighttpd.conf, do this:

alias.url += ("/Microsoft-Server-ActiveSync" => "/usr/share/d-push/index.php")


If you don’t have an SSL cert, you can generate one from bash:

openssl req -x509 -newkey rsa:2048 -keyout lighttpd-key.pem -out lighttpd-cert.pem -days 1001 -nodes
cat lighttpd-key.pem lighttpd-cert.pem > lighttpd-all.pem


Back in nano, let’s turn that SSL on, if it’s not on already:

$SERVER["socket"] == ":443" { ssl.engine = "enable" ssl.pemfile = "/etc/ssl/lighttpd-all.pem" }  And give lighttpd a kick: service lighttpd restart  There you go. At this point you should be able to connect via the “Exchange” settings on an iPhone. A quick note that I needed a patch /usr/share/d-push/backend/imap/imap.php to get my “Deleted Messages” folder working: else if($lid == "trash" || $lid == "deleted messages") {$folder->parentid = "0";
$folder->displayname = "Trash";$folder->type = SYNC_FOLDER_TYPE_WASTEBASKET;
$this->wasteID =$id;
}


A quick note on debugging — the z-push documentation suggests that to get logs you need to create some file called “debug.txt” in an unspecified location and then chmod it 777. The Debian Gods, however, have seen fit to give us a /var/log/d-push/d-push.log file to check instead. You can also check /var/log/lighttpd/error.log and access.log at the same path.

# Full-text search

Unfortunately, we need Java.

apt-get install solr-tomcat dovecot-solr


Due to some bug, you have to pull down solr-schema.xml from the orig.tar.gz package, linked here.

cp /path/to/solr-schema.xml /etc/solr/conf/schema.xml
service tomcat6 restart


Back to patching /etc/dovecot/conf.d/20-imap.conf. The “antispam” is there from when we installed dspam.

mail_plugins = $mail_plugins antispam fts fts_solr  Now we hit /etc/dovecot/conf.d/90-plugin.conf: plugin { ... fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ }  “break-imap-search” is a command that essentially says we should actually do full-text-search, which is against the IMAP specification. VERY IMPORTANT. By default, tomcat6 is globally accessible, which means just anybody with a web browser can query your mail! We need to turn this off. Inside /etc/tomcat6/server.xml, there is a line called <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" URIEncoding="UTF-8" redirectPort="8443" />  We need to add address="127.0.0.1" onto that: <Connector address="127.0.0.1" port="8080" protocol="HTTP/1.1" connectionTimeout="20000" URIEncoding="UTF-8" redirectPort="8443" />  For reasons that are not immediately clear to me, Solr comes with some queries out of the box that are configured for the schema that it ships with. You will see things in your log like this: SEVERE: org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1330) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getAnalyzer(IndexSchema.java:408)
at org.apache.solr.schema.IndexSchema$SolrIndexAnalyzer.reusableTokenStream(IndexSchema.java:383) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:574) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:206) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1429) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1317) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79) at org.apache.solr.search.QParser.getQuery(QParser.java:143) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:105) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:59) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1182)


To fix this, you need to place <!-- and --> around text in /etc/solr/conf/solrconfig.xml. It should look like this:

<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<!--<str name="df">text</str>-->
</lst>


We should also patch the data directory to be on the encrypted partition:

<dataDir>/decrypted-mail/solr</dataDir>


And give it the proper permissions:

mkdir /decrypted-mail/solr
chown -R tomcat6:tomcat6 /decrypted-mail/solr


Give tomcat and dovecot a kick:

service dovecot restart
service tomcat6 restart


I’ve found that it sometimes helps to remove dovecot’s indexes:

rm -f /decrypted-mail/awesomebox.sealedabstract.com/drew/dovecot.index*
rm -f /decrypted-mail/awesomebox.sealedabstract.com/drew/.MyMailboxName1/dovecot.index*
rm -f /decrypted-mail/awesomebox.sealedabstract.com/drew/.MyMailboxName2/dovecot.index*


IMAP search should work at this point. Unfortunately, if you’re using push (and thus z-push, and thus ActiveSync) it’s a little more complicated.

As best as I can tell, the official z-push doesn’t support this. However, there is a vibrant z-push fork community. fmbiete’s fork seems to be way ahead of upstream. Looks like somebody could use a new maintainer.

Anyway, I simply copied fmbiete’s fork to /usr/share/d-push, making sure to back up and restore the config.php file. z-push/d-push seems to want a little more of a kick when you move between versions; this is the recipe that worked for me:

rm /decrypted-mail/dpush-state/*


Followed by deleting and re-adding the account on my iPhone. You get a few “can’t connect to the server” errors, but that’s just because you’ve thoroughly confused the sync on the iPhone side by nuking everything.

## Debugging magic

There’s a neat trick to debugging IMAP search I discovered while I was working on this.

openssl s_client -connect localhost:993
2 Select "Inbox"
3 Search text "test"


That will let you test what search and search performance look like on the IMAP side. You can also see what solr is up to:

tail /var/log/tomcat6/catalina.2013-06-13.log
INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&q=body:"Test"&fq=%2Bbox:123a6f3ac671b951ca310000e529c561+%2Buser:"drew@sealedabstract.com"&rows=179} hits=2 status=0 QTime=3
Jun 13, 2013 4:23:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&q=body:"Test"&fq=%2Bbox:e6825420817bb951a9380000e529c561+%2Buser:"drew@sealedabstract.com"&rows=3} hits=0 status=0 QTime=1
Jun 13, 2013 4:23:16 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&q=body:"Test"&fq=%2Bbox:7301d918ab87b95174400000e529c561+%2Buser:"drew@sealedabstract.com"&rows=5205} hits=364 status=0 QTime=4


You can query solr directly:

curl 'http://localhost:8080/solr/select?fl=uid,score&rows=5205&sort=uid+asc&q=body:%22Why%20bec%22&fq=%2Bbox:7301d918ab87b95174400000e529c561+%2Buser:%22drew%40sealedabstract.com%22'


This means I’m searching for “test”.

Also, when in doubt: restart tomcat. In my testing it doesn’t react even to things like filesystem changes (???) without a kick.

# Conclusion

We did it! We’ve got a fast, modern, e-mail stack that (in my view, anyway) is much better than GMail, not only in the security aspect, but even in areas like speed, spam, and compatibility.

There is, however, more we could do:

• webmail – I don’t really need one, but I’ve heard good things about roundcube. Apple used to ship it in 10.7 Server, but stopped suddenly. Take that for whatever it’s worth.
• requiring TLS encryption – E-mail isn’t secure, but many servers route it over SSL anyway if both endpoints support it. Requiring TLS makes you non-RFC-compliant, so you might lose mail, but it also keeps the NSA from scooping up your mail in some datacenter.
• Greylisting. This is a cool idea involving providing a “temporary failure” each time a new sender sends a message. Legitimate senders will keep retrying, and after a few minutes you add them to a “good” list. Spammers will give up, and their mail messages will never even hit disk. It’s actually really easy configure, just apt-get install postgrey and then add check_policy_service inet:127.0.0.1:10023 under smtpd_client_restrictions in /etc/postfix/main.cf.

So, go forth and run your own mail. Now, your right to complain about government spying has been rightly restored. Enjoy your HN flamewars.

Want me to build your app / consult for your company / speak at your event? Good news! I'm an iOS developer for hire.

Like this post? Contribute to the coffee fund so I can write more like it.

1. Tue 25th Jun 2013 at 7:32 am

It’s a shame that the NSA will still read all your emails in conversation with GMail users… maybe a tutorial on mail crypto is next?

2. Tue 25th Jun 2013 at 5:01 pm

Awesome information, but how do I deal with the several hundre (if not thousands) of archived mails already in gmail? Just forward them to the new address?

3. Tue 25th Jun 2013 at 5:01 pm

@chrs martin, drag and drop between mailboxes from your IMAP client.

4. Wed 26th Jun 2013 at 12:45 am

Assume that this is completely rediculious because

A) I don’t have time to manage Yet Another Computer. Eventually it’ll need updates and/or something will break. Even Debian.

B) More importantly, everyone I communicate with either uses Gmail or another US-based provider, and they’re certainly not going to do this.

I went down the encryption rogue, except that’s also a pain to setup (though much better than this), and the UX is _terrible_, and nobody uses it.

I’m reasonably sure we’re screwed here. The NSA doesn’t care about any of my mail. Running mail servers and using encryption is well over the heads of 99% of the people who really need to be using it.

5. Thu 27th Jun 2013 at 1:14 am

Nice article, informative, but I concur that it is mostly trivial and not useful. And frankly, I see it as unnecessary. Do I really care if the NSA reads my email? I doubt even *they* care. They are trying to catch really really bad guys. I’m not one of those, and anyone who might poke through my email will quickly realize that. They are not going to waste any time on me, and I’m somewhat relieved to know they *are* looking into everyone to make sure events like 9/11 don’t happen again, god willing they are able to prevent it (and it is claimed that 7 major plots have been disarmed thanks to the NSA spying).

6. Wed 10th Jul 2013 at 5:02 pm

Would this setup run on a Raspberry Pi?

7. Thu 11th Jul 2013 at 12:29 am

83 code and config snippets doesn’t seem that easy to me.

I don’t see a way to protect yourself from the NSA. If you are using a US based CA, the NSA could FISC it. If you are using a local one, replace NSA with your home grown spy agency as necessary. And then it’s man-in-the-middle wiretaps on tier 1 carriers and a big data centre in the desert and bob’s your uncle. A well encrypted mail store seems like the least of their hurdles. Sounds a bit conspiracy theorist, I know, yet so would PRISM claims pre-Snowden.

I wonder what Wikileaks and similar organisations use.

8. Thu 11th Jul 2013 at 1:24 am

Yes you need privacy.

For now the US gov is “mostly” benign, but that is never guaranteed in the future. Also police, justice and so on make mistakes every day. They will use anything and everything to convict you if they think you deserve it for some reason even though you may be innocent. Don’t give them the rope to hang you.

Also, say you are interested in a gov job but you have some health issue, say. What is stopping the nice gov employer to never offer you the job because they know your condition but do not want to deal with it? Essentially anything should be private first.

Cheers.

9. Thu 11th Jul 2013 at 6:24 am

How about gmail plugin that encrypts/decrypts mail?

10. Thu 11th Jul 2013 at 3:32 pm

Completely misunderstand whats going on. James has it right. but if you want to waste your time maintaining a pointless mail server go for it.

11. Mon 15th Jul 2013 at 8:19 am

What if this was put into a set of Chef recipes? (or Puppet/Ansible/etc..)

We could host the recipes on Github and then maintain it together.

Steps for configuring:
1) Install config tool & GIT
2) Checkout recipes
3) Edit attributes file for settings specific to you.
4) Run recipes.
5) Do any leftover manual steps (DNS, etc…)

12. Mon 15th Jul 2013 at 9:25 am

Ok, so if the NSA really wanted to read your email, you can be darn sure they’ll figure out a way to do so with this setup. You host this with Linode, which is a US business, so if US law enforcement tells them to root your server, they will. They’ll get physical access if they have to. And as long as you keep accessing the encrypted disk you’ve got set up here, there will be a way for somebody who’s rooted your server to get to it without even breaking the encryption or whatever.

So, yes, you’ve made it more tedious for the NSA to access your mail. But it’s by no means “NSA-proof”. If they want it, they’ll get it. And then James is right, too, of course that everybody you communicate with will have their mail read also, so it’s pointless from the start.

But then again you knew that already. Good job on the headline, though. 😉

13. Mon 15th Jul 2013 at 9:30 am

I hate to burst your bubble, but the NSA has taps directly on the main pipe @ ISPs. They have a two pronged approach… deals with major internet service companies like Microsoft, Google, Facebook, etc. And black boxes that sit in ISPs and make copies of all the 1s and 0s. Just because you host your own email means NOTHING.

14. Mon 15th Jul 2013 at 9:33 am

Totally agree with the above: It’s a good exercise but, in the long run, too much work to maintain yet another server (or should i say stack of servers).

In any case, kudos to the poster, for the thorough tutorial 😀

Cheers,
Pedro

15. Mon 15th Jul 2013 at 9:56 am

This would be great if Comcast hadn’t started blocking INBOUND port 25 several months ago.

Now why would they do that, I wonder?

Now what? My choices seem to be “upgrade to a business class connection”, switch ISPs (there are SOOOOoo many choices), or find/create a relay sever for my domain, elsewhere.
Any other ideas?

16. Mon 15th Jul 2013 at 10:25 am

@Jason, so for you the “really really bad guys” are the Eruopeans ? If not, can you explain to me why they are spying the European councils and every “friendly” country?

17. Mon 15th Jul 2013 at 10:26 am

One point, the MySQL database is not in the encrypted partition.

18. Mon 15th Jul 2013 at 10:35 am

This workaround….it works on Android OS 2.3.6? I’d hate to start taking these steps and his a solid wall. Don’t be so snooty to open. Less than 10% are shown these “basic steps” and it all looks like greek to me.

19. Mon 15th Jul 2013 at 10:42 am

Everyone here seems to be missing the point. First, letting your impetus be swayed based on comments by anonymous “people” on the internet, is bad policy.

The naysayers in this instance, are all wrong. either way. The point is getting more people onto private e-mail. The more people that are taking security measures, the safer we all will be. And before anyone says anything, learned helplessness isn’t a factor. I for one think this is an amazing, and well-researched article. I for one will be setting up a private mail server.

20. Mon 15th Jul 2013 at 10:48 am

Nice artivle. A problem is that I heard is if your server is not trusted for big providers like google they will start to block/mark spam your emails. How much truthin in this piece of news?

21. Mon 15th Jul 2013 at 10:49 am

All this effort and you’re running the mail server on a cloud VPS: hardware you don’t control, so all this security theatre is in some ways in vain. the NDA just scoops everything from RAM anyway if it wants/needs to.

This setup does protect against easy data harvesting by eavesdropping, but how difficult is it for the NSA to just scoop stuff up from those machines?

If you run it on your own hardware, THEN they at least have to physically enter the datacenter and obtain acces, or waiste a zero-day exploit on your box.

22. Mon 15th Jul 2013 at 10:52 am

The NSA is being an idiot in the sense that eople who are serious about terrorism will actually use something else -.- Like a satellite phone (?) or their own encryted email server.

23. Mon 15th Jul 2013 at 11:07 am

The NSA has insane amounts of 0-day for everything electronic out there. If they want it they will take it; even off this 1337 debian box.

24. Mon 15th Jul 2013 at 11:17 am

Unfortunately, we need Java.
Was that really necessary? You use an awesome software based on a great and proven technology, yet you call it ‘unfortunate’ to do so?

25. Mon 15th Jul 2013 at 11:23 am

Worth noting that Microsoft ActiveSync is not really push, it’s an HTTP long-polling request (usually 30 minutes depending on mobile OS) so suffers from many of the same problems as IMAP IDLE. From experience several years ago on BlackBerry (I worked on a native ActiveSync client) setting fetch to every 5 minutes was more battery performant.

If you really need your email quicker than that, I feel bad for you son.

26. Mon 15th Jul 2013 at 12:23 pm

Jaxon Bridge said: “Do I really care if the NSA reads my email? I doubt even they care. They are trying to catch really really bad guys.”

Learn your history! The CIA engaged in widespread domestic wiretapping in the 50s and 60s as well, and their excesses prompted privacy policies which have since been eroded. Let me put it another way: Edward Snowden was a mid-level admin, employed by a private contractor, who had access to other people’s records. There are hundreds of others, many without his ethics, in similar positions. Do you trust every single one of those people with Americans’ personal correspondence?

27. Mon 15th Jul 2013 at 1:04 pm

I can setup this and host my email on some server I manage.

I can even host this somewhere outside the US (Netherlands).

Like James mentioned though, I would really only be able to email a very small number of people I know. I can’t force them to leave their Gmail, Yahoo (especially people like my mom, etc.)

Love the article, but the real issue is that our government should not be doing this in the first place.

28. Mon 15th Jul 2013 at 1:35 pm

The NSA is already siphoning data right off the wire, so if your email is unencrypted in transit then they will have it anyway. There are other reasons to not use GMail, or to run your own mailserver, but “NSA-proofing” is not one of them. If you want to have some expectation of privacy, then you should encrypt all your emails. Unfortunately email is a communication medium, and if the person you are talking with doesn’t use encryption they won’t be able to decrypt your messages, so you will be forced into not using encryption anyway.

Of course even if you use encryption the NSA is still going to store a copy of your encrypted email, and hang on to it until they can decrypt it (perhaps one day your key is compromised, or maybe even they will break the encryption with a rise in powerful quantum computers.)

29. Mon 15th Jul 2013 at 2:42 pm

While I agree with the thoughts on “this does nothing to really protect you..”. I really enjoyed you going through the steps to set this all up. So kudo’s and thanks.

but without end to end encryption they(PRISM) are still reading it..

30. Mon 15th Jul 2013 at 2:57 pm

encfs /encrypted-mail /decrypted-mail -o –public

Is wrong…

It says –public is an invalid option for -o. So, I am wondering if it’s supposed to be
encfs –public /encrypted-mail /decrypted-mail -o ,

but then I don’t know what you are trying to do with -o.

Debian 7 x64 fresh deploy on linode.

31. Mon 15th Jul 2013 at 3:07 pm

Given the negligible number of email users who use email securely (I’ve yet to see any e-commerce site offer to GPG their emails to you, and one would think it’d at least make commercial sense to do so), and the consequence that anything that is GPG-encoded is going to stand out from the normal flow of email traffic like a – well, a red flag, I suggest that email be abandoned for anything but the most anodyne of communication, and that anything one wouldn’t want to have exactly the visibility of a holiday postcard should take place by direct communication via SSH/SFTP. After all, if you’re thinking of making an email server, you already have all you need (SSH, public-facing connection) – and then it’s just a matter of exchanging public keys via email (or twitter, etc).

And if that isn’t secure, we’re all in trouble…

\begin{rant}
As for Jaxon and the rest of the “only bad people need privacy” brigade, I assume they’ll be handing out the IP address of the webcam in their bathroom any day now. Also, only 7 plots? Leaving aside the obvious begged question, that’s a shockingly poor rate of return; it indicates that most serious plotters already don’t use NSA-traceable means, and raises the question of the competence of those who do – if the NSA have only forestalled plots (whether 7, 70 times 7 or 7^7) whose plotters were so amateur that they could never have succeeded anyway, where’s that justification again? And how many plots are being missed altogether because the NSA is investing all its effort spying on everyone, rather than actually using some, er, intelligence? sigh The useful idiots’ arguments don’t even make sense from their own “to hell with civil liberties, we’re in a war here / we’re the good guys / trust your great and bountiful government, not that scary young family who lives next door that you talk to every day!” standpoint, let alone anyone sane’s. (But then, if we lived in a sane society, the suggestion that the people most familiar to us are somehow less trustworthy than some faceless bureaucrat, promising us that he was working in our best interests but refusing to let us in on how, would be met with the absolute contempt and derision it deserves.)
\end{rant}

32. Mon 15th Jul 2013 at 4:29 pm

Jaxon Bridge,
If you truly believe what you said then you’re part of the problem. It’s not just the fact the NSA is spying, its against the 4th amendment. It’s illegal for them to do this, yet they are doing it anyways. We have the RIGHT to privacy. Plus you need to stop watching FOX news, CNN etc… It’s propaganda. They have NOT stopped 7 major plots, they just said this so people like you that don’t question the government will just shut up and listen and do what they say. 9/11 was an inside job anways… they let it happen.. there is too much evidence to proove that it was a huge setup. Look up false flags, US goverment has been doing them for years to do what they want. I.E Vietnam, IRAQ, Afganistan etc.. It’s all BS, wake the fuck up and stop believing the government.

Thanks to Drew for putting out this info so others can help protect their privacy.

33. Mon 15th Jul 2013 at 4:59 pm

To those thinking that the NSA doesn’t care about them: Yes, it does, or at least the risk that they one day could is very real. They probably don’t target you specifically at the moment. But (a) that does not prevent them from storing your communication in case they do see a use for it lateron, and (b) you don’t need to be “special” to be of interest, it’s perfectly sufficient if a machine learning algorithm categorizes you into the same class as 20 million other people that have been found to be receptive to certain PR strategies in order to be able to manipulate you without you noticing what is going on. Also, everyone’s transparency because they “don’t have anything of interest” is what makes those who do have something to hide for a good reason stick out. If the freedom fighter is the only one who protects their privacy, they won’t have much time to fight for your freedom.

34. Mon 15th Jul 2013 at 6:39 pm

This is a great tutorial on setting up a pretty full featured mail server. Controlling your systems and data like this is important, regardless of eavesdropping.

The main drawbacks:
* it’s not set and forget – if it’s not continually patched and monitored it will be vulnerable to breaches, and not just by the NSA. The surface area of the system with the number of components in it is quite large. It only requires one of them to be compromised.
* unless you use proper SSL certs, or install the cert on all your devices, you’re still very vulnerable. Self-signed certs don’t make for a secure system.
* this system that you need to rely on is a single point of failure.

Without hardening, proper SSL or a backup strategy, setting up your own server would be a somewhat quixotic enterprise.

35. Mon 15th Jul 2013 at 6:44 pm

+1 to calling out how stupid it is to trust the NSA.

Unless you’re a well-connected political figure, the NSA won’t do jack to protect you. If they’re collecting data, it’s for their own agendas and not for our safety.

If terrorists send Mr. Nobody a bomb, the NSA will be too busy picking their noses to bother to stop it. But if an honest journalist criticizes their political buddy for corruption, then it’s a national emergency to sift through the journalists’ emails.

36. Mon 15th Jul 2013 at 6:49 pm

Everybody saying “email is sent in the clear” would be very surprised to see how much e-mail isn’t sent over the clear in 2013. Opportunistic encryption means that every mail server that supports TLS will use it to talk to you. That is a LOT of mail servers. Just for starters, every message that runs through GMail / Google Apps will be encrypted end-to-end, and that is objectively full third of my entire mail volume, just with that one provider.

It would be nice to get everybody to install PGP and do end-to-end encryption. But out here in the real world, you can get 95% of the benefit without making anybody do anything. That’s a big win in my book.

37. Mon 15th Jul 2013 at 7:47 pm

for those bothering to respond to the troll comments plz keep in mind:
http://www.guardian.co.uk/technology/2011/mar/17/us-spy-operation-social-networks

and

http://www.wired.com/dangerroom/2009/02/27000-work-in-p/

38. Mon 15th Jul 2013 at 7:49 pm

Thanks for writing this article. It was informative and appreciated.

39. Tue 16th Jul 2013 at 3:36 pm

would worry more of the PLA agents working in google server room, far more than the NSA who still has to answer to somebody …

40. Tue 16th Jul 2013 at 5:05 pm

“Just for starters, every message that runs through GMail / Google Apps will be encrypted end-to-end”…only if you consider Google’s server and the equivalent on the other side to be an “end”. It is by no means encrypted all the way from sender to you. It may well be encrypted (via https) from sender to sender’s provider, decrypted there and stored, re-encrypted and forwarded to your provider, decrypted and stored there, and then re-encrypted for the web browser connection while you attach, but this requires two completely trusted middle men who see it in cleartext, which is not “encrypted end-to-end” at all.

41. Tue 16th Jul 2013 at 5:56 pm

@Drew: Sure, but what good is end-to-end encryption if there’s a backdoor that can be used to read the mail anyway? Maybe they don’t read mail from your account directly, but the entire point of massive data aggregation is so they can put the pieces together. Therefore, any message sent from or to someone with an insecure server, or over unsecured connections, or whatever, is going to be sucked up just the same, and connected to you just the same.

So all this effort basically means that only your messages to others who have followed similar steps are at all protected. While better than nothing, that’s not very impressive.

42. Tue 16th Jul 2013 at 9:19 pm

can i use other distro of linux?

43. Wed 17th Jul 2013 at 9:22 am

Thanks, Drew, for the details instructions!

I’m curious why a MySQL database is required for virtual user authentication for postfix and dovecot. Wouldn’t it be simpler to store this in a file on disk rather than require a full SQL database server be running?

Most of the instructions I find online use a MySQL server, and I’ve never understood the benefit. Unfortunately, I don’t understand the process well enough to implement a non-SQL version.

44. Wed 17th Jul 2013 at 11:13 am

Hi

i should repeat what DerekS has said: fuse returns an error for “–public” option. this is what it outputs:

root@host:~# encfs /encrypted-mail/ /decrypted-mail/ -o –public
fuse: unknown option `–public’
fuse failed. Common problems:
– fuse kernel module not installed (modprobe fuse)
– invalid options — see usage message

and this is what lsmod returns:

root@host:~# lsmod | grep fuse
fuse 52184 1

45. Thu 18th Jul 2013 at 11:04 am

Won’t I need a home internet connection with a static IP address to have an SMTP server running?