Secure File Syncing

This is a follow up to my previous post Single Point of Failure, where I detailed my switch to self-hosted/distributed alternatives to various Google services so I wasn't completely reliant on one provider. At the time I didn't cover file syncing, since Dropbox was doing a fine job for me. However, for a couple of reasons, one being that the Dropbox Linux client is significantly slowing down my system, I've decided to move to a different tool. Conveniently, with the recent NSA Prism revelations, secure distributed alternatives to tools like Dropbox have been enjoying a lot of exposure, so it's been easy to find good alternatives.

The two tools that I'm now using are:

1. Git Annex + Git Annex Assistant
2. BitTorrent Sync

Both are really promising, but for different reasons.

Git Annex

Git Annex offers the ability to distribute files across many locations and uses Git to keep track of the location(s) any given file is available from. It also gives you the ability to recall a file from a remote location to your current device. This is really powerful for managing a large amount of data when you don't have room for it all on one machine.

The assistant functionality automates the syncing between repositories. It has a nice webapp which provides a GUI for managing remotes and sync strategies. These remotes can be additional git repositories, encrypted rsync destinations, local drives, EC2, and more. The assistant is still under active development, so it's not 100% stable or feature-complete, but it's definitely usable and the underlying functionality is pretty solid.

At the time of writing (July 20th) there's a crowd-funding campaign going on to support another year of full time development on git annex. If you want to support a good open source project, head ond over and contribute a few bucks (or bitcoins.)

BitTorrent Sync

BitTorrent Sync is the second tool I'm using for file syncing. It's much less complex then Git Annex. It's all about keeping folders in sync across multiple systems. The setup is pretty simple:

1. Add a folder to the client and generate a unique secret key.
2. From any other computer, choose a folder and give it the secret key.
3. Let the computers auto-discover each other, or specify an address.

BitTorrent Sync has clients for all the major desktop operating systems and an Android app.

What to use?

For the time being, I'm going to use both. I'm trying out Git Annex as a central store for anything I care about. Using it, I can setup a backup with the encrypted rsync remote as well as directly sync when I need to my different computers on an as-needed basis. This allows me to track all my old projects and miscellany in a single place.

BitTorrent Sync is going to handle more transient syncing needs. This includes things like my workspace and desktop wallpaper collection since they're ever changing and could be rebuilt from scratch if really necessary.

Web Access

In addition to the two syncing services listed above, I also have ajaxplorer setup on a remote VM and pointed at a BitTorrent Sync folder setup on the VM. This allows me to make files available when I'm on the road, at work or don't have access to a trusted machine with the client installed.

Further Reading

• BitTorrent Sync + Raspberry Pi
• Free/Open Source alternatives to popular syncing software

Single Point of Failure

Why I no longer use Google for everything

Periodically a story makes the top of Hacker News about someone who's just had their Google account disabled. Every time I read those stories, I think through what would happen if my account got disabled, and as you might imagine, it wouldn't go well for me, since I don't have any good friends who are Google employees who could bail me out.

I don't have any problem with what Google's doing, and I still intend to use some of their services, but it has become a massive single point of failure for me, and I don't have any recourse if something happens, since Google has no support.

I've decided to fix that by switching to separate providers for the services I depend on. Here's the breakdown of what I'm switching and what I'm keeping.

Switching

Email

This is the most important service I'm switching, since it's what I use and depend on the most. I've tried a number of alternatives, and I've decided to go with Fastmail, since they support custom domains, IMAP, SMTP, and have a nice web interface. It's also not expensive for 10GB of space at $40/year (or $32/year if you prepay 5 years). Fortunately, I've been using my own domain with Google Apps, so this switch will be completely seamless for everyone I communicate with.

Email Clients

As a side effect of dropping Gmail, I also needed to switch mail clients on my Android and iOS devices. The key feature I need in a mail client is identites, since I send mail from my custom domain, not the Fastmail account username. On iOS, this is simple, since the native mail client supports sending from a different email address.

Android however, does not support this in the native Mail application, so I had to find a good alternative. The one I ended up with is Kaiten Mail, which is excellent and well worth the $5. It has support for Identities, archiving mail and a plethora of customization options. My only gripe with it is the icon, which looks a lot like a trash can.

Contacts and Calendar

While Fastmail does provide contacts, they don't offer CardDAV syncing, which makes them unavailable on Android and iOS. My solution is to use a separate service, a self-hosted instance of Kolab for both Contacts and Calendar. This works with iOS and Android, since it has support for ActiveSync. It also has a very solid web interface for managing both.

I've also been experimenting with ownCloud's Calendar and Contacts, but I'm unimpressed with their web interface and I probably won't keep it. The main reason I'm trying it is the CardDAV and CalDAV support, since those protocols work well across all platforms, desktop and mobile. Kolab has announnced support for both in the next version, so I think it's still the better choice.

CalDAV/CardDAV Support (Mobile)

This is mostly for my own reference once Kolab has support for CardDAV and CalDAV - iOS has native support, Android does not, but there are good sync adapters available from Google Play for both - CardDAV-Sync ($2) and CalDAV-Sync ($2) which both support 2-way syncing.

Reader

I've been largely off of Reader for a while, since it's been obvious Google didn't care about it. The only place I still use it is on iOS, since it's the only service my favorite RSS reader (Byline) syncs with. Other than iOS, I've moved all my feeds over to both NewsBlur and Feedly, and have been alternating between the two. I'm waiting for the dust to settle after Reader shuts down to decide which I'll stick with.

Music

I'm setting up ownCloud to replace this as well. I'm using ownCloud's file sync to move Music up to a server and then for playback I'll use a separate tool (ownCloud has a built in music player, but it doesn't have many features for filtering albums or artists.) My intention is to setup, Ampache, Subsonic, Jinzora or something else on the server to use for playback and streaming.

This is an area where I might end up back with Google, since nothing I've tried has been very good and I also have everything in Amazon's CloudPlayer, so it's not really a single point of failure for me.

Keeping

Voice

I haven't found a good alternative to Google Voice; SendHub is close, but it's missing a number of key features like SMS to email and at $25/month for 500 minutes it's a little expensive on top of a cell phone bill. My current plan is to start experimenting with the Twilio API to see if I can duplicate the most important features of Google Voice myself.

Docs / Drive

I don't use Docs enough to merit finding an alternative and I already use Dropbox for syncing.

Search

Google still has the best search engine. It's also not strongly tied to my account, so I don't have to worry about losing data.

Summary

I have a few final thoughts on this process:

1. Google's great on the wallet

   Google's offerings are free, which is really hard to beat if you're trying to be economical. However, that comes at the cost of no support. Now that I'm out of college and have an income, but less time, I'm willing to pay for support and the assurance a service isn't going away.

2. Google's/Microsoft's/Yahoo's/etc... free offerings suffocate markets

   Looking around for good alternatives to products Google or Microsoft offer for free is really hard. There's no really good open solutions for online calendaring, or, until the recently announed death of Reader, RSS readers. Nobody wants to compete with Google.

3. Rent a dedicated server

   Nothing gives you more flexibility than having a big beefy dedicated server to run VMs on. When I want to test a new tool, all I have to do is clone a template VM and install it. If I like it, I keep the VM, if not, I delete or archive it and move on.

That's my experience switching from Google. If you have suggestions for good alternatives I'd love to hear about them. Feel free to post a comment or tweet @justjohn.

A Workspace That Works with Everything

One of the problems I've always had with the setup of my computer is
that each operating system has one thing I want that it does best, and I
would have to either compromise in one or reboot to the other. On Linux,
it's the development environment - I've always been most productive in an
environment with a usable terminal and minimal UI. On Windows it's the
media software available (and the games); I can't live without Winamp (and
sometimes iTunes) for their device syncing capabilities and excellent
playback capabilities.

With the last round of upgrades to my primary desktop (a quad-core i7
and 12GB of memory) I've finally reached the point where I can have what
I've always wanted: a reliable way to run a complete development
environment and simultaneously use the Media software of my choice on
the same machine.

My system currently runs the following:

• Windows 7 Ultimate as the host OS

• VirtualBox 4 running the following 3 VMs (all Ubuntu 10.10)

• A file server running Greyhole and Samba for redundant storage accessible from everywhere

• A network core VM for locally hosting network services such as DNS and NIS. (oh, and a Minecraft server)

• A development VM with the appropriate toolset for PHP development (Netbeans 7 and Firefox).

The features that make this all work:

1. VirtualBox supports multiple monitors, so my development VM can span
   both my monitors, or only one if I need some windows-only tool on the other.

2. Windows is the host OS, so it has all the access to the hardware it needs
   for device syncing with my Android phone and my iPod Touch. It's also a
   plus when I want to do some gaming; I can close the development VM to
   free up resources and since VirtualBox saves state it takes mere moments
   to bring it back up when I need it (typically under 30 seconds.)

3. Ubuntu has excellent support for running in a VirtualBox container. I haven't
   had any issues with the VirtualBox guest additions and drivers.

In short, if you have a powerful enough machine, you can get the benefits of both
Windows and Linux simultaneously without having to give up anything due to
VirtualBox's great support within Ubuntu.