What is Sandstorm?

It's a way for you to have your own 'Facebook', 'Reddit' and much else. Your very own. No faceless corporation involved mining your personal information to sell to advertisers (that's how gmail and facebook make money, after all).

Sandstorm is completely and privately YOURS and no one elses.

And it is very easy to install on your own server (regular Linux box on your LAN), even easier to have someone else do it and rent the space ( http://sandstorm.io ), and moderately simple to own your own instance that you can share publically. This little tutorial will show you how to have your own sandstorm:
- download and install Sandstorm on your own box
- configure it so that it is accessible by anyone on the internet
- and it will cost you exactly nothing per month to do it

Mind you, after I have tested Sandstorm on my local box behind a router, I WILL be moving it to a 'real' server (actually, just a vps these days). My power is dodgy and not redundant, I haven't configured it for off-site backups, and it's just a 10-year old box that would otherwise gather dust if I didn't experiment with it. Still, you could arrange for decent rsync or zfs/btrfs backups and get them offsite and buy a battery (UPS). Up to you.

Plus, I think it would be quite nice for a household to have their own private 'cloud' on their own private LAN.

You do need to be a bit geeky to install Sandstorm, but not much. And you need to be running Linux, not Windows.

Update 2015-11-06

As you can see below, you need to map some ports if playing at home behind a router. An interesting effect of using Sandstorm's free SSL via DNS is that you can also use other ports at the same address (but not with SSL). For example, I installed a VM of ERPnext and simply used the same personal Sandstorm address, but specified a different port.

Update 2015-10-17

A very cool new development: the Sandstorm team has figured out a way to give you free SSL! The road was a bit bumpy for doing it behind a NAT router, but now it works! NB it also takes care of updating your dynamic dns. Awesome!

 

You can now have free SSL Sandstorm apps behind a NAT router with dynamic DNS all in one go. Which means you can simply run it on your own box at home. The install script is painless to run. You just pick a name for your address and done! This happens while the script runs. For example, I'm now Totally painless.

The only (possibly) tricky bit for you is to map ports to your box in your router, but they have docs for that too. Their install script takes care of the rest:

https://sandstorm.io/install/

tl;dr is curl https://install.sandstorm.io | bash

... and just follow along. Then map ports 6080, 80 and 443 to the box you installed it on. Voila. You now have multiple, multiple ssl encapabled web sites. For free. And the ssl certificate changes weekly automatically. And the software itself updates automatically. And, well, it's just fun :-)

All the apps really are just click and done to install. It's very simple. Here's a list of what is currently available:

https://apps.sandstorm.io/

install

This is directly from https://github.com/sandstorm-io/sandstorm and it is perfectly adequate.

Please do read the folowing link. It is in very early stages of development. That said, I have found it to be quite stable, but just a bit dodgy on various platforms and browsers (mostly old versions of browsers and cell phones). The latest versions of Linux Mint 17 and Firefox are what I used successfully with every 'app' I tested -- no, I haven't tested them all yet.

Quoting:

====

https://github.com/sandstorm-io/sandstorm#installing-the-easy-way
To install on your own on a Linux machine, just do:

curl https://install.sandstorm.io | bash

Or, if you don't like piping directly to shell, download first:

curl https://install.sandstorm.io > install.sh
bash install.sh

This will install a self-contained and (optionally) auto-updating Sandstorm bundle. It won't touch anything on your system other than your chosen installation directory, optionally installing an init script, and placing two symlinks (spk and sandstorm) under /usr/local/bin.

====

That's all true and it works perfectly using my ancient test box (10 yr old amd pc) with a current version of Mint 17 Linux.

Sandstorm was immediately accessible at http://localhost:xxxx, where 'xxxx' was the port number I assigned during the install.

domain name

Of course, you will want to share Sandstorm with your many friends and relations. You can't easily do that unless you have a domain name e.g. 'myhomebox.ca'.

Problem 1: 'myhomebox.ca' doesn't exist as an internet address and
Problem 2: your ISP keeps changing your internet 'phone number' (IP address) randomly, so that a domain name can't point to it reliably!!

Problem 2: is the biggest deal. You normally need a permenant IP address on the internet in order to serve web pages of any kind.

BUT, we have a plan around it: we buy our own internet domain name. Yes, there are ways to get a free domain name, but you won't get all the goodness that comes from owning your own domain name. Just trust me on that for the nonce. The cost to register your own domain name is like ten to (max) fifteen dollars a year. Just do it. This will be your only cost for this whole game and it has many benefits besides Sandstorm (like being able to make up your own email addresses and other things, which are not addressed in this article).

The thing about registering Domain Names and using Domain Name Registrars like say, GoDaddy, is that people don't always appreciate what they are getting when they pay for a domain name. In essence, you get zilch — just the right to use the name. You don't get hosting, a website, email services or a computer 'in the cloud' — nothing! All you get is the right to use that domain name on the internet. They upsell you all that other stuff later...

Instead, we're going to take that same basic fact of registering a domain name and do it all by ourselves. No up-selling. Nothing. We'll provide ourselves with all that stuff and more.

All registration means is that the name 'myhomebox.ca' will not be used by anyone but you as an internet address — as long as you keep paying the yearly fee. That's all it does. Nothing more, the rest is up to you. And if you ever stop paying, someone else could buy it and use it. It's more like renting a domain name really.

The next thing we need to do is to somehow point that domain name toward our Sandstorm box. For that we'll need to have two other computers on the internet acting as a kind of 'phonebook' at all times (that's just a rule, live with it) and we need to have a permenant IP address or 'internet phone number'.

Now we encounter a couple of problems as normal users. 1) Our ISP keeps changing our IP address randomly (that's our "phone number") and 2) all we have is that one spare box we're installing on at home, never mind coming up with two other boxes (DNS Servers) to simply host our DNS/phone-book record! On top of that, the DNS servers need a permenant IP on the internet, seemingly tripling our problem.

Plus, even a cursory reading of the docs for Sandstorm informs us that we will also need something called 'wildcard DNS'! Arrgh! It's all just too much to bother with.

However, if you are even slightly geeky and have a bit of patience, we can get around all of that, plus a bit more.

get a DNS provider

At this point, I'm going to go ahead and assume that you've registered a domain name. You really need one of those and it's cheap, as I mentioned. Also note that domains that end with your own country are often cheaper than those that end with .com! There is no reason for this, it just is. So, if you live in Canada, names that end in '.ca' are cheaper. Ditto for other countries, like .pt, .es, .de and so on. You don't need a name ending in .com, so don't rent one.

I am NOT affiliated with any company I recommend in this post. I'm just cheap - careful with my money ;-) , and I'm passing on my own findings.

Payless Domains works for me in Canada. I'm just saying. Really, I've never had a problem with a registrar in the over 15 years I've been using them. They do vary in the amount of up-selling they try out on you and how aggressive they are while they're about it (I'm looking at you Godaddy). Also, the quality of their web interfaces vary from too simple to looking like the cockpit of an A380. In practice, once you've defined your Domain Name Servers, you never have to deal with them again, other than to pay. Oddly enough, the payment web interface is Always painless. Weird...

So that's the registrar for the domain name out of the way (and the only thing you might need to pay for), but how about those two pesky DNS servers we need?

Again, these guys pay me nothing (I pay THEM by choice), but this is a very good solution to DNS:
https://www.cloudns.net/

I see that they are now also offering free domain names too. Well, perhaps you are even more parsimonious than I am, but I still recommend controlling your own domain name directly. No dis to Cloudns or anyone else at all, just my opinion.

In any case, the guys at Cloudns are wonderful and you will be able to configure your domain name's records with a simple graphical user interface to your heart's content.

wildcard dns

I've been making websites and stuff for a lot of years, but I never needed one of these before Sandstorm. Heck, I didn't even know they existed. But you need to have one to use Sandstorm and it's just typing the '*' asterisk character you'd expect.

So this is a good place to mention about subdomains.

In a regular domain name, you have two parts: 1) the domain name and 2) the Top Level Domain (TLD) e.g. myhost.com
Where 'myhost' is the domain name and 'com' is the TLD.

Here is a subdomain for it (the 'sandstorm.' part is the sub-domain):
sandstorm.myhost.com

The Good Bit about subdomains is: they are free!! Free, free, free! They cost nothing and you can make as many as you like.

And you can also make sub-sub-sub-domains. Or even sub-sub-sub-sub-sub-domains... You get the idea. Just add words and more dots.

Installing your own Sandstorm requires that you make a wildcard subdomain. Cloudns makes this very easy. Just make a new 'A' record with an asterisk ('*') and that's it.

For this fake case, the 'A' record for the wildcard domain would look like:

*.sandstorm.myhost.com

And your 'sandstorm.conf' would look like:

SERVER_USER=sandstorm
PORT=1234
MONGO_PORT=1235
BIND_IP=0.0.0.0
BASE_URL=http://sandstorm.myhost.com:1234
WILDCARD_HOST=*.sandstorm.myhost.com:1234
## MAIL_URL=PASSWORD@SMTP_HOST:SMTP_PORT" title="smtp://USERNAME:PASSWORD@SMTP_HOST:SMTP_PORT">smtp://USERNAME:PASSWORD@SMTP_HOST:SMTP_PORT
## Free account at mailgun
:This email address is being protected from spambots. You need JavaScript enabled to view it.">MAIL_URL=smtp://This email address is being protected from spambots. You need JavaScript enabled to view it.:This email address is being protected from spambots. You need JavaScript enabled to view it.:25
UPDATE_CHANNEL=dev

cronjob for dynamic ip address

Another little problem you'll have, if you want to self-host Sandstorm at home, is that your 'phone number' changes randomly. it just does. Unless you pay extra money to Comcast, Telus, Bell, whoever, they will simply change your IP address every few days or weeks. Obviously, this does not play well with the phone books (DNS) which are pointing domain names to IP addresses! Not good when your IP address keeps changing randomly...

Fortunately, you can simply create a cronjob that will tell Cloudns (or whoever — this is a common mechanism) about your current IP address. Cloudns has a button in their GUI which will give you a copy/paste for that. It looks like this:

#dyndns at cloudns.net
05 * * * * /usr/bin/wget -q --read-timeout=0.0 --waitretry=5 --tries=400 --background http://ipv4.cloudns.net/api/dynamicURL/?q=Mjk5NDUyOjc0MD...muchmore

That's a cronjob that will fire five minutes after every hour and tell Cloudns what your IP address is currently.

Just for your interest, the syntax for those cronjob lines is:
#min(0-59) hours(0-23) day(1-31) month(1-12) dow(0-7) command
So, the first number is minutes, the second is the hour of the day... asterisks are wildcards that mean 'any'.
http://linux.die.net/man/1/crontab

And, no, you do NOT need to use vi to edit cronjobs in linux. That is so irritating to me, seeing that message over and over. Just set your $EDITOR env var to MicrosoftOffice (joke) or whatever and it still works just fine. I set mine to mcedit, but vi, emacs, nano, whatever are just fine too. The files of interest to set your default editor are .bashrc and .profile. Just add a line like:

EDITOR=mcedit

The command for listing your cronjobs is:

> crontab -l

Where '-l' means 'list'.

To edit, just type:

> crontab -e

'e' for 'edit'

port mapping on home router

So. Now we have a domain name pointing at our house, but to which computer on our LAN?

Remember, we only have one external IP address and the buggers keep changing it.

On top of that, we have more than one computer behind our router and only one of them has our Sandstorm on it. Problem...

The good news is that each network card/device has its own, unique other address, a MAC address (you can look it up or just believe me for this article) and the router can Assign incoming messages for a certain port to an internal port based on the local machine's unique MAC address.

What's a 'port number'/doorway? Well, and I admit that this is a fairly crappy analogy, but a port number is kind of like an extension number for a telephone system. Every computer that is capable of being on the internet has over 65,000 port numbers! The most common port number is 80, which is the one for web sites. It's so common that you never even have to mention it — it's just assumed. However, you can pick any of your favourite websites and type ':80' after its address and it will always work perfectly, because that port number was assumed in the first place!

http://google.com:80

Try anything you know, it will always work.

Anyhow, we're going to use up one of those other 60,000 plus numbers available to us. So our addresses will look like this:
http://mysandstorm.myhost.com:1234

It doesn't much matter what number you choose for a port. Just stay away from those below one thousand, because many of those are pre-defined (like port 80 for web-servers). Me, I made a subdomain using my street name and the port using my house address, because it would make the URL easy to remember:
http://mystreetname.myhost.ca:4321

Obviously, you can make up anything you want for a subdomain and a port number. Just make sure they are the same everywhere, i.e. in sandstorm.conf, DNS and your router.

After you've done it once or twice, you'll be able to see that with just a little thought you could easily map the incoming port to a different port on your home-based Sandstorm box using your router. Really, you could just use port 80 all the way through, however I'm already using port 80 for a common webserver.

Also, while I'm never going to recommend 'security through obscurity' I don't think it hurts to add it on top of actual due dilligence — it always keeps your logs a lot tidier, if you don't use the standard service port for e.g. ssh. At least, it keeps the lamest script-kiddies at bay.

email

OK, here is the straight help from the Sandstorm guys themselves:

https://github.com/sandstorm-io/sandstorm/wiki/Configuring-your-server-for-email
Outgoing SMTP
All you need to do is set MAIL_URL in sandstorm.conf (by default at /opt/sandstorm/sandstorm.conf) to a working SMTP server that will accept e-mails with the SMTP envelope's bounce address set to your grain's local address. If running at home, you can usually use your ISP's SMTP server. Otherwise, Sendgrid and Mailgun also provide such services. Set the MAIL_URL like so:
MAIL_URL=PASSWORD@SMTP_HOST:SMTP_PORT" title="smtp://USERNAME:PASSWORD@SMTP_HOST:SMTP_PORT">smtp://USERNAME:PASSWORD@SMTP_HOST:SMTP_PORT

Usually, you will be good to go at this point. There are a couple of things to consider though:
1) ISP's normally only hand out one email name per account. Sometimes more, but...
2) Common receivers of emails won't accept more than 50 or 100 emails per day from the same IP address, which is a problem if you have a mailing list of, say, 300 people or more, as it will take days for a newsletter to go out.

getting a free email server

To get around this problem, several companies have sprang up, which exist only to provide a zillion IP addresses for your email to come from. duh. Had to happen.

I have tried a couple of them commercially, that is to say that I or my clients paid them. Among them are Mailchimp and Elasticmail. I found Mailchimp to be quite aggressive in their upselling and bountiful sales mails to me. Elasticmail was fine in this regard and also had a nice gui and tracking. Sandstorm offered three more I'd never heard of: Sendgrid, Mailgun, and Mandrill. Of these, I picked Mailgun, for the very good reason that it offers a free account that is good for 10,000 emails a month, which is probably over-kill for your self-hosting needs. And they are totally cool with that, I know, because I wrote them. They also possess about the best UX I've ever encountered in my life, so I don't feel the need to give any guidance about how to sign up or configure your DNS:

https://mailgun.com

All in all, just the exercise of testing Sandstorm will introduce you to several great teams: sandstorm, cloudns and mailgun.
paul


p.s. please be a little gentle with your comments. I learned years ago that I am not a 'writer'. I know that. I just want to promote sandstorm, because it is an exciting thing to me. You can whinge directly to 'pauls' and the website domain name, if you really feel the need ;-)