Stavros' Stuff

Angry rants of programming and other things.

I found the Holy Grail of backups

Finally, encrypted, fast offsite backups

If you’ve been reading my ramblings for a while, you will remember that I have been looking for a good way to back up my files, without much luck. My requirements are:

  • Encrypted (on the original machine).
  • Fast/bandwidth-efficient (deduplication is a plus).
  • Open-source.
  • Cheap.

My files are mostly code and photos, which aren’t really critical (the critical files are stored in repositories and multiple computers), but losing all my photos wouldn’t be too great. That’s why any backup solution I wanted had to be cheap enough to justify the cost.

Every single piece of software I have ever tried fails in at least one of the above. I currently use SpiderOak, which is everything except open-source, so that fails too.

However, my quest is finally over. I have discovered the Holy Grail of backups, and its name is

Continue reading…

Arduino-powered irrigation system

Control your field’s irrigation system with your mobile

A few weeks ago, I was showing my dad my latest project, the Arduino RF remote control, and he lamented the fact that I still hadn’t made an irrigation system remote control for him. We have a few acres of fields that need irrigation, which is done with a system of pipes from a central water pump. At that point, however, the fact that he needed a remote control for the pump was as news to me as it is to you, so I asked him what he needed exactly.

He told me he needed a system that would allow him to call or text a phone number and turn the pump on and off remotely, as well as something that would text him if there was a problem that

Continue reading…

Mozilla Persona accounts with disposable email addresses

Mozilla Persona, now with extra disposable email address

If you follow the news, you’ve hopefully heard of Mozilla Persona, the wonderful new way of centralizing logins on the web. In a nutshell, Mozilla Persona works the same way as the “log in with Facebook” you see on many sites, except that, instead of Facebook, you use your email account to log in. The great thing is that sites never have your password, and you only need to have one password for every site on the web (Persona uses your email address as your username).

One downside of the default implementation is that you have one account per email on the default bridge (this isn’t strictly true, but adding more accounts is cumbersome and not meant to be done for every site). This is a problem for me, as I use the excellent 33mail service to create disposable email addresses for every site I visit.

Fortunately, there is a way to use disposable email addresses as usernames on Persona. Here’s how:

Continue reading…

Remotely controlling IR/RF devices with an Arduino

Control your household devices: Arduino edition

If you’ve been following this site, you’ll know that I’ve been going through lots of alternatives in trying to achieve the Holy Grail of home automation, which is to get my garage door to open remotely with a Raspberry Pi. To this day, I haven’t managed to do that, but I did get some great guides going.

For example, I managed to turn the Raspberry Pi into an infrared remote and control my TV and AC unit with it, and, after that, the next step was to control RF wall sockets, which was also great.

After all this, I connected my house lights to RF sockets so I can control them from the Pi, but having a Pi running all the time is a bit of a hassle. It requires an SD card, a USB hub for the wifi, it freezes sometimes, etc, so I wanted a solution that was less of a complete PC. The obvious choice was

Continue reading…

Developing with pull requests

Pull requests are fantastic

As you may know, my favorite way of beginning a blog post is with “as you may know, my favorite way of beginning a blog post is with ‘as you may know’”. As you may also know, for the past year or so I’ve been working with Silent Circle, where I’m part the team that develops the website and backend.

The team is composed of roughly seven people now (developers and designers), and, as it has grown, we have been trying things to find a good workflow that allows us to be productive while staying on top of the changes made to the code by coworkers.

This post is about our needs and the development/release workflow we have arrived at to address them.

Continue reading…

Automatically check your Python code for errors before committing

PEP8 or GTFO (but friendlier)

A few days ago, I wrote a post on how you can reject commits that violate PEP8 using git’s pre-receive hooks. This is evaluated on the server, which, in the end, led to many commits that were just fixing PEP8/syntax errors. Wouldn’t it be much better if you checked for errors before you pushed, so you could fix them right there? It would indeed.

To do this, we can use flake8 and git’s pre-commit hooks (I’m assuming you already have flake8 installed, from the previous post). A pre-commit…

Continue reading…

Writing a full-text search engine using Bloom filters

Search engines and Bloom filters: A match made in heaven?

A few minutes ago I came across a Hacker News post that detailed a method of adding search to your static site. As you probably know, adding search to a static site is a bit tricky, because you can’t just send the query to a server and have the server process it and return the results. If you want full-text search, you have to implement something like an inverted index.

How an inverted index works

An inverted index is a data structure that basically maps every word in every document to the ID of the document it can be found in. For example, such an index might look like {"python": [1, 3, 6], "raspberry": [3, 7, 19]}. To find the documents that mention both “python” and “raspberry”, you look those terms up in the index and find the common document IDs (in our example, that is only document with ID 3).

However, when you have very long documents with varied words, this can grow a lot. It’s a hefty data structure, and, when you want to implement a client-side search engine, every byte you transmit counts.

Client-side search engine caveats

The problem with client-side search engines is that you (obviously) have to do all the searching on the client, so you have to transmit all available information there. What static site generators do is generate every required file when generating your site, then making those available for the client to download. Usually, search-engine plugins limit themselves to tags and titles, to cut down on the amount of information that needs to be transmitted. How do we reduce the size? Easy, use a

Continue reading…

Reject unPEP8 pushes with git hooks

PEP8 or GTFO

Like many people in our profession, I am a bit of a stickler when it comes to style and consistency. I like my code (and my team’s code) to have a specific style, and to stick to it, which is why I love tools like go fmt.

The Python equivalent is, of course, PEP 8. Although I don’t agree with everything (I like spaces but think tabs make more sense in theory, because they separate presentation and formatting), I observe PEP 8 because I think it is important to have one consistent style, no matter what it is.

Sometimes, however, code which doesn’t follow that style slips past my editor. In those cases, I would like my git server to reject my commits and tell me what’s wrong so I can fix it.

Continue reading…

No-hassle emails from your home server

Emails: Why is sending them so hard?

Since I like to think I’m pretty handy with computers, I like to have one of them running all the time. To that end, I bought an HP ProLiant Microserver some time ago, and always keep it on. It’s a great little low-power server, and I use it for things like storing all my files so I can access it from computers around the house, automatically making on- and off-site backups every day, allowing access to my home networks, etc.

This setup is fairly straightforward, but there is one considerable pain point: The server needs to send email. It’s currently using ZFS with RAID-Z to store my files, and I would like to be notified if there are some disk errors so I can replace the disks. The system can easily send emails, but having those emails actually get delivered and not go into spam is a very different matter.

Mail nowadays: Not so easy.

If you’ve ever tried to run your own mailserver, you know that ensuring deliverability is hard. Mail servers on the internet today are very suspicious of new servers, due to the spam problem, so if you just start sending email from your new server and your home connection, one of two things will happen: Best case, it will end up in your spam inbox, and, worst case, it will be blocked and you’ll never hear about it.

The consequences of this can range from the inconvenient to the disastrous, depending on how important the email that was being sent is, so it’s easy to see that our server’s sending capabilities have to be as robust as possible. However, this isn’t a production mail system in a large application, so how can we solve the problem without spending a lot of time on it? Well, there’s a very easy way.

Continue reading…

Winning at Candy Crush

The Candy Crush Saga saga

I find Flash games on Facebook great fun. Not playing them, of course, that’s boring. As you may remember from my previous post, “winning at Puzzle Adventures“, I like to take a look into their guts and figure out how they work, and whether or not I can get insane scores with no effort.

When I discovered Candy Crush Saga, I was intrigued. All my friends appeared mad about this game, sending me so many requests for candy that their dentist would surely commit harakiri. I started playing a bit, and it wasn’t long until I had to stop playing, since the game only allows you a set number of lives per hour in an attempt to either extract money from you or coax you into spamming your friends with requests for the game, to increase its popularity.

Cheating at online games

This, however, wouldn’t do, so I fired up the Swiss army knife of web debugging, Charles Proxy (it’s a fantastic tool for this job). I started looking at the requests the game was making to the server, and saw one that looked promising:

Continue reading…