Category Archives: Programming

Dependence on the “cloud”

Leave a reply

I’ve been thinking a lot about “cloud” services I’m at least partially dependent on. And they fall into three general categories:

  1. Services I use to make my own personal computing easier (Evernote, Dropbox, Newsblur, Google calendar).
  2. Services I use to share things publicly or with a set of contacts (Flickr)
  3. Services I use generally for communication (Google+, Facebook, Twitter, various IM services.)
  4. A possible fourth category: virtualized linux hosts that I pay for.

If you know me, it wouldn’t surprise you that this bothers me. It bothers me a bit to not own my own tools — it bothers me a little to use the ones where I have a clearly defined business relationship with the service provider where I pay for what I’m getting; but it bothers even more when I’m getting something for “free”, because they’re typically getting something of even greater value out of me than a few bucks a month: marketing data, patterns of who I associate with, etc. In some sense, the users of these social networking services are merely a means of crowdsourcing content to be used by the provider.

I prefer the old vision of personal computing: once you’ve bought the hardware and software, you can do things indefinitely with it, no one has the power to take it away from you, you are independent of outside interests. With that perspective in mind, I’m attempting to untangle myself from some of these services. For the moment, short of devoting a lot of time to trying to get distributed social networking off the ground, I’m not even going to try to seriously touch #3. Also, #4 is mostly unoffensive to me, though it’s not clear that even today that virtual hosts offer the same legal rights (fourth Amendment) as hardware which you own.

So some plans: Flickr is slowly dying, I’d like to go back to some kind of self-hosted photo gallery software. I don’t really like a lot of the options out there, I may roll my own. Dynamically-generated content is dumb for everything but commenting, everything else should be static HTML, thumbnails.

I only use Google calendar because it’s an easy way to sync my iPhone to a calendar server, and it is an easy way to share my calendar with Tien. On the first part, I think I could solve my problems by running a CalDAV server. Not sure about whether it would accomplish the sharing too.

Dropbox, while I just use the free 2GB account, is darned convenient — I use it for 1password syncing, and I use it as a place to put low-security documents so I can easily share them between computers. It is amazing how long it took to give people what they want in a network filesystem — even though it would have cost more years ago, none of it was impossible (though the market was probably much smaller.) Most of the self-hosted open source alternatives to this are pretty fiddly.

Evernote is the one I’m least likely to want to give up. It’s been so useful, and this is why I’m more dependent on it than almost anything else I listed. I frequently shoot photos of small pieces of text with my iPhone and add them to my Evernote account. I use it to remind me of food or drink I enjoyed. I use the desktop version to replace my former use of things like Eaglefiler (and before that, a directory full of text files.)

For the photo gallery (and an eventual move away from WordPress), I want to cook up my own open-source, self-hosted equivalent of Disqus. I just need to find a little more free time…

Haskell and Erlang, part 1

Leave a reply

I’ve been rather quiet here lately. For the most part, I’ve been just too busy, and in what free time I’ve had, I’ve preferred to spend it learning programming languages and related stuff.

I spent a few days playing with Haskell. I should write some more about what I found interesting about a strongly-typed pure functional language, one where you have to essentially do side-effects (such as I/O) by passing the state of the world to the function and returning it from the function, albeit with some neat shortcuts (Monads, etc.) to make that less awkward than you’d think. It was possibly even more fascinating than when I first learned Standard ML back in college.

I found that it was really difficult to express non-trivial things in Haskell however. I think this may become easier with more exposure to it in the future, but for now I’m putting it on the back burner. I think I’m going to aim to make one day a month “Haskell day” wherein I advance my exposure to Haskell, because I find it very perspective-expanding.

Largely because this renewed my belief in the usefulness of functional programming languages, I then decided to take a closer look at Erlang than I have in the past, including ordering the Joe Armstrong book. At the moment, the toy project I’m working on is a twitter client that acts as a irc daemon, so I can just connect to this in my irc client (which easily handles multiple irc networks), and then I can read twitters and send twitters from within it, not wasting interface-space with another application, or time by going to the twitter homepage.

Erlang is rather ideal for applications that do a bunch of network stuff, it provides for the easy creation of lightweight processes and the easy sending of messages between them. Sending messages is non-blocking, and the system insures that the message will be in the receiving process’ mailbox, ready to be received with a pattern-matching construct. It does not provide object-oriented programming constructs in the usual way people think of OOP, however, it can be argued that passing messages to processes is directly analogous to invoking methods on objects. (See a good blog post by Ralph Johnson and a comment thread on LtU.)

So far, I feel like I understand 95% of the language, and about 50% of OTP (Open Telecom Platform, an included set of libraries and conventions for building applications.) It feels pretty good, and practically quite usable. It does feel a little less modern as functional programming languages go. It doesn’t have currying, and some of the syntax is a little less clean than you find in Haskell. But, on the whole, it does have a lot of the features I do want, including a decent pattern matching syntax.

Yahoo! Pipes

Leave a reply

I finally got around to trying Yahoo! Pipes. What this is, primarily, is a visual programming language for operating on RSS feeds. It also handles some other data formats, but that’s the essence of it.

While it seemed cool when I first heard of it, I only recently had something which made it useful to me. I upgraded the Squeezecenter software for my Squeezebox (wireless ethernet music player), and it broke my favorite third-party module, SuperDateTime. I use SuperDateTime to replace the builtin clock functionality with one which shows the weather.

While I could just fix SuperDateTime (or wait for the maintainer to fix it), I noticed that the Squeezebox has a very simple RSS reader. This led me to googing for RSS feeds of weather and configuring it to display one. I happened to opt for Yahoo’s, because it’s well documented. But, this kind of sucked — The verbose form of the weather report and forecast was so much text that you could stare at the the squeezebox for half a minute while it scrolled all the text by.

Here’s where the Pipes comes in. I decided, why not minimize the feed to only the essential information. Basically, a simple task is required:

  • Fetch the weather RSS for the appropriate location.
  • Extract the current temperature and condition (Sunny, Cloudy, etc.)
  • Extract the temperature and condition from the nearest (in time) forecast.
  • Build a string out of this, one line of text.
  • Spit out a feed where there is one item, where the title is the location and the description is this string.

But, while Pipes makes some things so easy (parsing XML, for example), it makes other things relatively hard or clumsy, at least to someone who is familiar with (text) programming languages. It was not without a learning curve.

Here is the result of my efforts. (Click view source if you want to follow along.)

What it does, in Pipes modules is:

  • Get a zipcode and units (F or C) from the user who is instantiating it. (Might as well make it generic.)
  • Convert the units to lowercase if they are not already (for the RSS feed syntax.)
  • Convert the units to uppercase for display
  • Fetch the corresponding weather feed.
  • Loop over the items in the input, and for each, use the String Builder module (a lot more clumsy than a simple s(n)printf or string concatenation operator) to build up the output string, assigning this string to the description field of the item.
  • Yet another loop to build a feed of new items, assigning the zipcode to the title of the item and the description to the description, eliminating the irrelevant data in the source feed.

Now, I suppose that wasn’t so bad. I know that my second use of Yahoo! Pipes will be much less troublesome than the first time I used it, but the lack of easy to use sprintf-like constructs and the need to use this explicit “loop over this feed and apply this module to every item in it” seemed a bit tiresome.

It does however provide a bit of inspiration for how one could create an easy to use (for the experienced python programmer) set of classes in python for doing this sort of processing of RSS feeds while abstracting away the low-level details. Also, I’m pleased to see an attempt at a visual programming language — something I think we see so little of, not for reasons involving the limitations of computers, but because of the power of plain text.