a view from afar - Home

Building darcs 2.0.2 on Ubuntu manually

2008-06-29T15:03:00Z

Recently had to work with a Ubuntu server and I wanted to get the latest darcs working on it. Simple install guide follows, pending this making it into.

One thing I've learnt in the process is that Ubuntu is very conservative about its packages - by default, you only get those that were approved at the time of a release, and even if you choose to get updates, they must be specifically ported to your version (I have tried this using 7.10 and 8.04). In FreeBSD and OS X (MacPorts) land, the ports tree marches on regardless.

Following are the commands you need. Apologies if there are some redundant steps on either Ubuntu 7.10 or 8.04, the packages are slightly different as 7.10's package repo contains GHC 6.6, whereas 8.04 has 6.8. (Again, an annoying frustration I found with Ubuntu.)

First install the build tools (note some of these are only needed if you wanted to repackage to submit back to the Ubuntu pacakge repo):

    # apt-get install devscripts build-essential wget cdbs \
                   fakeroot liburi-perl debhelper pbuilder dpatch quilt

Then install GHC and all the required Haskell modules:

    # apt-get install ghc6 libghc6-quickcheck-dev libghc6-mtl-dev \
                      libghc6-html-dev libghc6-http-dev libghc6-harp-dev \
                      libghc6-regex-base-dev

On 8.04 only (because it uses GHC 6.8) you need the regex-compat library:

    # apt-get install libghc6-regex-compat-dev

Finally a few libraries that darcs like to have around:

    # apt-get install zlib1g libcurl3 libcurl4-openssl-dev

Then the usual build procedure (I used --without-docs as this command will be sitting on a server, it saves time and also installing LaTeX packages):

    $ wget http://darcs.net/darcs-2.0.2.tar.gz
    $ tar xf darcs-2.0.2.tar.gz
    $ cd darcs-2.0.2
    $ ./configure --without-docs
    $ make
    # make install

Final note: You can get the older 2.0.0 release of darcs from Iain Lane's personal package archive. (I found this after I installed by hand.)

GeekUp Sheffield II / Encouraging Agile Discipline

2008-06-05T19:45:00Z

Last night I gave my first presentation in the form of the huddle at GeekUp Sheffield II. A few people asked me for the slides so I've posted them on my blog. You can grab the PDF version here.

I think the discussion could have gone on longer. If anyone wants to bash out some ideas, I hang around on the GeekUp mailing list, or you can email me direct (address on the first slide).

Many thanks to everyone who attended and spoke at the event, and to Jag for organising it. It was a great night, and I hope enough people will come forward to speak next month to make it even better.

My first adventure in github

2008-04-06T18:38:00Z

I now have an account on the github beta - and it's pretty cool. Since I've only done one thing with it I can only comment on one feature: forking. It's pretty effortless, and only took me a few minutes to get my Mailtrap LogParser extension up there. Best thing - assuming they are telling the truth :) - it's easy for the maintainer of the parent repo to merge the changes back in.

I suspect github will eventually change the nature of open source collaboration. But where is darcshub?

A simple log parser for Mailtrap

2008-04-06T15:05:00Z

I just came across a need to test emails sent in a Rails project. It's a RSpec story integration test I'm working on (using Selenium, not a RailsStory), and using ActionMailer in test mode is not good enough. Instead I'm using Matt Mower's handy tool Mailtrap

I created a listener class to start and stop mailtrap as the story runs - which is easy enough - but I needed something to parse the log file from Mailtrap. So I created Mailtrap::LogParser - code available here for anyone that's interested.

As soon as github gives me a beta login, I will fork the Mailtrap repo and add this in. For the mean time, you can use the tarball above.

This is just a stopgap solution really, until Mailtrap has a way of outputting a more structured (and therefore easily-parseable) output.

Probably going offline for a bit soon...

2008-04-05T14:16:00Z

Thanks to the incredible incompetence of our soon-to-be new broadband provider DST, my blog may be out of action for a while (10 days is apparently the worst case). DST let the MAC code we provided lapse, and Pipex are refusing to issue a new one. (Pipex actually claim that issuing a MAC code is considered a form of cancellation, so I'm not even sure why we still have a net connection at all.)

Unfortunately, due to the time I'm spending at work lately, I don't have time to move the blog to shared hosting. I intend to do that as soon as I can though... I'm getting ever so slightly sick of the universal contempt that consumer ISPs show for their customers.

Feed URLs moved - now using FeedBurner MyBrand

2008-03-24T16:41:00Z

Since Google bought FeedBurner, their MyBrand feature has been free. This means you can access FeedBurner feeds via "feeds.yourdomain.com". All it takes is a CNAME record in DNS to point feeds.yourdomain.com to feeds.feedburner.com, then you can access FeedBurned feeds via a URL you own. It's pretty generous of Google seeing as it means you can switch feed tracker with a simple DNS change...

Here is my Apache config (with added line breaks) to send the Mephisto-generated URLs to the FeedBurner ones:

    RewriteEngine on
    RewriteCond %{HTTP_USER_AGENT} !FeedBurner
    RewriteRule ^/feed/atom.xml \
                http://feeds.aviewfromafar.net/aviewfromafar-home \
                [R=301]

Actually the feeds.feedburner.com addresses haven't moved, so if you are subscribed to this blog via the Atom feed you will still get updates. But the correct address is here:

http://feeds.aviewfromafar.net/aviewfromafar-home

...and geeks like being correct, right? :D

Dealing with suckitude - installing a CanoScan LiDE on Mac OS X

2008-03-08T12:11:00Z

I just bought a CanoScan LiDE 25. Apparently I'm not the only person that thinks Canon's scanner software is a crime against humanity. I now use Yep for scanning, so I just need the TWAIN driver provided by the Canon installer. But the closest I got to making it work was seeing it install the files, then delete them immediately after. Yes, that's right. I WATCHED THE FILES DISAPPEAR from /Library.

I think Bart Simpson has the best phrase to describe this software - it sucks and blows at the same time.

My installation guide follows...

Basically the trick I used is to extract the file hierarchy manually from the installer. Download the package from the CanonScan page. Make a folder canon somewhere in your home folder (I did it on the Desktop). Drop the installer file in, then run these commands:

    $ cd ~/Desktop/canon
    $ cp ScanGear\ CS\ 11.0\ Eng_Installer.pkg/Contents/Archive.pax.gz .
    $ gunzip Archive.pax.gz
    $ cd /
    $ sudo pax -r -p e -f ~/Desktop/canon/Archive.pax ./Library ./Users

For those unfamiliar with pax, it's very similar to tar. The options are: -r to put it into read mode (to restore from an archive); -p e to preserve every attribute on the original file; -f to specify the archive filename; and finally, the list of patterns (paths) to extract from the archive.

If you extract everything, you'll end up installing a file called CNQL1213_ClassicNotSeize.kext in /System/Extensions. This makes me go pale with fear - the thought of a company that releases software that doesn't let you save scanned images until you quit installing kernel extensions on my machine just makes me shudder. It isn't needed to use the TWAIN driver, so I make sure it never goes near my OS.

If you follow all the instructions above, you should be able to use the CanoScan directly from Yep. It still launches the Canon interface, but at least you get to save the pages as you scan them.

Well Pingdom works at least

2008-02-21T14:03:00Z

I got my first two text messages today - one for this blog and one for www.patchspace.co.uk. I've got the polling resolution set to five minutes and to alert after 30 minutes' downtime, so I was pretty confident it was a power cut (and I was right). It's the second we've had lately, the last was in the middle of the night, before I subscribed to Pingdom.

I guess I was wrong in my last post - the real weak spot in my setup is being on a residential power grid. But then, it's not like I'm not hosting paid-for clients' sites from my home server, so the worst it does it make me look like a fool when nobody can access my blog.

Next step: buy a UPS. (A generator seems a bit excessive.)

A few notes about my infrastructure

2008-02-20T22:48:00Z

As part of starting my own business I've decided it's about time I get some essentials done right - namely backups, uptime monitoring, DNS hosting and email hosting. I'll also detail some of the other software and hardware I use. Nothing ground-breaking - but some of these services may be new and/or useful to my readers (24 now!!!).

UPDATE: Many Ayromlou wrote a blog post about setting Google Apps up with easyDNS, which is exactly what I've done.

Backups

I tackled this first. Since I'm running OS X Leopard, Time Machine is the obvious place to start. I bought myself a Lacie Quadra d2 500GB drive for £99, which is not bad seeing as a bare metal 500GB SATA drive will set you back about £75, and doesn't come with a FireWire 800 port on the back, or a funky flashing blue light on the front.

I'm actually a little disappointed with Time Machine. Clearly they pulled the encrypted backup feature but Leopard went gold - which means your backup hard drive is a portable history of your entire life and/or business just waiting to go walkies. And with no network backup features, you've got no protection against fire, earthquakes or particularly aggressive coffee spills.

I solved this with an online backup service. I considered Mozy, but settled on Jungle Disk. Jungle Disk seems more committed to openness: their desktop client is truly cross-platform (Win/Mac/Linux are all equal) and they provide sample source for decrypting data in the event they go under. (Having said that, the client is dog ugly, but hey, nothing's perfect.) Also, strictly speaking, Jungle Disk is really a network share - the backup is an extra feature you get from the client software. So it may turn out useful for document sharing.

DNS

For outgoing DNS I don't use my ISP's servers - I use OpenDNS. According to their own stats, they have 3 million users and serve 3 billion(!) DNS requests a day. They don't offer DNS hosting, but when I contacted them they recommended the Canadian company EasyDNS, who I now use for patchspace.co.uk, my company's domain (this domain to follow, when I get round to it). EasyDNS is the sort of company the internet needs - their customer service is prompt and helpful, they are actively improving the security of their services, and offer some useful freebies (eg myprivacy.ca to help stop WHOIS email harvesting). They also have a bearable web interface - anyone who has used 123-reg.co.uk will know what I mean.

EasyDNS let you edit TXT records on each domain, so you can configure Sender Policy Framework to help cut down on spam forged from your domains. SPF is something I only discovered recently, and I'm in the process of applying it to all my domains.

Mail

For all accounts @patchspace.co.uk I'm using Gmail as part of Google Apps. I avoided Gmail for ages because I hate web interfaces for things I consider desktop apps, but since they rolled out IMAP access I'm hooked. You can point your DNS MX records to the Google servers - and configure SPF of course - and send and receive everything from your desktop under your own domain. I use Apple's Mail.app, which takes a bit of configuration to map the sent/received/drafts etc, but afterwards integrates perfectly. (POP access is supported directly, so hopefully Mail in OS X 10.6 will support Gmail IMAP natively too). Gmail maps label to IMAP folders, and it respects slashes, so the label "Shops/Amazon/Marketplace" produces a nesting three levels deep.

Uptime monitoring

I looked at a few (I can't remember which others now) but eventually I settled on Pingdom, a Swedish startup. For $9.95 a month, the basic package lets you watch 5 sites/servers, with email and text message notifications. The web interface is a dream to use, and works exactly how you want it to. You can configure HTTP and HTTPS requests to check for text strings, handy in case your server is there but actually showing a proxy error. They have monitoring servers round the world, and let you check as often as every minute.

This server

This server is actually a desktop machine, running FreeBSD. It cost about £200, and for that I got a dual-core AMD64 processor, 1GB RAM, and two 80GB SATA drives, which are configured as RAID-1 using gmirror. When FreeBSD 7 is released I'd be interested in using gjournal, as the lack of a journalled filesystem in FreeBSD is frustrating. I'm not yet sure how to configure it alongside gmirror, although the intention of GEOM was to make the modules pluggable, so some combination should work.

Software wise, the web sites are hosted in or through Apache httpd (this blog uses mongrel through mod_proxy). I chose Apache partly by default, even though nginx is pretty trendy right now. Apache can do pretty much everything, and I've never had it crash on me (except, that is, when I was meddling at the time...).

The rest

The only weak spot in my setup is the really crappy SafeCom router that used to crash on me on a weekly basis. It's been well behaved lately, so I won't rock the boat...

I was actually surprised how much I ended up using hosted services. There are some pretty good deals out, and they are really dropping the cost of setting up an IT infrastructure. Obviously I've only gone through a few solutions here, mainly the critical ones that maintain your minimum bare minimum functioning state (working off a borrowed laptop in an internet café).

Certainly, out of all the barriers to forming a Web 2 startup or other IT setup, the cost of the supporting technology is pretty much a non-issue these days.

Tiscali is shit (oh, and monitor your web page load times with Ruby)

2008-02-05T22:49:00Z

UPDATE: apparently the inconsiderate bastard claiming ownership of my ADSL pipe is, in fact, quite entitled to do so - it's my ISP, Tiscali. Their bandwidth throttling is apparently a bit over-zealous. The Reg has more details on it - it's unusual of me to not follow their RSS feed but work has kept me busy lately. Suffice to say I won't be with Tiscali much longer.

Previous article follows...

It's become apparent that some inconsiderate bastard* in my neighbourhood has decided that the entire ADSL pipe for our area belongs to him. Every evening after about 5 or 6pm, our network grinds to a halt. At first it's just slow to load pages, but later it starts to take several minutes to access anything. Then, suddenly, late in the evening, everything returns to normal.

The ISP support guy (Tiscali, was Pipex) told me today that the second line support is in India, and they have no hesitation in passing the slightest ambiguity back to the UK. He wouldn't even believe me that it couldn't be a DNS issue with their old servers because I don't use their DNS servers (I use OpenDNS).

As part of the diagnostics, I was left collecting speed checks. Unfortunately, these turn out fine, as it's establishing the connection that is affected. So I wrote a Ruby script to load Google every 5 minutes for 24 hours.

* Now I've said that it will probably turn out to be an intermittent technical fault

Code follows, for anyone that may find it useful...


require 'open-uri'

def time
  start = Time.now

  begin
    yield
  rescue Timeout::Error => e
    return -(Time.now - start)
  end
  
  Time.now - start
end

start_time = load_time = nil

File.open("connection_test.out", "w") do |f|
  f.sync = true
  
  (12*31).times do
    start_time = Time.now
    time_result = time do
      open("http://www.google.co.uk/")
    end

    load_time, timeout =
      if time_result < 0
        [-time_result, true]
      else
        [time_result, false]
      end

    output = "#{start_time}:" +
                  "#{timeout ? "Timeout in" : "Loaded in"} " +
                  "#{load_time}"
    puts output
    f.puts output
    
    sleep 5*60 - load_time
  end
end

It's running now, and it due to finish about 6am Thursday morning...

Vote for agile developers on We Vouch For

2008-01-24T10:35:00Z

I came across this the other day on Kevin Rutherford's blog: a network for agile developers called We Vouch For. I'm not sure I like the name, but the idea is very worthwhile. (Think LinkedIn but for agile peer certification.) This was a topic that came up in the Q&A session at the end of Agile North 2007, and it was clear from the amount of discussion that the problem is far from solved.

I hope the site takes off, because hiring good developers is excruciatingly difficult. It's quite possible to get a hundred developer CVs and not one that's even heard of agile (or even something fundamental like automated testing). In fact, just being able to define TDD would put them in my top 5%, based on CVs I've seen in the past.

A quick brain-dump of things I think might be good on We Vouch For:

categorisation of skills, currently it's purely organic
some use of the certifications to rank developers (I wonder if the Google PageRank algorithm would work?)
some sort of geographical mapping
dating certifications (that is in the sense of attaching a date to a certification)
(possibly) a use of the Apprentice-Journeyman-Master system, although while I like those terms they are difficult to pin down precisely

I have signed up, but I haven't worked with any of the developers on there so I'm not likely to be certified any time soon :)

I blog with...

2008-01-19T17:37:00Z

...Mephisto. This is my folksr.de vote for the poll What is your favourite blogging engine or site?.

I Get Things Done with...

2008-01-19T16:00:00Z

Vitalist. This is my folksr.de vote for the poll What electronic system do you use to Get Things Done? I just created.

I've actually tried a few GTD systems to one extent or another, and all of them have advantages and disadvantages. Vitalist doesn't work perfectly for me, but it does have some really useful features.

I will probably review the system I use every year or so - I was astonished how many new online systems there are around that I'd never heard about. You might want to check out the poll and see if it throws up some new ones you haven't seen (I didn't label which ones are web and which are desktop, though).

[OLD] Class-table inheritance in ActiveRecord

2008-01-19T13:53:00Z

This article is nothing new. I first published it on November 7th, 2006. The site it went up on is no longer running, but it seemed a shame to let it disappear. I imagine the ActiveRecord codebase has moved on significantly enough that the patch would be very difficult to apply now.

There were quite a few comments to the original post (not preserved here) expressing interest in CTI. Unfortunately, there were none from any of the ActiveRecord team, but then, DHH doesn't even like foreign keys, so expecting interest in CTI would be downright naive.

The original article

I'm pretty fanatical about making sure databases are correct and unambiguous. Unfortunately, single-table inheritance makes this hard, as you have to store attributes about distinct types of entities together in one table. To me, class-table inheritance is the correct way to record information when you have an entity type hierarchy in your database (and therefore a class hierarchy in your application).

About the implementation

Normalised data

I started by solving the relatively simple problem of reading and writing attributes in satellite tables. This is useful in itself for various reasons I won't bore you with now, but it's an essential part of the CTI implementation.

There's a new normalized_data_test.rb file in the tests that shows how this works. It consists of two class level methods in ActiveRecord::Base, normalized_attribute and normalized_attribute_group. Columns of the satellite table appear to your application to be in the model table itself.

Reading the class-tables

I've tried to make CTI work as transparently as STI. Given these tables...

CREATE TABLE people (
  id serial PRIMARY KEY,
  first_name varchar,
  last_name varchar
);

CREATE TABLE employees (
  person_id integer NOT NULL REFERENCES people (id),
  salary decimal(10,2),
  hired_at timestamp
);

and these classes...

123	class Person < ActiveRecord::Base; endclass Employee < Person; end

then ActiveRecord will follow this procedure (or something evquivalent, as the algorithm has to rerun when classes are reconfigured) to initialise inheritance:

check the people table exists
check for a type column in people
- if it does - use STI and stop here
- if not - assume CTI
check the employees table exists

The convention is that the primary key in each table is named after the immediate supertype. To add a Manager subtype to the above example:

CREATE TABLE managers (
  employee_id integer NOT NULL REFERENCES employees (employee_id),
  has_pointy_hair boolean
);

12	class Manager < Employee; end

Manual overrides

If you are working with a legacy schema, you can override the table name and the primary key (which is actually a foreign key but I wanted to re-use the method name).


class Customer < ActiveRecord::Base
  set_table_name "suckers"
  set_primary_key "sucker_id"
end

Mixing inheritance types

I've tried to make CTI integrate as seamlessly as possibly with STI. You can use both in the same inheritance chain, as long as STI is at the end. For example, to extend the example above:

ALTER TABLE managers ADD COLUMN type varchar;

You can then do:

12	class ImportantManager < Manager; end

There are three reasons I didn't attempt to make STI work in the middle of an inheritance chain:

it would massively complicate the algorithm that determines the class type
it would mean maintaining the inheritance column ("type") to store names of classes that actually have their own table, potentially an integrity nightmare (not to mention the fact you could potentially have multiple blocks of STI in an inheritance chain)
I don't imagine anyone would actually want to do it (if you want to switch from STI to CTI, refactor your database!)

What's left to do

There are a few minor things, mainly:

Currently CTI objects let you call attribute accessors that should only exist in subclasses, but I'll fix that soon.
I need to test that the mixed inheritance types work as described above.

I've tested that CTI works on its own. I've tested that everything (well, except for one thing) that used to work in ActiveRecord still works with CTI in place, just not using it. However... I haven't tested that all the existing ActiveRecord features work with CTI classes. The reason is that this could potentially involve duplicating most of the existing unit tests, and I want to get some feedback from the Rails community first. I imagine the associations still need some major work.

The patch is available here: [http://dev.rubyonrails.org/ticket/6566]

Feel free to leave feedback

Links

I've had a search on the interweb and these are the most interesting articles about CTI in Rails:

How ironic can an unsubscribe email be?

2007-12-28T00:20:00Z

I have recently been sick to death of PacificPoker.com emails. After clicking the unsubscribe link in their spam^H^H^H^H newsletter for the fifty thousandth time, I can say - without hyperbole - that I was not very happy. My festive spirit somewhat dampened by yet another promise of a chance at the $75m dream ticket, I sent them this rather terse email:

    Subject:    TAKE ME OFF YOUR MAILING LIST
    From:       <my address removed for spam prevention>
    Date:       December 27, 2007 11:50:19 am GMT
    To:         support@pacificpoker.com

    I have lost count of the number of times I have tried to
    unsubscribe.  Please remove me IMMEDIATELY

Their reply is priceless...

    From:        support@pacificpoker.com
    Subject:     Re: [R] TAKE ME OFF YOUR MAILING LIST
    Date:        December 27, 2007 3:40:33 pm GMT
    To:           <my address removed for spam prevention>

    Dear Ashley,

    Thank you for contacting 888.

    Ashley, we would like to inform you that we have
    removed you from the mailing list.

    We apologize for any inconvenience our advertisements
    may have caused you, as it is certainly not our
    intention to offend or upset anyone.

    Ashley, Check out the ?$80,000? tab in Pacific Poker to
    find out how to play for your share of at least $80,000
    every Sunday at 15:30 PPT! Buy into the Qualifier for
    only $88 + $8, play a Micro-Satellite for as little as
    50¢ or win your seat for free from a Freeroll
    tournament.

    Thank you for being a member of 888 and for the
    confidence you have placed in us. Please contact us
    whenever we can be of further service.

    Kind regards,
    <Some Hispanic-Named Support Guy>
    Member Support Representative
    support@pacificpoker.com
    www.888.com
    **********************************************************
    Get $150 just by inviting a friend to Pacific Poker! And
    your friend will get $50! It's so easy! For details,
    visit www.pacificpoker.com and click on the INVITE A
    FRIEND icon.
    NOTE: *Exceptions apply according to country of registration.