John of All Trades

So, you may have noticed that I recently migrated this blog from Pebble to WordPress. Actually, I migrated two former Pebble blogs, plus one former WordPress blog, to this new one. The WP install was simple, and so was the transfer of blog entries from one WP instance to another. Migrating from Pebble was a little more challenging, particularly since I had a lot of posts that I didn’t want to handle manually.

I started out by looking at the Pebble2Wordpress importer code, which is written in Ruby. (I first found a reference to this project from The Spritle Blog blog post, which has some more information about how it’s meant to be used.) I selected this project in particular because it’s written in a language I can fine tune as needed.

I’m not sure whether this project ever really worked as-is out of the source repository, but I can say for sure that it didn’t work for me using the instructions provided on the blog post. In an attempt to understand where the developer was coming from, and determine whether I could cobble together something to work for my own particular migration, I dug into the source. What I found seemed pretty overwrought and confusing, particularly for something that amounts to a simple migration script to morph XML documents into DB calls (not something that will ever attract a huge user base). After refactoring the code to provide minimal error handling, and to inline all class definitions into one ruby script alongside the code that uses these classes, I thought I had something understandable to start testing.

The general approach of the code is to create ActiveRecord representations for terms (read: categories), term-post relationships, posts, and comments. Once the database is connected, the script iterates through all *.xml files in the inputXML directory, and parses the Pebble XML format into Post/Comment/Category instances which are then saved into the WP database. During the parsing process, certain things like image URLs are massaged from the old Pebble layout to something more compatible with WP. Since the default WP uploads directory has the date baked into the file path, I opted to simplify things and put all migrated files into the uploads/ base directory. This has an added benefit of giving me one-stop shopping for anything that breaks during the migration process, instead of picking over a nested directory structure.

Before you can start using the code, you’ll need to install some RubyGems and OS hooks:

$  sudo apt-get -y install rubygems mysql-dev
...
$  gem install activerecord activerecord-jdbcmysql-adapter
...

Next, you need to create a new directory alongside the convert.rb script, called inputXML/. Into this you should copy all of the XML files from the pebble data directory. My data directory was in /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/, so I used the following commands to achieve this:

$ find /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/2009 -type f -name '*.xml' -exec cp '{}' inputXML/ \;
...
$ find /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/2008 -type f -name '*.xml' -exec cp '{}' inputXML/ \;
...
$ find /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/2007 -type f -name '*.xml' -exec cp '{}' inputXML/ \;
...
$ find /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/2006 -type f -name '*.xml' -exec cp '{}' inputXML/ \;

I know, using some more arcane shell-fu I could have combined the individual commands into one. However, rather than take a chance of fouling something up with an overly-complex command line, I chose to use the Base arrow-up feature to retrieve and modify my previous command. Simple, effective, and nearly fool-proof.

Luckily, in my case I had a bare WP install that I could abuse a bit while I fine tuned the migration code. Along with a decent database client (I love the free version of DBVisualizer for this), rolling back failed migrations are a breeze. After a failed migration attempt, I simply issued the following SQL commands to get back near enough to the base install:

DELETE FROM wp_posts;

DELETE FROM wp_comments;

DELETE FROM wp_term_relationships WHERE object_id $  999

NOTE: I chose to delete term relationships where the object ID is 1000 or greater, since it seems like the auto-increment field for posts starts at around 1000. This may not be the case in all environments.

I’m not going to pollute this blog post by giving a full listing of the migration script, but you can download it here: convert.rb. It has an accompanying database configuration YAML file that looks like this: database.yml. The script expects the inputXML/ directory and the database.yml file to be in the same directory as the script itself. Use the script if you want to, enjoy it, but above all, DON’T BLAME ME IF YOU BLOW SOMETHING UP.

I then tested, rinsed, and repeated until I was pleased with the result. As a final step, I copied the contents of the files/ and images/ directories from the Pebble data directory into my WP install:

$  cp -rf /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/images/* /opt/web/sites/johnofalltrades.name/htdocs/wp-content/uploads
...
$  cp -rf /opt/web/sites/ejlife.net/var/pebble/blogs/buildchimp/files/* /opt/web/sites/johnofalltrades.name/htdocs/wp-content/uploads
...

That’s about all there is to it. Hackish? Maybe. Effective? Definitely. I’m very happy in my new WordPress digs!

UPDATE: I’ve submitted my changes back to the project in this issue.

Yes, I took the weekend off from blogging. Sorry if that caught anyone by surprise, but this weekend I was pretty caught up getting ready to host my youngest brother and his wife for Thanksgiving. These are plans we set in motion months ago, and Emily and I are both really excited to see them. So, no time for blogging what with all the cleaning, grocery shopping, and menu-planning.

This morning found me at the doctor’s office, attempting to avoid the various black clouds of death circling the room as I asked for a new prescription for my allergy meds. If you don’t have allergies but always wondered what it’d be like, you should visit Florida sometime. It’s kind of like those 747’s they take people up in so they can see what it’s like to be in space, in a weightless environment…except, instead of getting to know what it’s like to be a cool astronaut, you’d find out what it’s like to see the world through watery eyes and a congested brain. It seems like just about everyone I know here has some sort of allergy. Yet another sign that Florida was never designed for human habitation, in my opinion. Anyway, back to the story. While I was talking to the doctor this morning, it came out that I’d been laid off recently. I never would have guessed, but I think people’s worries on my behalf are worse than my own worries for the situation. It’s kind of like having a very distant relative die, then having that fact come up in conversation over and over (say, because of your absence to attend the funeral). People’s sympathy is very nice, and I do appreciate it; but I don’t feel deserving of this much pity. I know unemployment is a nasty, desperate situation for many people, but I’m still hoping to land on my feet. Any day now.

Which actually brings up another point, in a seemingly contradictory way. I entered this process feeling like it was simply time to roll up my sleeves and get about the business of becoming re-employed. I still feel that way…mostly. But while I do still feel hopeful that this will end well and relatively soon for me, I’ve started to notice something slightly predatory lurking in the back of my mind. Stalking me. I’m getting whiffs of its scent, and it smells like fear. Last week it was no big deal to stay upbeat about my prospects, and I think it was largely because I had a lot to do. I got in up to the elbows, started talking to people, writing my resume, and looking around to see the lay of the land employment-wise. Things were – and are (did you see that shadow?) – going well. But this weekend, things changed subtly. I finished my resume, insofar as you can finish a thing like that, and ran out of people to talk to. I had hit the first interval of waiting since last Tuesday, and I’ll admit to feeling a little helpless about it at the time. I’m unsure about putting this in a blog post, but I can definitely see that, if this process were to drag out for months and months (or, for some, more than a year), it could turn pretty damned depressing.

Today, I also learned that keeping busy isn’t the same thing as staying productive. I have plenty of coding ideas I’ve been meaning to work on for some time now, and I took some time today to explore DNS SRV records, CouchDB, and some other fun stuff. And, while my explorations are actually done with a goal in mind, it is a personal goal, not something with which I’ll be reversing the recent financial tides. The last time I had this sort of time to spend exploring ideas, I was fresh out of school and looking for a job. I was in no particular hurry then to get started with working life, so I suppose this nagging drive to be productive wasn’t quite as well developed then. I expect retirement to be like this in some ways…

Well, I guess that’s enough semi-philosophical blathering for one night. I’m still determined not to fall through the cracks here, and there are still a lot of moves I can make. My approach here so far is still self-directed and voluntary, to a large degree. So, if you’re thinking of offering condolences like you would for a recently deceased fifth cousin, twice removed…don’t worry, I’m still fighting.

Today the watchword is patience. I’ve gotten through all the paperwork that needs attention for the time being, and wrote a fairly decent resume to get things started. I’ve got a job lead or two that I’m following up (I may have mentioned this before), but now I’m through a lot of the fast running, and its time to decide on a longer-term strategy. I’m really hoping to land a telecommuting job, since Gainesville doesn’t have much to offer (again, covered this) and it’s not my first choice to move for now. Since I’ve got a couple ideas for telecommuting, Emily and I agreed to see where they lead before I start a cross-country interview tour trying to land Any Old Job.

So, we wait. Waiting is a little disconcerting after running so hard the first few days, but in the meantime, I’m getting feedback on my resume, and trying to spit-polish it until it glows. I’m also doing a little professional self-improvement in the meantime, and taking advantage of the time to clean up some loose odds and ends related to family IT support.

This probably isn’t the most thrilling post to read, and I think that’s somewhat appropriate to the mood of the day. I’m still humming with excitement and hope, but for now I have to be patient.