Archive for the 'CoolStuff' Category

Note: I've reorganized this site to use tags; the category archive remains to support old links. Only posts prior to April, 2006 are categorized. Tag Archive »

Magic with wget

As mentioned previously, I’m in the process of switching hosts. Last night I uploaded my entire site at the new host, and set everything up. Along the way, I decided to refine my internal directory structures a bit. Everything is working fine on the new host, and once I get the e-mail accounts squared away I’ll be ready to make the DNS switch.

In the meantime, I have a synchronization problem. When I uploaded my site to the new host, I used a site backup (tarball) from the old host. This made it easy to preserve directory structures as well as timestamps. Because of the way Blosxom works, the file datestamps on my posts are very important… Blosxom uses them for the post date/time. After the upload (and after I moved a few things around), both old an new sites were in sync, with the same posts in the same categories, and having the same timestamps. From that point on, new posts to the old site are out of sync with the new site. Posting to both locations is no good; the timestamps will be off and it breaks the programmer’s first virtue (laziness). I can’t use scp because my old host doesn’t offer shell access.

I went looking for a way to transfer a file from the old host to the new host, preserving the timestamp, and preferably making it easy to keep things in the right directory. I’ll have to run this after each post until the old site goes away. I looked at cURL first, but it didn’t quite do all I needed, so I turned to wget. Magic ensued.

The setup: from the shell account on my new host, grab the file from my old host. The file can only be retrieved from the old host via FTP. As an example, I wanted to sync my prior post on the new iPods. It’s in the category Apple and the “stub” title is newpods. On the old host, the file is in /blosxom/Apple/newpods.txt. On the new host, it needs to go into ~/ I didn’t want to specify the directory in both the source and destination. The solution:

cd ~/
wget -N -x -nH --cut-dirs=1 

The first command just puts me in the base directory for my posts on the new server. The magic is in the wget command. -N turns on timestamping, preserving my timestamps. -x forces wget to create directories locally to match the remote (this is the default for recursive fetches). Normally, the dirs created would start with the host name (e.g.,, but -nH removes the host name. Finally, –cut-dirs removes directories from the front of the path, so the file blosxom/Apple/newpods.txt on the remote end becomes Apple/newpods.txt locally. This combined with the initial cd lets me handle the changes I made to my directory structure. After I publish this post (on the old host), I’ll run the same command from the new host, plugging in the new file/path.

One detail of note: the above wget command will try to login anonymously, and give up if it fails. You can specify user and password on the command line, but bad idea on a shared host (think ps -aux, although my host protects against this). If you specify the user without password, you don’t get prompted for a password. The way around this is an old UNIX standby, the .netrc file.


I’m too tired tonight for a proper post, so I thought I’d just toss out some comments on what’s been catching my notice lately. In a word, Podcasting. To quote Les Orchard, “I’ve found myself falling for the Podcasting hype.”

If you haven’t heard of Podcasting, think newsfeed agregators meets audio blogging. Go read Les’s post, it’s an excellent look at why this is cool, and what’s new about it. Then go check out, the current epicenter of the podcasting universe. The site is run by Adam Curry, former MTV veejay, internet entrepenuer, and a driving force behind the podcasting craze.

If you want to give a listen, there’s already tons of content springing up, but try starting with Adam’s Daily Source Code. It’s big… the show runs 45 min. to an hour each day, and the bitrate is high, but it’s a good example of what’s possible. The production values are excellent, and he does it all from his Mac, eschewing professional audio equipment. The content tends to be very focused on podcasting itself (as with any new web tech, the initial level of navel-gazing is very high in podcasting so far)- but it’s a good way to get a feel for what’s going on.

Assorted Bits

Haven’t done a link dump in for-ages, and still don’t have a linkblog going. Here’s a few bits and bobs I’ve been looking at lately.

  • Simon Willison discusses a very clever bookmarklet by Nic Wolfe to help you manage your passwords. Click the bookmarklet and get prompted for a Master Password. This is combined with the domain of the website you are visiting to “create” a password for the site, which is auto-pasted into all password fields on the page. The “created” password is an MD5 hash of your master password and the domain name. This means your Master Password and your created password are never stored anywhere. Just use the bookmarklet from any browser, anywhere- use the same master password and you always create the same site password for any site, and every site is different.

    Check out both links, including the comments at Simon’s site. There are a few issues with the bookmarklet. I’ve been taking a shot at resolving them. May have something to post soonish.

  • Been meaning to learn XUL, the UI language of Mozilla browsers, and other bits related to Firefox extension programming. Found a few resources: XulPlanet, Firefox Extension Tutorial, Mozilla Application Object Model reference.

  • I also have a copy of Nigel McFarlane’s Rapid Application Development with Mozilla on the way. I usually go for O’Reilly books, but their Mozilla Dev book is pretty out of date, an I’m interested in Firefox dev.

  • Remember Magic Eye Puzzles, aka Stereograms? Check out one in ASCII.

  • dugh at has challenged all comers to a week of solid blogging. Since I actually managed to post the last couple of days, I’m going to try and up the ante, and post every day in October. This post counts.

  • They didn’t play their best game of the season today, but the Eagles did beat Chicago to go 4-0 on the season. Be sure to check out the Style links below on the right for the exclusive Eagles Stylesheet (if it’s not already active). Next week is the dreaded bye. Go Birds!

You CAN take it with you…

…at least in the case of Mozilla Firebird. David Tenser’s Firefox Help has a set of simple instructions for setting up Firefox for Windows to run from a USB memory stick or other removable media device, including a copy of your profile. Now you can run Firefox (with all of your bookmarks, plugins, skins, and settings) from any Windows PC with a USB port, in moments. Groovy.

While my primary personal machine is my Mac Powerbook, I do use a Win2K machine all day at work, and I have Firefox customized for my use. I’d bet that creating a similar portable setup for OS X or Linux would be just as easy, however, carrying a Win install around on my 64 Meg USB thumb drive is probably the best for flexibility. For example, the internet cafe I used in St. Andrews, New Brunswick recently while on vacation had PCs, not Macs. Wish I’d had Firefox on my thumbdrive then.


It’s only been two days since my last blog entry, but it feels like its been much longer. I’ve had a number of posts kicking around in my head, but I’ve been too busy to write them. I still have a few full-length items I intend to post, but for now, I’m going to do a bit of a brain dump.

  • Spanish studies are progressing. I’m still really enjoying it. I’ve completed 22 of 30 lessons from Pimsleur’s Spanish I course. I should be further, but I ended up repeating around 15 lessons when I switched from the “bait” edition, Pimsleur Instant Conversation Spanish, to the full edition. I’ll post more on that later. Meanwhile, I’m about ready to start searching eBay for Pimsleur’s Spanish II.

  • I’ve been using Firefox and its antecedents for over eight months. Only in the past few days have I figured out that CTRL+TAB lets you toggle through tabs in the current window. Boy, have I been looking for that shortcut.

  • While typing the prior bullet, I decided to check the spelling of antecedent. I automatically went to– it’s where I always go. What’s your favorite online dictionary? Drop a note in the comments.

  • Speaking of comments, the comment system on this site still sucks. I think I’ve finally figured out how I’m going to handle it, I just need to spend a little time on implementation. No hints. Well, maybe one.

  • Speaking of the ‘Fox, I’m continuing to use version 0.8 on my Mac after shunning it for two months. It’s mostly working well, but there are a few issues. I haven’t taken the time yet to figure out if these are problems with the browser or the Tabbrowser Extensions, but I can no longer drag-and-drop to re-order tabs within an window. Dragging tabs from one window to another continues to work.

  • After giving my last spare monitor (20″ trinitron) to my brother a couple of months ago, I’ve been itching to play with some of my old PC hardware that was laying around my basement. Having a clean garage complete with a big workbench and tons of shelf space hasn’t helped. I finally broke down this weekend and bought an el-cheapo monitor, so now I’m playing with out-of-date hardware.

  • It’s amazing how the price of technology continues to drop. The monitor I bought today, after rebates, cost me about USD 80.00 at Circuit City. It’s a 17″ flat CRT, with resolution up to 1280×1024. It looks great. It was only a couple of years ago this would have been an expensive monitor. You can’t even buy a CRT smaller than 17″ at most retailers these days… it’s just not worth it for the retailers to carry them, they are so inexpensive. Flat panels have really taken over the market.

  • To help me figure out what kind of hardware I have, I went looking for a Linux live CD (a la Knoppix) that featured hardware reporting of some kind. Live CDs already know how to scan your system, but I don’t know how to get a report of what was found. I didn’t exactly find what I was looking for, but I found a list of Linux Live CDs, containing dozens of the things. What I did find that met my needs was the Ultimate Boot CD, a bootable CD containing images of dozens of mostly DOS-based boot floppies, each containg a different utility. You boot the CD, select the image you want to use, and it boots from it. I was able to inventory my hardware with Navrátil Software System Information, or NSSI.

  • I pieced together a AMD K6-2 300 MHz system with a 9G hard drive and 384M RAM. Time remaining on my Fedora Core 2 Test 3 installation: 15 minutes. I’ll be blogging more on this later.

  • Sean is contemplating dial-up internet over VoIP. Seems vaguely Oedipal to me (dial-up over VoIP… not Sean).