Archive for January, 2005

Site Linking Policy

All of the content on jclark.org is Creative Commons licenced - you may use it as you like as long as you give attribution and share your changes. This has been my policy almost since the beginning of the site.

Linking to the site is also encouraged; I make every effort to ensure my permalinks always work. However, linking directly to images is prohibited. If you’d like to use one of my images on your site, please make a copy and host it yourself. Someone has begun linking directly to one of my badges, which I consider a gross abuse of my bandwidth.

I have searched all over the offending website, and cannot find an email address for the site owner. Since I’ve never posted a linking policy, I would like to contact him and allow him to change his site before I take steps to prevent this. Therefore I’m posting this now, to allow him time to correct the problem. By this weekend I will be actively blocking this and possibly serving alternate content instead.

The Mail Dilemma

One of the side-effects of getting my new iMac G5 is that I now have two systems from which I want to read email. I currently use POP to retrieve my mail to the Powerbook. If I move my POP setup to the iMac, I can no longer check my mail from anywhere via the Powerbook. This is compounded by the fact that I currently have 7 POP accounts from which I fetch my mail.

Looking at the list of accounts, there are really only two that I need to access from anywhere. One is my jclark.org address which is published on this site, and is my primary address. The other is an email address provided by my broadband provider; it’s the address I use for all business transactions, including my domain registrations and hosting plan. That has to remain separate from my hosted email accounts, obviously.

So, even If I move all of the other accounts to the iMac as POP accounts, I still have to address which I’d like to be able to access from anywhere (defined as: from the iMac at home and from the Powerbook anywhere). I want to have my old mail available as well, not just new mail. I don’t like webmail; even GMail doesn’t really cut it for me.

So, I’m considering switching to IMAP. I don’t have any experience with IMAP, so I’m not sure how to judge it. The next issue is where to host it. My webhost offers IMAP, and they recently increased my storage limit, so that might be reasonable for my hosted email address. The other address is not hosted there, and I don’t know if there’s a way I could relay it. I’m aware of things like fetchmail, but I don’t know if I could have my host fetch mail from an external account and put it into an IMAP account.

There’s also the possibility of running my own mail server at home. The iMac could likely handle it, but I’ve also got an old box or two that could easily become a mailhost. The problem then becomes one of addressing, as I don’t have a fixed IP address. In reality however, my IP address almost never changes, and I’m sure I could setup a script on the mailhost to update a file on my webhost whenever it does.

The real dilemma here is that I don’t really know enough about the technologies to make an informed decision. I understand the basic differences between IMAP and POP, but I know little about IMAP or how to setup a server. Does IMAP use mbox files? I’m aware of unix-y tools like fetchmail and procmail, but I know little about them. Eventually I’m going to have to spend a bunch of time Googling and reading. Suggestions for a one-stop web (or book) resource covering all of this would be appreciated.

HOWTO: Build DB XML 2.0 with Python Bindings on OS X

Sleepycat Software recently released Berkeley DB XML 2.0, a native XML database built on top of Berkley DB. The major change from DB XML 1 to 2 is the addition of XPath 2. and XQuery 1.0 support. Here’s what I did to build and install DB XML 2.0 along with python bindings on OS X 10.3.7.

Having previously built DB XML 1.0 on OS X, I was pleased to see that all of the prerequisite libraries (Xerces-C, Pathan 2, Berkeley DB) are now included in the DB XML source package. After downloading the tarball from the website (see link above), extracting it and building everything was easy: %> cd /Users/jclark/ext %> tar -xzf dbxml-2.0.9.tar.gz %> cd dbxml-2.0.9 %> sh buildall.sh

The buildall.sh script creates an install directory in the build root and installs all of the packages to this location. This gave me the following dir structure:

  Users
   + jclark
      + ext
         + dbxml-2.0.9
            + install
               + bin
               + docs
               + include
               + lib

Next I wanted to build python support, which is not built by buildall.sh. The README cautions that the python bindings require the pybsddb package, which ships with python 2.3 and later, but which must be compiled against the same version of Berkeley DB as XML DB (the XML DB package builds BDB 4.3.0 from source). OS X 10.3 comes with python 2.3 installed, but the default copy of bsddb doesn’t seem to work. I downloaded the pybsddb for BDB 4.3 from Sourceforge (via link above), and built it. Because of the nonstandard install location used by buildall.sh, I had to pass an extra param (--berkeley-db) to setup.py. :

  %> cd /Users/jclark/ext
  %> tar -xzf bsddb3-4.3.0
  %> cd bsddb3-4.3.0
  %> python setup.py --berkeley-db=/Users/jclark/ext/dbxml-2.0.9/install/ build
  %> python test.py
  %> python setup.py install

With a working bsddb, It was time to build XML DB’s python bindings. As mentioned in the main README and in the dbxml/src/python README, the custom BDB/DBXML/etc install location requires an edit to dbxml/src/python/setup.py. I adjusted the paths thusly:

  if os.name == "posix":
    db_home = "/Users/jclark/ext/dbxml-2.0.9/install"
    xerces_home = db_home
    pathan_home = db_home
    xquery_home = db_home

Building and installing was straightforward:

  %> cd /Users/jclark/ext/dbxml-2.0.9/src/python
  %> python setup.py build
  %> python setup.py install

Time to test:

  %> cd /Users/jclark/ext/dbxml-2.0.9/dbxml/examples/python
  %> python examples.py test
  Traceback (most recent call last):
    File "examples.py", line 6, in ?
      from bsddb.db import *
    File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/
        python2.3/bsddb/__init__.py", line 40, in ?
      import _bsddb
  ImportError: No module named _bsddb

Uh-oh. However, a quick look inside examples.py reveals the solution. The import from bsddb.db applies when using the version of pybsddb that ships with python; when using a custom pybsddb build you must import from bsddb3. This involved commenting out one line and uncommenting another in examples.py. Let’s try again:

  %> python examples.py test
  Traceback (most recent call last):
    File "examples.py", line 7, in ?
      from dbxml import *
    File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/
        python2.3/site-packages/dbxml.py", line 5, in ?
      import _dbxml
  ImportError: Failure linking new module: libxerces-c.26.dylib: dyld: 
        python can't open library: libxerces-c.26.dylib  
        (No such file or directory, errno = 2)

Still no joy in Mudville. It’s at this point that I began to have trouble with how to proceed; my knowledge of *nix linking, loading, and such is limited. My rough understanding of dylibs is that they are dynamically linked libraries, a bit like Windows DLLs. The errors above (a couple of newlines were added for formatting) seem to indicate that the xerces-c lib can’t be located. I checked the output from building dbxml.py and confirmed that the correct (to my untrained eye) -l and -L flags were used to locate all of the required libs from /Users/jclark/ext/dbxml-2.0.9/install/libs.

My problem was, I didn’t really understand the dyld process. It was unclear to me whether dynamically linked libraries have the library location linked into the executable (as I suspected, but which didn’t explain the error) or if they rely on a search path (which probably defaults to /usr/lib or somesuch). In the end, man dyld proved somewhat enlightening. It turns out that there’s a slew of environment variables which can influence the locations searched for dylibs. It also turns out that otool -L can reveal information about linked libraries. Let’s see what we can learn (the two-step cd avoids a linebreak):

  %> cd /System/Library/Frameworks/Python.framework/Versions/2.3/lib/
  %> cd python2.3/site-packages
  %> otool -L _dbxml.so
    /System/Library/Frameworks/Python.framework/Versions/2.3/Python 
        (compatibility version 2.3.0, current version 2.3.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libdbxml-2.0.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libdb_cxx-4.3.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libxquery-1.0.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libpathan.3.dylib 
        (compatibility version 4.0.0, current version 4.1.0)
    libxerces-c.26.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /usr/lib/libSystem.B.dylib 
        (compatibility version 1.0.0, current version 71.1.1)

For some reason, libxerces-c was linked without path information, unlike all of the other libraries. If anyone knows why this happened, or how to avoid it, please email me or leave a comment. I couldn’t determine how to change this, so I set an evironmental variable to provide the correct search path:

  %> setenv DYLD_LIBRARY_PATH /Users/jclark/ext/dbxml-2.0.9/install/lib
  %> python examples.py test

Success!

As I said earlier, I’m a bit under-experienced with some of the details of building software on *nix systems. If anyone can provide any help with the xerces-c dylib issue, or any improvements to these directions, let me know; I’ll update this post with any additional information I find/receive.

The Ultimate iPod Accessory

With one thing and another, I never got around to posting about my pre-Christmas present. Having been very happy with my Powerbook for the past two years, I’ve been planning to buy a desktop Mac, sometime in January ‘05. Originally I had been looking at the PowerMacs, but after getting my hands on an iMac G5 at the Apple store, I began to have second thoughts.

Initially, I was reluctant to consider an iMac G5. As incredible as the machine is, I’ve always had a problem with computers that have integrated monitors. “What happenens if something breaks? It’s just like a TV with a built-in VCR,” I reasoned. Then one day, it occurred to me that I’d owned just such a machine for years - the Powerbook. After that, reason took over, and I forgot my dislike for integrated monitors (surely a mental holdover from my younger, pro-PC days).

A couple of weeks before Christmas, the need presented itself. My Powerbook does not have a Superdrive, but Sherri and I wanted to put together a DVD for her parents as a Christmas present. Last year, a Christmas display her family had visited for 40 years closed down, and we shot some video to remember it by. I could produce a video using iMovie, but without a Superdrive I had no way to put it on DVD.

In an act of ultimate sacrifice, I bought my new computer a month early. I bought the 20″ iMac G5 with the Bluetooth package. What a gorgeous computer. The screen is huge and looks fantastic. The entire design is sleek and elegant. The bluetooth keyboard was an excellent addition, although I’m not using the bluetooth mouse… no amount of love for Apple will ever drive me to use a one-button mouse.

This machine will also become the ‘primary’ computer in the home office. Sherri still has an XP laptop, and the kids still have a PC in the basemaent, but there’s now a Mac that everyone can use. Eventually, I intend to win them all back from the dark side. I’ve already created an account for each of them, and enabled fast user switching.

Of course, now I have to wonder if I’ll miss out on anything this Monday, when His Steveness announces all of the cool new whizbangs in his Macworld Expo Keynote. I’m not too woried about new hardware… this G5 is big enough and fast enough for now. What I’m concerned with is new software… if the much rumored iWork suite and iLife ‘05 start shipping with every new Mac on Monday, will there be any retroactive love for recent Mac buyers? I believe they did something similar last year when Panther shipped, but I don’t recall if that extended to iLife.

Guess we’ll see on Monday.

(p.s.: I’m certain Steve Jobs or Apple marketing or someone refered to the iMac g5 as “the ultimate accessory for the iPod” or something like that. I’ve searched all over the internets (several of them) and cannot find the quote. I’d be grateful to anyone who can provide a link. )

Subtraction

Jeremy at Alpha-Geek.com pointed today to Subtraction.com as a website design that stands out from the crowd (in a good way). He writes:

For some reason, I am really taken aback by the site. It’s so simplistic in design but so rich in all of the nuances.

Boy, do I agree. As I’ve noted before, I love clean, simple layouts (and do not hold up my own as a shining example). Subtraction is almost breath-takingly clean and stark, without feeling bare. There are so many nice things about this design.

I’ve been wanting to do a redesign for some time. As I first looked at Subtraction, I had a number of “I wish I’d thought of that” moments. With a design as distinctive as this, any thoughts of borrowing design elements are quickly suppressed by respect for the original. Take the links in the ‘Colophon’ section of the right column. They are text links with little pictures, so similar and yet so different to the ‘badges’ seen here and on so many other websites.

A couple of ideas which I saw were not completely new to me. I’ve actually been kicking around the idea of a black and white layout for a little while. Of course, if I do it now, it’ll feel like copying. The little image blocks at the beginning of many of the stories is another idea I’ve kicked around, but this implementation is much better than anything I’ve considered. The categorization scheme is also something I’ve been thinking alot about - Subtraction uses tags, like del.icio.us. I’ve already made plans to switch this site to a tag based system as soon as I figure out how I want to handle the permalinks.

Although the tag-category idea isn’t surprising, the four-column layout for displaying all of the categories is. As is the linkblog at the bottom of the page, which is more than just a link blog. Each item has a star-rating, and many of the comments are more extensive than the average linkblog. The layout is (like the rest of the design) understated and elegant.

Other details are subtle, but accumulate to great affect. The use of simple fonts, the font sizing, and the use of several shades of gray in addition to black and white form a whole greater than the sum of the parts. The use of horizontal lines also strikes me as stunning in a way I can’t quite put my finger on.

My only dislike is the use of done-to-death bright orange as the highlight color- in such a breakout design, it’s disappointing to see such a cliche. But this is a minor (and quickly forgettable) flaw, and every true masterpiece must have a single flaw.

Rediscovery

Having recieved an iPod for Christmas (thanks Sherri!), tonight I began updating my music library. For a number of reasons, many of my CDs are not loaded into iTunes, and so not available to my iPod. In addition, my CDs were scattered - some at work, some in my car, some here in my home office. Today I gatherered everything up at work and brought it all home.

Many are still missing. Among those that are not, I’ve found several things I haven’t seen (or listened to) for a long time. Rediscovering them has been great. I’m particularly pleased to have found my copy of the Police boxed set. Some of these songs I haven’t heard in years- like “So Lonely” from Outlandos d’Amour or “Invisible Sun” from Ghost in the Machine or so many others… Man, there’s a lot of good music in there.

Once I get the whole library up-to-date (as much as I can… I think a sizable part of my CD collection from high school/college is still back in Virginia somewhere), I need to start rating everything. Eventually I want to create smart playlists that play things I haven’t listened to in a while… more frequently for higher-rated stuff, less often for lower rated stuff. Put it all on shuffle and see what shows up.

Something else I rediscovered… I find it interesting how slow it feels to rip a stack of CDs. Even though the average import speed is north of 10x, which means 5 or 6 minute imports, I’ve been spoiled by broadband, fast processors and plentiful memory. Waiting just seems so old fashioned.

January Blogging Challenge

Uncle Roger has again laid down the gauntlet, establishing the January challenge for posts about resolutions:

…come up with an evaluation of last year’s goals and a new set of goals for this year.

This should not be a facile collection of cliches like “I will lose weight” or “I’m going to save some money” but carefully thought out, significant goals for your life. Don’t just list indefinite, non-specific platitudes, but specific, achievable goals. Include a plan for accomplishing each goal with concrete milestones and dates.

And:

…this challenge includes a review of your goals from last year (if any).

I’m pleased to say that I’m half finished with the challenge, as I’ve already reviewed last year’s resolutions. I’ll be posting this year’s by the end of the week(end).

These challenges have been alot of fun… I encourage you to join in this one. Be sure to link to Roger’s post, and comment over there (and here if you want). If you’ve already posted your resolutions, go link them to the challenge.

Googlebot, Update These

This post is for Googlebot. Go follow these links, and index them:

Thanks pal.

For the rest of you, who are probably wondering if I’ve hit my head (not that I remember), I’ll explain. While checking my referers briefly (whole other post on that topic forthcoming), I noticed what looked like a search engine hit for a topic I don’t normally post about, of an adult, or more likely a teenage male, nature. (I’m not a prude, but listing the search terms would defeat my purpose). A quick check showed that the site was a non-English search engine powered by Google.

It seems that Googlebot stopped by the day I was hit by over 2000 comment spams. Although I took the entire comment system offline to remove that crap as soon as I saw it, Googlebot must have indexed a few of the pages. Those pages linked above are the pages it indexed that day and has apparently not reindexed since. I like website traffic as much as the next blogger, but I’m really not in the market for search engine hits for animated attacks on women. I certainly don’t want to be the number #6 Google hit for that search (which I am, for the moment). The sooner Googlebot indexes the clean versions of the pages above, the better.

Also of interest, Googlebot found those pages via www.jclark.org, instead of jclark.org. They are the same, but I omit the www as it is unnecessary. Other people who link to me occaisionally use the www, which is why it is indexed both ways. Of course, it shouldn’t be indexed twice, so I’ll be adding a mod_rewrite rule to my .htaccess file to permanently redirect all www. URIs to their sub-domain-free equivalents. Just to be safe, however, I think I’ll wait until Googlebot reindexes the above links.

Site Browsing Stats, December 2004

Last month, I posted my browser stats for November 2004. It was so much fun I decided to do it again. No fancy graphs (yet), but maybe one of these days. :

Browser              Hits      %age
-------              ----    ------
FireFox             36593    40.1 %
Internet Explorer   28143    30.8 %
Safari               9464    10.3 %
Unknown              7236     7.9 %
Mozilla              3490     3.8 %
NetNewsWire          2453     2.6 %
Netscape             1106     1.2 %
Opera                 936     1.0 %
Konqueror             549     0.6 %
OmniWeb               462     0.5 %
Others                774     0.8 %

Of interest- while Firefox usage is up from 37.9% to 40.1%, IE usage is also (slightly) up, from 30.4% to 30.8%. Safari usage dropped just over 1%, and Camino disappeared. I guess Mac users just get it . However Windows users are still 59% of my December stats.

One caveat- while the above stats don’t include errors (like the 403’s I’m handing out to abusive bots and a few spamtastic IP addresses, they do include all the spam that isn’t IP banned- whether my comment filter catches it or not. Maybe I’ll work on that this month.

Resolution Wrap-up: 2004

Last January 1st, I decided to break with tradition and make a few resolutions. Last night at midnight, time ran out. So how did I do? Let’s check the score. I’ll also give some thoughts for the upcoming year’s resolutions, but that will be a separate post, after I think it out a bit.

Personal
  1. Spend a little extra time with my kids. I don’t think I do too badly at this now, but why not improve anyway?

    This one’s an unqualified success.

  2. Have at least one weekend getaway with my wife… and no children. We’ve been planning this forever, and never seem to get it done.

    Another success. Sherri and I spent a weekend at a B&B near Lancaster, PA last winter. It was wonderful; we’re hoping to do it again this year.

  3. Play more Chess (and hopefully improve).

    A qualified success. I definately played more in the first quarter of the year, and I think I was even starting to improve. Since that time, however I’ve played next to none.

  4. Write some fiction.

    Utter failure… never found the time or inspiration. A good goal to keep for this year, however.

Professional
  1. Finish every last piece of the Client Reporting project.

    This turned out not to even be plausible, as business needs changed. This was a poor choice of resolution, in the future I’ll keep any professional resolutions project-nuetral.

  2. Make the whole thing obsolete with a ground-up rewrite, using no VB (build directly in XML via Office 2003 native XML formats).

    This turned out not to be possible because Micro$oft once again over-promised and under-delivered; this time promising a native XML format for Excel and delivering a crippled XML format with no support for graphs.

jclark.org
  1. Add timestamps and notifications for comments.

    Success.

  2. New design. Something a bit less… understated.

    Failure. I began work on a new deisgn, but couldn’t decide what I wanted to do. I still want to redesign for this year.

  3. Hit the 300 posts milestone.

    70% success. My first post of 2004 was number 100, so I needed to write 200 posts. I managed to deliver 142- in spite of a lengthy blogging hiatus over the summer. Big thanks to dugh for the October Blogging Challenge which helped get me back on track.

  4. Re-organize the categories. Use permanent redirects for moved posts.

    Success. For 2005, I want to think about restructuring the blog… moving away from strict categories, and using a ‘tag’ system like del.icio.us uses. This may mean a move away from Blosxom. Also, whatever I do can’t break existing URIs. Still some planning to do.

  5. Add a second, topic specific blog. Potential topics are astronomy, chess, or writing.

    I cancelled this one. Having less time for any of my other interests, let alone less time to write for this blog, convinced me of that. Instead, for 2005, I want to start writing some more personal, daily-summary/thoughts posts in addition to the more topical stuff I usually post. More on that idea in another post.

All in all, not a terrible first attempt. Later today, or during the next few days, I’ll post my 2005 list.