Archive for the 'OSX' Category

Note: I've reorganized this site to use tags; the category archive remains to support old links. Only posts prior to April, 2006 are categorized. Tag Archive »

HOWTO (Build and) Install Carbon XEmacs on OS X

Update: added instructions for installing elisp packages after building.

There are as many Emacsen as there are grains of sand in the desert. For various reason (largely habit) I prefer XEmacs to Emacs, when given a choice- even when in text mode (xemacs -nw for the uninitiated; I usually alias this to emacs). And while I love a good command prompt as much as the next geek, I do own a Mac, and sometimes I want a more Mac-like experience.

Carbon XEmacs is a derivative of XEmacs, developed by Andrew Choi. Carbon XEmacs does not require X11, but instead uses the OS X Carbon libraries for its GUI display. Of the various OS X-specific Emacsen I have tried, it is my current favorite. At the time of this writing, the current version of Carbon XEmacs is Beta 4, and is available as a series of patches against the XEmacs source. This HOWTO describes how to build Carbon XEmacs Beta 4 for Mac OS X 10.4.4, and attempts to describe more generally how to build the latest Carbon XEmacs for any OS X 10.3+. Since many Mac users (myself included) are spoiled by the many binary packages and package managers (such as Fink and Darwin Ports) available for OS X, this HOWTO goes into some detail about how to download, patch, and build the source. If you already know this stuff, feel free to skim.

(If you’ve never done so before, you’ll need to install the XCode developer tools from your OS X install CD/DVD. If you’ve lost these, it’s available from Apple’s Developer site. It’s basically a click-and-forget installer package.)

Let’s do this

To begin, you will need to download the XEmacs source for the same XEmacs version that your Carbon XEmacs patches apply to. At the time of the writing, the latest Carbon XEmacs is Beta 4, which is supplied as a diff (set of patches) to xemacs-21.5.23. Visit the XEmacs.org download page to locate a suitable mirror, and download the source package for the correct version. As of this writing, all 21.5.x releases are considered Beta releases of XEmacs, so I chose the beta download page, and a local mirror therein.

When you arrive at the mirror, it will be a directory on an FTP server, full of files. look for a tarball- a file ending with .tar.gz– matching the xemacs version you need. In this case, I’ll be downloading xemacs-21.5.23.tar.gz. You will also see files with similar names ending in .patch.gz, .asc, and .md5. You will not need these (unless you need them- in which case you already know you need them. The rest of us will soldier on).

After downloading the file, open Terminal and change to the directory where the .tar.gz file was downloaded. I downloaded mine to my ~/ext directory:

cd ~/ext

If you don’t know where your downloads go, they probably go to your desktop; try cd ~/Desktop (capitalization counts). Now we’ll extract the source code:

tar xzf xemacs-21.5.23.tar.gz

Download the diff file from the Carbon XEmacs page to the same directory where you downloaded the xemacs source. Uncompress the diff file:

bunzip2 xemacs-21.5.23-carbon-b4.diff.bz2

And patch the original sources:

patch -p0 < xemacs-21.5.23-carbon-b4.diff

An aside: this is some pretty cool stuff if you’re not familiar with diffing and patching; the diff file is just a list of the changes between the original and Andrew Choi’s version, patch applies these changes to our original files, resulting in files that match Choi’s modified source. For the curious, -p0 (essentially) tells patch that it doesn’t have to adjust the paths; that’s why we made sure we put the patch in the same directory as the original zipped sources.

Now, time to make the donuts, er, software:

cd xemacs-21.5.23/carbon
sh build-app.sh

And now, the waiting. The hardest part, if we are to believe Tom Petty. This will take a while. On my 1.8 GHz iMac G5 (1.5GB RAM), it took around 7 minutes or so; it would probably take twice as long on my old G4 Powerbook.

When it’s done, you’ll have a shiny new XEmacs.app in the current directory (carbon). Unfortunately, your shiny new XEmacs.app doesn’t do a whole lot- none of the lisp packages that provide a significant portion of XEmacs’ features are included in the source code package. Instead, you’ll need to download them separately. The easiest way is to download the “xemacs-sumo” package, which contains all of the official packages in a single download. For multibyte character set support, you’ll also want the “xemacs-mule-sumo” package (mule is the emacs/xemacs multibyte subsystem).

From the XEmacs.org download page, find the link to download Official Packages, and select a mirror. You’ll end up in an FTP directory, as before. Download the following packages into the same directory as your other downloads:

xemacs-sumo.tar.gz
xemacs-mule-sumo.tar.gz

The packages can live a couple of different location, but I prefer to keep the packages directly inside the XEmacs.app application bundle. To unpack the tarballs to the right place run the following (remember to change the path if your downloads aren’t in ~/ext):

cd ~/ext/xemacs-21.5.23/carbon/XEmacs.app/Contents/Resources/lib/xemacs
tar xzf ~/ext/xemacs-sumo.tar.gz 
tar xzf ~/ext/xemacs-mule-sumo.tar.gz

Your XEmacs.app is now fully loaded. To move this to your Application directory, you can drag-n-drop in Finder, or just:

cd ~/ext/xemacs-21.5.23/carbon
mv XEmacs.app/ /Applications/

Just double click the teddybear icon to launch. If you want to see it run in a terminal window, use this (assuming you moved it to your Applications folder):

/Applications/XEmacs.app/Contents/MacOS/XEmacs -nw

As additional (Carbon) patches against the same base version of xemacs are released, you can download and apply them in the same manner, so keep the directory around. Remember to re-build the app (and copy it to Applications) after each patching. If a new patch-bundle is release against a newer XEmacs version, just dump the old source code directory and start over from the top.

Backing up a Windows Laptop with OS X

Update 9/10/2006: I’ve improved this procedure, and removed the need for NFS. See HOWTO Backup an Entire Windows Drive with OS X and Ubuntu for details.

My wife’s XP laptop continues to get slower. She’s had it for about 2 1/2 years, and it’s still running the original XP install, so no surprise that performance is lousy. I’ve considered dropping a desktop Linux install on it, but she has a couple of apps that still require Windows. For now, I decided to format her hard drive and revert it to factory condition with the restore CDs that came with the machine.

So now I needed to back up her machine. It doesn’t support Firewire or USB2, so external drives were out. In the past, I’ve always copied alot of files over the network, but this is time consuming, and usually error prone- If windows decides it can’t copy a file, the whole copy operation stops, and you have to figure out what has copied and what hasn’t.

I wanted something similar to Ghost. Ghost is a PC backup solution the PC admin folks at work used to use. It’s now a Symantec product, but I don’t think it was at the time. The original was great- a single floppy (we used floppies back then) would boot, connect to the network, and copy the entire hard drive to an image file on a network server. Modern versions allow you to grab files from inside the image; I don’t recall if the original did. This is a feature I need; I’m not restoring the whole image to a drive.

I tried to find an open source alternative (the current Symantec Ghost is overpriced, and I don’t trust Symantec software at all) without much luck. I found Partition Image for Linux, but the images can’t be opened for access to individual files. While researching, I came up with another idea… make a direct copy of the windows partition from a Linux Live CD (such as Knoppix) using dd. I figured the image might be mountable, just as you can mount an ISO CD image file.

A word about efficiency: This method copies the entire partion, empty blocks along with the rest. So, copying Sherri’s 27G partion would result in a 27G image file. However, I have the space on the iMac, the backup is only temporary, and I wanted to be sure I had everything. Seemed worth a shot.

In order to make the backup over the network, I needed to share a directory on the iMac as an NFS share, since I’d be connecting from a Linux Live CD. OS X supports this, but not via a nice little applet like with Windows Sharing. You need to fool around with netinfo, which I dislike, and run several daemons. Since this is only for temporary use, I decided not to get my hands too dirty, and found a shareware utility called NFSManager to handle the details for me. Once I had created an NFS share, it was time to use it.

I booted Sherri’s laptop using a Knoppix LiveCD, opened a shell, and mounted the NFS share (/Users/jclark/Netmount on the iMac):

sudo su
mdkir /mnt/mac
mount -t nfs 192.168.1.105:/Users/jclark/Netmount /mnt/mac

The first line makes me root for the ensuing commands. I then make a mountpoint, and mount the NFS share to the mountpoint. Interestingly, when I ran the mount command, the command appear to hang, but opening another terminal showed that the mount worked.

The laptop only had one drive (C:) so I suspected that I only needed to backup /dev/hda1, but I checked it with QtParted just to be safe. QtParted is a GUI shell around GNU parted, and is accessible from the Knoppix start menu. I would have used parted from the command line, but couldn’t find it in the Knoppix install.

Once I had confirmed what I needed to backup, making the image was simple:

dd if=/dev/hda1 of=/mnt/mac/laptop_drive

The transfer rate was about 10G/hr, which isn’t too bad I suppose. When it was finally complete, I tried to mount it under OS X using the mount command, with no success. A little while later I realized I hadn’t specified to mount it via loopback (in other words, treat a file like a drive). After a few minutes trying to figure how to do this in OS X, I got lazy and Googled. One of my hits suggested an idea for opening a floppy image that seemed too good to be true, but I tried it anyway.

I renamed the image file, adding .img to the name, and then double clicked the icon in Finder. A few seconds later the drive image was mounted, and I had access to the entire drive backup. Very cool.

Now all I have to do is restore her factory drive image, clean off all the crap it came pre-loaded with, patch the crap out of XP, reinstall her software, and restore all of here files.

Maybe I should just buy her an iBook….

OS X and Multi-button Mice – A Question

As stated recently, I’ve been looking for a 5 button bluetooth mouse to use with my iMac. One of the contenders has been the Microsoft Intellimouse Explorer for Bluetooth, but I’ve resisted buying it because I’m not sure how I like the positioning of buttons 4 and 5, and due to the asymetrical shape. Also, I didn’t really want to pay for the bluetooth adapter included with every MS Bluetooth mouse, since my iMac has Bluetooth built in.

A neighbor offered me his MS bluetooth keyboard and mouse kit, which he wasn’t using, to try out. I left the keyboard and bluetooth adapter in the bag, but hooked up the mouse. After installing the latest bluetooth firmware update and bluetooth software update from Apple, the mouse seems to be working reasonably well. However, OS X is not a support OS for Microsoft’s Bluetooth offereings, and the Intellipoint 5.1 for OS X software I’ve been using won’t recognize the mouse. This leaves me with no way to map buttons 3, 4, and 5 to any type of functionality.

The OS can recognize these buttons, however. I can map them to Expose functions in the Expose preference panel. However, I can’t map them to keystrokes (as the Intellimouse software did), and other applications (most notably Firefox) do not recognize the native button clicks or allow a way to map them. I’d like to map button four to Firefox’s Back button (aka Cmd+Left Arrow) and Button 3 (wheel-click) to the Firefox cmd Open Link in New Tab (Cmd+Mouse Click).

My primary need is for Firefox; I’ve looked for Extentions that would enable this functionality, but all I’ve found are extentions to enable mouse gestures, a feature I’m not fond of. I also stumbled across something call USB Overdrive, and OS X app to enable extra buttons on a USB mouse. Unfortunately, it’s designed of OS X 10.1, hasn’t been updated in over three years, and states that Bluetooth support would appear “in the next version”.

So, does anyone know how to generically map buttons 3,4, 5 in OS X, or a way to make Firefox recognize these buttons? Until I can configure this mouse the way I want it, It’s not going to be of much use.

HOWTO: Build DB XML 2.0 with Python Bindings on OS X

Sleepycat Software recently released Berkeley DB XML 2.0, a native XML database built on top of Berkley DB. The major change from DB XML 1 to 2 is the addition of XPath 2. and XQuery 1.0 support. Here’s what I did to build and install DB XML 2.0 along with python bindings on OS X 10.3.7.

Having previously built DB XML 1.0 on OS X, I was pleased to see that all of the prerequisite libraries (Xerces-C, Pathan 2, Berkeley DB) are now included in the DB XML source package. After downloading the tarball from the website (see link above), extracting it and building everything was easy: %> cd /Users/jclark/ext %> tar -xzf dbxml-2.0.9.tar.gz %> cd dbxml-2.0.9 %> sh buildall.sh

The buildall.sh script creates an install directory in the build root and installs all of the packages to this location. This gave me the following dir structure:

  Users
   + jclark
      + ext
         + dbxml-2.0.9
            + install
               + bin
               + docs
               + include
               + lib

Next I wanted to build python support, which is not built by buildall.sh. The README cautions that the python bindings require the pybsddb package, which ships with python 2.3 and later, but which must be compiled against the same version of Berkeley DB as XML DB (the XML DB package builds BDB 4.3.0 from source). OS X 10.3 comes with python 2.3 installed, but the default copy of bsddb doesn’t seem to work. I downloaded the pybsddb for BDB 4.3 from Sourceforge (via link above), and built it. Because of the nonstandard install location used by buildall.sh, I had to pass an extra param (--berkeley-db) to setup.py. :

  %> cd /Users/jclark/ext
  %> tar -xzf bsddb3-4.3.0
  %> cd bsddb3-4.3.0
  %> python setup.py --berkeley-db=/Users/jclark/ext/dbxml-2.0.9/install/ build
  %> python test.py
  %> python setup.py install

With a working bsddb, It was time to build XML DB’s python bindings. As mentioned in the main README and in the dbxml/src/python README, the custom BDB/DBXML/etc install location requires an edit to dbxml/src/python/setup.py. I adjusted the paths thusly:

  if os.name == "posix":
    db_home = "/Users/jclark/ext/dbxml-2.0.9/install"
    xerces_home = db_home
    pathan_home = db_home
    xquery_home = db_home

Building and installing was straightforward:

  %> cd /Users/jclark/ext/dbxml-2.0.9/src/python
  %> python setup.py build
  %> python setup.py install

Time to test:

  %> cd /Users/jclark/ext/dbxml-2.0.9/dbxml/examples/python
  %> python examples.py test
  Traceback (most recent call last):
    File "examples.py", line 6, in ?
      from bsddb.db import *
    File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/
        python2.3/bsddb/__init__.py", line 40, in ?
      import _bsddb
  ImportError: No module named _bsddb

Uh-oh. However, a quick look inside examples.py reveals the solution. The import from bsddb.db applies when using the version of pybsddb that ships with python; when using a custom pybsddb build you must import from bsddb3. This involved commenting out one line and uncommenting another in examples.py. Let’s try again:

  %> python examples.py test
  Traceback (most recent call last):
    File "examples.py", line 7, in ?
      from dbxml import *
    File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/
        python2.3/site-packages/dbxml.py", line 5, in ?
      import _dbxml
  ImportError: Failure linking new module: libxerces-c.26.dylib: dyld: 
        python can't open library: libxerces-c.26.dylib  
        (No such file or directory, errno = 2)

Still no joy in Mudville. It’s at this point that I began to have trouble with how to proceed; my knowledge of *nix linking, loading, and such is limited. My rough understanding of dylibs is that they are dynamically linked libraries, a bit like Windows DLLs. The errors above (a couple of newlines were added for formatting) seem to indicate that the xerces-c lib can’t be located. I checked the output from building dbxml.py and confirmed that the correct (to my untrained eye) -l and -L flags were used to locate all of the required libs from /Users/jclark/ext/dbxml-2.0.9/install/libs.

My problem was, I didn’t really understand the dyld process. It was unclear to me whether dynamically linked libraries have the library location linked into the executable (as I suspected, but which didn’t explain the error) or if they rely on a search path (which probably defaults to /usr/lib or somesuch). In the end, man dyld proved somewhat enlightening. It turns out that there’s a slew of environment variables which can influence the locations searched for dylibs. It also turns out that otool -L can reveal information about linked libraries. Let’s see what we can learn (the two-step cd avoids a linebreak):

  %> cd /System/Library/Frameworks/Python.framework/Versions/2.3/lib/
  %> cd python2.3/site-packages
  %> otool -L _dbxml.so
    /System/Library/Frameworks/Python.framework/Versions/2.3/Python 
        (compatibility version 2.3.0, current version 2.3.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libdbxml-2.0.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libdb_cxx-4.3.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libxquery-1.0.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /Users/jclark/ext/dbxml-2.0.9/install/lib/libpathan.3.dylib 
        (compatibility version 4.0.0, current version 4.1.0)
    libxerces-c.26.dylib 
        (compatibility version 0.0.0, current version 0.0.0)
    /usr/lib/libSystem.B.dylib 
        (compatibility version 1.0.0, current version 71.1.1)

For some reason, libxerces-c was linked without path information, unlike all of the other libraries. If anyone knows why this happened, or how to avoid it, please email me or leave a comment. I couldn’t determine how to change this, so I set an evironmental variable to provide the correct search path:

  %> setenv DYLD_LIBRARY_PATH /Users/jclark/ext/dbxml-2.0.9/install/lib
  %> python examples.py test

Success!

As I said earlier, I’m a bit under-experienced with some of the details of building software on *nix systems. If anyone can provide any help with the xerces-c dylib issue, or any improvements to these directions, let me know; I’ll update this post with any additional information I find/receive.

OS X Users: Get This Now.

Everyone else, go buy a Mac.

Michael McCracken made the world a better place back in February, and I only just found out.

Incremental Search, or i-search, is the magic that powers Firefox‘s in-page search. It’s been around for years in emacs. If you haven’t seen it (why aren’t you using Firefox?), the search takes place in real time, as you type. I miss it in other apps… but not any more.

Michael’s I-Search plugin for NSTextView is a library (a bundle in Cocoa-speak) that adds i-search to all NSTextViews – which is nearly every bit of text you see in an OS X Cocoa application. You just drop the bundle and a keybindings file into a couple of special folders in ~/Library and you’re all set. TextEdit, XCode, and SubEthaEdit all have i-search now.

If you only know i-searching from Firefox, you should note that the I-Search Plugin uses emacs keybindings, not Firefox’s. To begin searching, use ctrl-s. To see the next match, press ctrl-s again (why does Firefox use a second keycombo for this?). To search backwards, use ctrl-r. To exit i-search and return your cursor to it’s pre-search location, press ctrl-g. To exit i-search and keep the current match selected, press return (or esc). Having pressed return or esc, you can return to your pre-search location with ctrl-x ctrl-x (yes, do it twice). You can even make an edit first. In emacs terms, i-search updates the mark before searching. Since Cocoa text editing supports many emacs features and keybindings, it works as expected. (If you don’t know what the last two sentences mean, don’t worry about it.)