{"id":256,"date":"2005-01-10T07:37:00","date_gmt":"2005-01-10T07:37:00","guid":{"rendered":"http:\/\/jclark.org\/weblog\/Apple\/OSX\/dbxml2-py-osx.html"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-30T04:00:00","slug":"dbxml2-py-osx","status":"publish","type":"post","link":"https:\/\/jclark.org\/weblog\/2005\/01\/10\/dbxml2-py-osx\/","title":{"rendered":"HOWTO: Build DB XML 2.0 with Python Bindings on OS X"},"content":{"rendered":"

Sleepycat Software<\/a> recently released Berkeley DB XML 2.0<\/a>, a native XML database built on top of Berkley DB<\/a>. The major change from DB XML 1 to 2 is the addition of XPath 2<\/a>. and XQuery 1.0<\/a> support. Here’s what I did to build and install DB XML 2.0 along with python bindings on OS X 10.3.7. <\/p>\n

Having previously built DB XML 1.0 on OS X, I was pleased to see that all of the prerequisite libraries (Xerces-C, Pathan 2, Berkeley DB) are now included in the DB XML source package. After downloading the tarball from the website (see link above), extracting it and building everything was easy:\n%> cd \/Users\/jclark\/ext\n%> tar -xzf dbxml-2.0.9.tar.gz\n%> cd dbxml-2.0.9\n%> sh buildall.sh<\/p>\n

The buildall.sh<\/code> script creates an install<\/code> directory in the build root and installs all of the packages to this location. This gave me the following dir structure:<\/p>\n

  Users\n   + jclark\n      + ext\n         + dbxml-2.0.9\n            + install\n               + bin\n               + docs\n               + include\n               + lib<\/code><\/pre>\n
Next I wanted to build python support, which is not built by buildall.sh<\/code>.  The README cautions that the python bindings require the pybsddb package<\/a>, which ships with python 2.3 and later, but which must be compiled against the same version of Berkeley DB as XML DB (the XML DB package builds BDB 4.3.0 from source).  OS X 10.3 comes with python 2.3 installed, but the default copy of bsddb<\/code> doesn’t seem to work.  I downloaded the pybsddb for BDB 4.3 from Sourceforge (via link above), and built it.  Because of the nonstandard install location used by buildall.sh<\/code>, I had to pass an extra param (--berkeley-db<\/code>) to setup.py<\/code>. :<\/p>\n
  %> cd \/Users\/jclark\/ext\n  %> tar -xzf bsddb3-4.3.0\n  %> cd bsddb3-4.3.0\n  %> python setup.py --berkeley-db=\/Users\/jclark\/ext\/dbxml-2.0.9\/install\/ build\n  %> python test.py\n  %> python setup.py install<\/code><\/pre>\n
With a working bsddb<\/code>, It was time to build XML DB’s python bindings.  As mentioned in the main README and in the dbxml\/src\/python<\/code> README, the custom BDB\/DBXML\/etc install location requires an edit to dbxml\/src\/python\/setup.py<\/code>. I adjusted the paths thusly:<\/p>\n
  if os.name == "posix":\n    db_home = "\/Users\/jclark\/ext\/dbxml-2.0.9\/install"\n    xerces_home = db_home\n    pathan_home = db_home\n    xquery_home = db_home<\/code><\/pre>\n
Building and installing was straightforward:<\/p>\n
  %> cd \/Users\/jclark\/ext\/dbxml-2.0.9\/src\/python\n  %> python setup.py build\n  %> python setup.py install<\/code><\/pre>\n
Time to test:<\/p>\n
  %> cd \/Users\/jclark\/ext\/dbxml-2.0.9\/dbxml\/examples\/python\n  %> python examples.py test\n  Traceback (most recent call last):\n    File "examples.py", line 6, in ?\n      from bsddb.db import *\n    File "\/System\/Library\/Frameworks\/Python.framework\/Versions\/2.3\/lib\/\n        python2.3\/bsddb\/__init__.py", line 40, in ?\n      import _bsddb\n  ImportError: No module named _bsddb<\/code><\/pre>\n
Uh-oh.  However, a quick look inside examples.py<\/code> reveals the solution.  The import from bsddb.db<\/code> applies when using the version of pybsddb that ships with python; when using a custom pybsddb build you must import from bsddb3<\/code>.  This involved commenting out one line and uncommenting another in examples.py<\/code>.  Let’s try again:<\/p>\n
  %> python examples.py test\n  Traceback (most recent call last):\n    File "examples.py", line 7, in ?\n      from dbxml import *\n    File "\/System\/Library\/Frameworks\/Python.framework\/Versions\/2.3\/lib\/\n        python2.3\/site-packages\/dbxml.py", line 5, in ?\n      import _dbxml\n  ImportError: Failure linking new module: libxerces-c.26.dylib: dyld: \n        python can't open library: libxerces-c.26.dylib  \n        (No such file or directory, errno = 2)<\/code><\/pre>\n
Still no joy in Mudville.  It’s at this point that I began to have trouble with how to proceed; my knowledge of *nix linking, loading, and such is limited.  My rough understanding of dylibs is that they are dynamically linked libraries, a bit like Windows DLLs.  The errors above (a couple of newlines were added for formatting) seem to indicate that the xerces-c<\/code> lib can’t be located.  I checked the output from building dbxml.py<\/code> and confirmed that the correct (to my untrained eye) -l<\/code>\nand -L<\/code> flags were used to locate all of the required libs from \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/libs<\/code>.  <\/p>\n
My problem was, I didn’t really understand the dyld process.  It was unclear to me whether dynamically linked libraries have the library location linked into the executable (as I suspected, but which didn’t explain the error) or if they rely on a search path (which probably defaults to \/usr\/lib<\/code> or somesuch).  In the end, man dyld<\/code> proved somewhat enlightening.  It turns out that there’s a slew of environment variables which can influence the locations searched for dylibs.  It also turns out that otool -L<\/code> can reveal information about linked libraries.  Let’s see what we can learn (the two-step cd<\/code> avoids a linebreak):<\/p>\n
  %> cd \/System\/Library\/Frameworks\/Python.framework\/Versions\/2.3\/lib\/\n  %> cd python2.3\/site-packages\n  %> otool -L _dbxml.so\n    \/System\/Library\/Frameworks\/Python.framework\/Versions\/2.3\/Python \n        (compatibility version 2.3.0, current version 2.3.0)\n    \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/lib\/libdbxml-2.0.dylib \n        (compatibility version 0.0.0, current version 0.0.0)\n    \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/lib\/libdb_cxx-4.3.dylib \n        (compatibility version 0.0.0, current version 0.0.0)\n    \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/lib\/libxquery-1.0.dylib \n        (compatibility version 0.0.0, current version 0.0.0)\n    \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/lib\/libpathan.3.dylib \n        (compatibility version 4.0.0, current version 4.1.0)\n    libxerces-c.26.dylib \n        (compatibility version 0.0.0, current version 0.0.0)\n    \/usr\/lib\/libSystem.B.dylib \n        (compatibility version 1.0.0, current version 71.1.1)<\/code><\/pre>\n
For some reason, libxerces-c<\/code> was linked without path information, unlike all of the other libraries.  If anyone knows why this happened, or how to avoid it, please email me<\/a> or leave a comment.  I couldn’t determine how to change this, so I set an evironmental variable to provide the correct search path:<\/p>\n
  %> setenv DYLD_LIBRARY_PATH \/Users\/jclark\/ext\/dbxml-2.0.9\/install\/lib\n  %> python examples.py test<\/code><\/pre>\n
Success!  <\/p>\n
As I said earlier, I’m a bit under-experienced with some of the details of building software on *nix systems.  If anyone can provide any help with the xerces-c<\/code> dylib issue, or any improvements to these directions, let me know; I’ll update this post with any additional information I find\/receive.<\/p>","protected":false},"excerpt":{"rendered":"
Sleepycat Software recently released Berkeley DB XML 2.0, a native XML database built on top of Berkley DB. The major change from DB XML 1 to 2 is the addition of XPath 2. and XQuery 1.0 support. Here’s what I did to build and install DB XML 2.0 along with python bindings on OS X […]<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[],"class_list":["post-256","post","type-post","status-publish","format-standard","hentry","category-osx"],"_links":{"self":[{"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/posts\/256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/comments?post=256"}],"version-history":[{"count":0,"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/posts\/256\/revisions"}],"wp:attachment":[{"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/media?parent=256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/categories?post=256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jclark.org\/weblog\/wp-json\/wp\/v2\/tags?post=256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}