XML is Just S-expressions with Pointy Brackets and Better Marketing

by Trenton Henry

Saturday, February 11, 2006

[Ok, I didn’t really blog this today. I actually wrote it a long time ago. It has been a “musing in gestation” for, well, like a long time...]

A long time ago I began hearing about markup languages like HTML, and VRML. The idea wasn't new to me, as I'd had some minimal experience with TEX by then. But when I actually got interested enough to examine these newer markup languages I was immediately left with the thought "This will never last; you can already do all of that better with s-expressions."

Well, eventually I encountered XML, which I largely ignored hoping it would fade away before I ever needed to learn it. Since I work (for money anyhow) almost exclusively with embedded systems, this seemed pretty reasonable. But, like the flu, XML infected pretty much everything it touched. It even became the default format for property lists on OSX, replacing the venerable old s-expression format. Well, that got my attention. I thought "This XML must be pretty good stuff if its bid for world domination has reached that far." So I looked into it to see what made it tick.

Wow. Its HTML on steroids. There is nothing new under the sun. But what makes XML so much better that the entire universe as fallen all over itself trying to adopt it? It does have a lot of useful accouterments, like namespaces, and schemas, XML-RPC, XSL, Soap, Xpath, etc. Ok, that's quite a bit to learn, but I decided to bite anyhow. Um, yeah.

Before I gave up, I tried to engage a number of my colleagues in discussions on XML versus s-expressions. I was mostly laughed off, though occasionally agreed with. But the overall reaction I received was essentially "get over it". No one cared.

Well, I decided to keep it to myself and get on with business as usual. Then I read Eric Raymond's wonderful book "The Art of Unix Programming". In that book he stated the following:

"Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot."

And I thought to myself "But why wouldn't I use it?" And then I thought to myself "Why haven't I been using it?" And so I embarked upon a project to actually use Lisp in a "real" project; I.e., as part of a product, or at least the development of a product.

At the time, I was using Rational Rose Real Time at work for the purpose of modeling embedded systems. These were tiny 8051 based systems, where 64K of code space was an astonishing embarrassment of riches. RoseRT wasn't designed to generate code for such tiny systems, much less to understand the dialect of C that we were compiling.

So I set out to extract the model from RoseRT, into my own proprietary model, and then generate "proper" C code from it. To do this, I wrote a script in Rose Basic that extracted the relevant portions of the model (components, classes, attributes, operations, state machines, states, events, transitions, etc) and stored them into a plain text, human readable file. As s-expressions.

(I also noted with some pleasure that the actual Rose "petal" files are stored as s-expressions. But they were dense, and there was no specification for them. I did try to mine the format, but gave up and focused on a useful subset. Time being money and all.)

I dusted off my trusty copy of Betz’s XLisp-1.4 and in a whirlwind of lambda induced coding euphoria I set to work. I wound up inventing my own programming language, to bring object orientation to deeply embedded firmware. I wrote specifications and manuals and code. Code to generate code, code to model code, code to just be code. Wow. It was a glorious orgy of self gratification. I was in peak form:)

I modified XLisp in C, adding all of the special primitives I needed. I wrote a Prolog-like theorem prover in XLisp. I wrote a parser in that proved language, which involved parsing most of C as well as the “my language” extensions. I used VIM because it had Lisp syntax highlighting. I was coding in Lisp again. I had returned to my roots and was speaking the Mother Tongue once more. I was home.

At some point towards the end of this Nerdvannah, Eric Raymond came to town to discuss his book at a Linux convention. I had remembered his comments on Lisp and thought I'd ask for more information. I was fortunate enough to get a seat on the front row, and to be called on for a couple questions. I asked things like "Given that C++ is generally considered to be insufficiently maneuverable in tight spaces, and assembly is not portable, what other languages may challenge C for embedded systems?" The answer was, of course, "None". (Having built a reverse polish lisp interpreter into an 8051 based product some years back, I was half hoping to hear something silly like "Why, Lisp, of course!"... Yeah, I know. Let it go.)

Then I asked "What can you do with XML that you couldn't already do with s-expressions?" The answer came back "Nothing. XML is just s-expressions with pointy brackets and better marketing". But I never got a chance to follow up. He moved on to other questions and never came back to me. Sigh. He did sign my copy of his book, though.

I went home and began the first draft of this article. But his response was so matter of fact; so cut and dry. It was as if he had said "Duh!". The implied "WHO CARES!?!" sort of bitch-slapped me. I dwelled on it too heavily, and lost inspiration for the article, and it gathered dust.

So anyhow, I finished the lispy project. That company dropped its Rose licenses, and moved on to new and hopefully better things. I moved on as well. And I began speaking in harsher, more guttural tongues again. C, C++, Perl. Sigh. I still pine for the fjords. But there is something to be said for cash in the bank.

I have continued playing with Lisp in my limited spare time. I'm porting XLisp-1.4 to be compilable on OSX with GCC. Old style function declarations. Way old style. (1.4, possibly 1.6, was the last version I am aware of before it went down the "Oooh! I'm Common Lisp too!” path...) I'm also slowly dabbling with "Scrisp", a lisp like scripting language that I can embed into other programs, or run from the command line. But that's another story. What was I on about to begin with? Oh yes... pointy braces.

[I actually finished the XLisp recompile. Perhaps I shall share...]

So, I felt all alone in the world, somehow sure that Lisp was my secret weapon, despite what the Luddites may have to say on the matter. But it was sort of a dirty little secret. Until 08/25/05. That was the day that Jack Ganssle came to my workplace for an in-house one day seminar I had arranged for all of our developers.

It was fantastic. Everyone was engaged. Jack was enthusiastic, even passionate, about teaching us to write better firmware faster. We all left that day with a great buzz on. I didn't even check my email until about 10 PM that evening because I wanted the high to last as long as possible.

Anyhow, during his presentation, Jack talked about a language called SPARK, which is a derivative of ADA. I have not done my due diligence on SPARK yet, but its on my to-do list. But that got several of us talking about sundry languages, and at some point I said something stupid like "Java, Perl, Python, and Ruby are all poser languages. Lisp is the pure Mother Tongue." Well, that started some lively discussions. But out of that came the fact that a colleague sent me a link to information on Paul Graham. www.paulgraham.com.

I began reading it and I found a few tidbits that made me, well, a bit weepy:) He agrees that Lisp is a "secret weapon" and has used it to achieve major victories. But more importantly, he states this:

"Running code at read-time lets users reprogram Lisp's syntax; running code at compile-time is the basis of macros; compiling at runtime is the basis of Lisp's use as an extension language in programs like Emacs; and reading at runtime enables programs to communicate using s-expressions, an idea recently reinvented as XML." [Emphasis mine.]

[He also hit on Raymond’s quote, it turns out, and wrote stuff about it that’s actually potentially useful.]

So, now I am going to ask a question that has been gnawing at me ever since I first heard of XML: "Why are America's corporations spending millions of dollars reinventing a technology that has been just fine for over 40 years?"

I think that there’s a real lack of knowledge about the history of computing. I think its because folks just don’t grok the fullness of the goodness that is already out there, waiting to be harvested. Lack of historical information. And NIH syndrome...

Everywhere I work I notice that most of the software developers don’t seem to have a very deep historical insight into computing. Don’t get me wrong... they are good crews. But there is a tendency to overcomplicate, and reinvent the wheel. Segway into Greenspun’s 10th rule...

[Wait a minute... am I seeing a pattern here? everywhere I work... overcomplication... Hmmm... No way. Just a coincidence. Don’t scare me like that...]

But, this is turning into a different topic altogether. Its just supposed to be a blog test. Perhaps I’ll get back to turning this into a proper musing...

[By the way, the Boston Museum photo is like a subtle link to historical-idity. You know, what’s missing from the heads of today’s software professionals... according my theory anyhow...]

[end]