Lisp Hackers: Peter Seibel

Peter Seibel has helped more people (including me) discover and become user of Lisp as probably noone else in the last decade with his Practical Common Lisp. Dan Weinreb, one of the founders of Symbolics and later Chief Architect at ITA Software, a succesfull Lisp startup sold to Google for around $1B in 2011, wrote, that their method of building a Lisp team was by hiring good developers and giving them PCL for two weeks, after which they could succesfully integrate under the mentorship or their senior Lisp people.

A few years after PCL Peter went on to write another fantastic programming book Coders at Work - here's my summary of it with the social network of Coders :)



Aside from being a writer he was and remains a polyglot programmer, interested in various aspects of our trade, about which he blogs occasionally. His code, presented in PCL, laid the foundation for a wide-spread CL-FAD library, which deals with filenames and directories (as the name implies), and more recently he created a Lisp documentation browser Manifest. Before Lisp Peter had worked a lot on Weblogic Java application server.

He's also active on twitter: @peterseibel

Tell us something interesting about yourself.

I’m a second generation Lisp programmer. My dad discovered Lisp when he was working at Merck in the 80s and ended up doing a big project to simulate a chemical plant in Lisp, taking over from some folks who had already been trying for quite a while using Fortran, and saving the day. Later he went to Bolt Beranek and Newman where he did more Lisp. So I grew up hearing about how great Lisp was and even getting to play around with some graphics programs on a Symbolics Lisp Machine.

I was also a childhood shareholder in Symbolics—I had a little money from some savings account that we had to close when we moved so my parents decided I should try investing. I bought Symbolics because my parents just had. Never saw that money again. As a result, for most of my life I thought my parents were these naive, clueless investors. Later I discovered that around that time they had also invested in Microsoft which, needless to say, they did okay with.

Oh, and something I learned recently: not only was Donald Knuth one of the subjects in my book Coders at Work, but he has read the whole thing himself and liked it. That makes me happy.

What's your job? Tell us about your organization.

A few months ago I started working half-time at Etsy. Etsy is a giant online marketplace for people selling handmade and vintage items and also craft supplies. I’m in the data group where we try to find clever ways to use data to improve the web site and the rest of the business.

Do you use Lisp at work? If yes, how you've made it happen? If not, why?

I always have a SLIME session going in Emacs for quick computations and sometimes I prototype things in Lisp or write code to experiment with different ideas. However, these days I’m as likely to do those things in Python because I can show my co-workers a sketch written in Python and expect them to understand it and I’m not sure I could do that with Lisp. But it makes me sad how slow CPython is compared to a native-compiling CL like SBCL. Usually that doesn’t matter but it is
annoying sometimes mostly because Python has no real excuse. The rest of my work is in some unholy mishmash of Scala, Ruby, Javascript, and PHP.

What brought you to Lisp? What holds you?

As I mentioned, I grew up hearing from my dad about this great language. I actually spent a lot of my early career trying to understand why Lisp wasn’t used more and exploring other languages pretty deeply to see how they were like and unlike Lisp. I played around with Lisp off and on until finally in 2003 I quit the startup I had been at for three years, which wasn’t going anywhere, with a plan to take a year off and really learn Common Lisp. Instead I ended up taking two years off and writing Practical Common Lisp.

At this point I use it for things when it makes sense to do so because I know it pretty well and most of my other language chops are kind of rusty. Though I’m sure my CL chops are rusty too, compared to when I had just finished PCL.

Did you ever develop a theory why Lisp isn’t used more?

Not one that is useful in the sense of helping it to be used more today. Mostly it seems to me to be the result of a series of historical accidents. You could argue that Lisp was too powerful too early and then got disrupted, in the Innovator’s Dilemma sense, by various Worse is Better languages, running on systems that eventually became dominant for perhaps unrelated reasons.

Every Lisper should read The UNIX-HATERS Handbook to better understand the relation between the Lisp and Unix cultures—Lisp is the older culture and back when the UNIX-HATERS Handbook was written, Unix machines were flaky and underpowered and held in the same contempt by Lisp geeks as Windows NT machines would be held by Unix geeks a few decades later. But for a variety of reasons people kept working on Unix and it got better.

And then it was in a better position than the Lisp culture to influence the way personal computing developed once micro computers arrived—while it would be a while before PCs were powerful enough to run a Unix-like OS, early on C was around to be adopted by PC programmers (including at Microsoft) once micros got powerful enough to not have to program everything in assembly. And from there, making things more Unix-like seemed like a good goal. Of course it would have been entirely possible to write a Lisp for even the earliest PCs that probably would have been as performant as the earliest Lisps running on IBM 704s and PDP-1s. My dad, back from his Lisp course at Symbolics, wrote a Lisp in BASIC on our original IBM PC. But by that point Lispers’ idea of Lisp was what ran on powerful Lisp machines, not something that could have run on a PDP-1.

The AI boom and bust played its role as well because after the bust Lisp’s reputation was so tainted by its failure to deliver on the over-promises of the Lisp/AI companies that even many AI researchers disassociated themselves from it. And throughout the 90s various languages adopted some of Lisp’s dynamic features, so folks who gravitated to that style of programming had somewhere else to go and then when the web sprang into prominence, those languages were well positioned to become the glue of the Internet.

That all said, I’m heartened that Lisp continues to not only be used but to attract new programmers. I don’t know if there will ever be a big Lisp revival that brings Lisp back into the mainstream. But even if there were, I’m pretty sure that there would be plenty of old-school Lispers who’d still be dissatisfied with how the revival turned out.

What's the most exciting use of Lisp you had?

I’m pretty proud of the tool chain I’ve built over the years while writing my two books and editing the magazine I tried to start, Code Quarterly. When I first started working on Practical Common Lisp I had some Perl scripts that I used to convert an ad-hoc light-weight text markup language into HTML. But after a little while of that I realized both that Jamie Zawinski was right about regexps and that of course I should be using Lisp if I was writing a book called Practical Common Lisp.

So I implemented a proper parser for a mostly-plain-text language that I uncreatively call Markup and backends that could generate HTML and PDF using cl-typesetting. When I was done writing and Apress wanted me to turn in Word files, I wrote an RTF backend so I could generate RTF files with all the Apress styles applied correctly. An Apress project manager later exclaimed over how “clean” the Word files I had turned had been. For editing Code Quarterly I continued to use Markup and wrote a prose diff tool that is pretty smart about when chunks of text get moved and edited a little bit.

What you dislike the most about Lisp?

I don’t know if “dislike” is the right term because the alternative has its own drawbacks. But I do sometimes miss the security of refactoring with more static checks. For instance, when I programmed in Java, there was nothing better than the feeling of knowing a method was private and therefore I didn’t have to look anywhere but in the one file where the method lived to see everywhere it could possibly be used. And in Common Lisp the possibilities for action at a distance are even worse than in some other dynamic languages because of the loose relation between symbols and the things they name. In practice that’s not actually a huge problem and some implementations provide package locks and so on, but it always makes me feel a bit uneasy to know that if I :use a package and then DEFUN a function with the name of an inherited symbol I’ve changed some code I really didn’t mean to.

From time to time I imagine a language that lets you write constraints on your code in the language yourself—kind of like macros but instead of extending the syntax your compiler understands, they would allow you to extend the set of things you could say about your code that the compiler would then understand. So you could say things like, “this function can only be called from other functions in this file” but also anything else about the static structure of your code. I’m not sure exactly what the API for saying those things would look like but I can imagine it being pretty useful, especially in larger projects with lots of programmers: you could establish certain rules about the overall structure of the system and have the compiler enforce them for you. But then if you want to do a big refactoring you could comment out various rules and move code around just like in a fully dynamic language. That’s just a crazy idea; anyone who’s crazy in the same way should feel free to take it and run with it and see if they get anywhere.

Among software projects you've participated in what's your favorite?

Probably my favorite software I ever wrote was a genetic algorithm I wrote in the two weeks before I started at Weblogic in 1998, in order to build up my Java chops. It played Go and eventually got to the point where it could beat a random player on a 5x5 board pretty much 100% of the time. One of these days I need to rewrite that system in Common Lisp and see if I can work up to a full-size board and tougher opponents than random. (During evolution the critters played against each other to get a Red Queen effect—I just played them against a random player to see how they were doing.)

Describe your workflow, give some productivity tips to fellow programmers

I’m not sure I’m so productive I should be giving anybody tips. When I’m writing new code I tend to work bottom up, building little bits that I can be confident in and then combining. This is obviously easy to do in a pretty informal way in Common Lisp. In other languages unit tests can be useful if you’re writing a bigger system though I’m often working on things for myself that are small enough I can get away with testing less formally. (I’m hopeful that something like Light Table will allow the easy of informal testing with the assurances of more strict testing—I’d love to have a development environment that keeps track of what tests go with what production code and shows them together and runs the appropriate tests automatically when I change the code.)

When I’m trying to understand someone else’s code I tend to find the best way is to refactor or even rewrite it. I start by just formatting it to be the way I like. Then I start changing names that seem unclear or poorly chosen. And then I start mucking with the structure. There’s nothing I like better than discovering a big chunk of dead code I can delete and not have to worry about understanding. Usually when I’m done with that I not only have a piece of code that I think is much better but I also can understand the original. That actually happened recently when I took Edi Weitz’s Hunchentoot web server and started stripping it down to create Toot (a basic web server) and Whistle (a more user friendly server built on top of Toot). In that case I also discarded the need for backward compatibility which allowed me to throw out lots of code. In that case I wasn’t going for a “better” piece of code so much as one that met my specific needs better.

If you had all the time in the world for a Lisp project, what would it be?

I should really get back to hacking on Toot and Whistle. I tried to structure things so that all the Hunchentoot functionality could be put back in a layer built on top of Toot—perhaps I should do that just to test whether my theory was right. On the other hand, I went down this path because the whole Hunchentoot API was too hard for me to understand. So maybe I should be getting Toot and Whistle stable and well-documented enough that someone else can take on the task of providing a Hunchentoot compatibility layer.

I’d also like to play around with my Go playing critters, reimplementing them in Lisp where I could take advantage of having a to-machine-code compiler available at run time.

PCL was the book, that opened the world of Lisp to me. I’ve also greatly enjoyed Coders at Work. So I’m looking forward for the next book you’d like to write. What would it be? :)

My current theory is that I’m going to write a book about statistics for programmers. Whenever I’ve tried to learn about statistics (which I’ve had to do, in earnest, for my new job) I find an impedance mismatch between the way I think and the way statisticians like to explain stuff. But I think if I was writing for programmers, then there are ways I could explain statistics that would be very clear to them at least. And I think there are lots of programmers who’d like to
understand statistics better and may have had difficulties similar to mine.

Discussion on Hacker News

submit