Common Lisp's BLOCK / RETURN-FROM and UNWIND-PROTECT

I was just chatting with Ben Titzer on Twitter about control flow in his Virgil language (which is cool and you should definitely check out), when I felt the need to once more promote how Common Lisp does non-local control flow (stuff like returning early from a function or breaking from a loop), because I think it's a very nice solution.

So in Common Lisp we have BLOCK / RETURN-FROM (which work as a pair) and UNWIND-PROTECT.

BLOCK / RETURN-FROM

BLOCK and RETURN-FROM effectively offer the same functionality as C's setjmp/longjmp -- non-local exits -- but nicely wrapped as we expect it in a lexically-scoped, expression-oriented language.

BLOCK / RETURN-FROM lets you do:

• Early returns from functions or arbitrary code blocks
• Breaking from loops, including any number of nested loops
• Continuing in loops, including any number of nested loops
• Even arbitrary GOTOs in a code block (with some macrology & trampolining, see Baker's TAGBODY)

(block name forms*) lexically binds name within the forms as a non-local exit from which you can return a value with (return-from name value). Just (return-from name) without a value uses nil as the value.

A BLOCK without any RETURN-FROM just returns the last value: 

(block b 1 2 3) returns 3.

This prints 1 and returns 2:

(block b (print 1) (return-from b 2) (print 3))

You can have any number of nested blocks:

(block b1 ... (block b2 ... (return-from b1) ...) ...)

To do an early return from a function, place a block at its beginning:

(defun foo ()

    (block b 

        ...

        (return-from b)

        ...))

(Common Lisp automatically places an anonymous block around every function body, so you don't need to do this in practice, but my hobby Lisp doesn't, and I'm using this explicit approach, and I like it.)

To break from a loop, place a block around it:

(block break

    (loop

        ...

        (return-from break)

        ...))

To continue in a loop, place a block inside it:

(loop

    (block continue

        ...

        (return-from continue)

        ...))

You can have multiple nested loops, like in Java:

(block break-outer

    (loop

        (block break-inner

            (loop

                ...

                (return-from break-inner)

                ...))))

UNWIND-PROTECT

UNWIND-PROTECT is effectively a try/finally block (without a catch).

(unwind-protect protected-form cleanup-forms*) evaluates the protected form, and regardless of whether it returns normally or does a non-local exit with RETURN-FROM, the cleanup forms are evaluated.

(unwind-protect (foo)

   (bar)

   (quux))

is analogous to

try {

   return foo();

} finally {

   bar();

   quux();

}

Both of the following expressions print 2 and return 1:

(unwind-protect 1

   (print 2))

(block exit

   (unwind-protect (return-from exit 1)

      (print 2)))

Conclusion

Common Lisp's BLOCK / RETURN-FROM and UNWIND-PROTECT offer a minimalistic and expressive system for non-local control flow.

(Lobsters, HN)

What Makes Lisp Great

Faré: What Makes Lisp Great:

[U]nlike most other languages, Lisp is not an arbitrary design based on the whims of some author, but it is a discovery produced by a long evolution and a deep tradition. It has seen several revolutions: transition from M-expression to S-expression, from EVALQUOTE to EVAL or from dynamic binding to static binding, introduction of data-structures, of object-oriented programming and of a meta-object protocol, the invention and latter standardization of many many individual features. Many dialects of Lisp also introduce their own revolutions, such as continuations and single name space in Scheme, the graphical user interface and dreaded DWIM features in InterLISP, etc. If anything remains among all these revolutions and variants, it is the ability to evolve. Lisp is a language fit for evolution -- not just by a small group of wizards who control the language implementation, but by every single programmer who uses the language. And this wasn't a conscious intended part of the original design: this was the great discovery of Lisp. (My emphasis.)

Now I understand better why I'm wary of Clojure: Clojure is big on revolution and small on evolution. And in a revolution, people get killed. Or, in Clojure's case, language features that have evolved over decades. Maybe devolution is the correct term.

The Lisp Ethos of Total Dynamicity

All true Lisps have a very important feature: you can enter any expression any time. (Subject to some static rules of course. E.g. you can't use a GO outside of an enclosing TAGBODY.) Heck, you can redefine a class while you're in the debugger that's waiting for you to continue with a restart.

So, I was quite shocked to learn that Python, heralded as a dynamic language, doesn't allow you to redefine classes. Of course, redefining classes isn't something you do all the time, but in Lisp implementations, supporting stuff like this is simply a question of honor.

Thank god Slava is kicking everybody's asses left and right outside of Lisp proper. I hope this will lead other dynamic language implementors to adopt the same ethos.

In my own Lisp implementation adventures, I was (of course!) seduced to think that maybe such total dynamicity could be sacrificed. After all, I can just restart the app I wrote to have the changes take effect. But in the end, it just wouldn't be a true Lisp. And it would add a load of bollocks to the language and its specification.

No New Ideas, Please

Nothing bothers me more than a flawed language feature, for which there's a well known, working solution in another language. Really, I think PL design is so hard, that if you design something new (like some abstraction or concept), there's a 98% chance that it will suck. Hard. So hard.

So, one of my goals with my new Lisp is to faithfully reuse large, modular parts of functionality developed by others, and proven in the field. Preferably for decades. In a way, this changes PL design from invention to curation, and I'm very happy with it.

Here's what Ell is made of:

• evaluation and expansion: R6RS Scheme (but as a Lisp-2)
• control flow: Common Lisp
  
• micromodule system: Chez Scheme
• compilation and phase separation: PLT Scheme
• macros: SRFI 72
• object system: proper subset of CLOS
• inline C: Goo
• condition system: Dylan
• dynamic variables: ISLISP
  

All of these parts are copied, in what I hope is a faithful way, that exactly preserves the original design, or a subset of it. That's the goal, at least. Of course there's some need for editing these features, so that they fit well together, but that shouldn't change the original intent and abstractions.

Like Peter van Roy, I think that programming-in-the-small is a solved problem. Really, innovating in language design microfeatures, like say, some new way to pass parameters, is so boring, and will with great likelihood make the language landscape even more trite than it is now. Much better to reuse something that already exists, and adapt it faithfully.

All of this applies only, of course, if you're building a language that follows some common plan, like say "object-oriented dynamic language". In that case, please reuse. If you're doing something totally new and/or crazy (like, say, a functional reactive PL), you're of course free to invent what you want.

Next Lisps

I'm a sucker for articles mentioning "the next Lisp". I just read Mark Tarver's, and I thought I'd jot down some quick notes. Not to be taken too seriously, but still.

Quotes are from The Next Lisp: Back to the Future.

Recursion as the primary mean of expressing procedure calling.

Recursion is highly overrated. When I call MAP, I don't care whether that's implemented using recursion or gotos. Also, recursion will blow the stack. :P And for some good old appeal to authority: Guy Steele says that we have to get rid of the accumulator idiom, and I heartily agree.

Heterogenous lists as the way of composing and dissecting complex data.

I'd much rather use real data structures, like java.util's List, Set, Map, and TreeMap. They seem to work great for about 99% of my data structure needs. The cons is a relic.

Programs as data.

Code is more than data.

The drive was to make [Common Lisp] compatible with the previous dialects and the easiest way was to union these features together.

Agreed. CL's kernel could be a bit smaller, without losing any cool.

The Common Lisp community has tried to deal with this through libraries. But these libraries are unsupported, and either undocumented or poorly so.

Agreed. The next Lisp has to attach itself to another platform. I root for GNU/Linux. Using the JVM means mixing business and pleasure way too much.

It’s a great language but it doesn’t look great.

I don't think Lisp can be made to look great, and I don't think it matters. Clojure seems to try this, and it's even more butt-ugly than CL.

For example, nearly every other newbie who hits CL for the first time complains about the brackets. And when they do they are told to shut up because they don't understand Lisp.

Until we have generalized 2D hypercode, the parentheses are the only sane, rational thing to do, in the ugly reality of a universe built on plain-text files.

Python is not new; its just a sugaring on the Lisp genotype in favour of a syntax that people feel comfortable with. And look how far that idea went.

%#$^#@$@$^#$%#@^@^!!!???

IMO, Python is farther away from the Lisp genotype than Java. At least Java has post-1980's scoping rules.

Also, except for popularity, Python didn't go anywhere as a language.

Doing the Right Thing

As I've said before, the great thing about Lisp is it's age. Like a good wine, Lisp has had a long time to age.

I think one of the best examples of the ripeness is R6RS' expansion process. It's a very good specification of a quite complex process, namely the processing of a number of expressions by the compiler.

The complexity stems from the fact that Lisp is at its core a multi-phase language: source code contains runtime values, such as variables and functions, as well as compile-time values, such as macros.

In a Lisp source file, one expression may be for execution at runtime, while the next is for execution at compile-time.

The intermingling of these different phases in a single source file seems to be somewhat of a historical accident, as one day we'll all be programming in a hypercode environment, but well, that's how it is.

Anyway, if you're writing a Lisp, study R6RS' expansion process and enjoy! :)

Dylan and the Lisp family tree's central branch

Since the early eighties (beginning with Scheme and T), most Lisps began to settle around a common core.

(This also coincides with the point in time when static scoping was finally understood, once and for all, after a painful and embarrassing history.)

With the exception of Scheme, most Lisps don't have multi-shot continuations. They seriously complexify a language, as can be seen in weird implementation techniques like Cheney on the MTA.

It's also hard (or even impossible?) to make a good UNWIND-PROTECT in the face of multi-shot continuations. And UNWIND-PROTECT is surely one of the most important control flow operators.

So what is the common core I'm talking about? You can see it best in Dylan, I think.

First of all, a Lisp that follows this common core design can be efficiently implemented on real hardware. No need to do weird stuff like stack copying or Cheney on the MTA. In fact, Dylan has, with a bit of squinting, the same dynamic (control flow) semantics as C.

Second, the common core is simply nice to program in.

Some features of the common core:

• lexical exits
• unwind protection
• condition system with restarts
• global and lexical bindings
• first-class functions with optional, keyword, and rest parameters
• dynamic (thread-local) variables
• class-based object system
• generic functions
• powerful macros
• multiple return values
• semi-functional stateful programming style

This core can be seen in languages that I consider the central branch of the Lisp family tree:

• Common Lisp
• Dylan
• EuLisp
• ISLISP
• Goo
• PLOT

All of them are great languages, and worth detailed study.

In the future I hope to write about each of the features these languages share in more detail.

Forms

There is beauty in the fact that Lisp expressions are called forms.

Form is really fundamental. And so are Lisp expressions.

While Lisp expressions use a limited syntax (s-expressions) they are nevertheless fundamentally free-form.

One day we will all be programming in 2D graphics, and Lisp already accommodates these. To Lisp, it doesn't make a difference whether a form is 1D or 2D.

A form is simply something to be evaluated.

LISP (de)Motivator

(Thanks to Cpunx for the tagline.)

Code is (more than) data

In Lisps like Common Lisp, code is data.

(In CL, code is represented as cons lists, which totally sucks, because you can't attach any metadata (such as line number information) to the code.)

One of the insights of the "Scheme community" is that code is more than data. They found this out by investigating hygiene.

In essence, code has some features that aren't easily captured by simply using s-expressions.

Namely, code has lexical structure. Code containing LET for example introduces new bindings, and if you simply render that as lists, you're losing important information.

If you want hygiene (hint: you want it), simply using s-expressions isn't enough. You need syntax objects.

More later.

Why I ignore Clojure

Because Clojure ignores Lisp.

Put that gun down, and let me explain.

Lisp is a process. Progress is measured in decades.

Lisp gets lexical scope right, for example, simply because we have a head start of decades over the competition others.

I'm quite certain that the unwashed masses will discover macros around 2020 and hygiene around 2040.

It's simply a matter of time.

Lisp moves slooooooooooooooowly.

And because of this slowness, individual contributions simply don't matter. They're just a blip on the radar. You can measure a programmer's inexperience by the amount of saliva generated by new, unproven language features.

Yeah, great, Clojure does some newfangled stuff with persistent data structures. Well, if it works, Lisp will adopt it in a couple of decades. In the meantime, we'll just keep on using the same tools we've been using for decades, well.

Yeah, great, Clojure has some semi-solution for hygienic macros. Again, we'll see how that plays out in a couple of decades.

If you have some great new idea for PLs, you can be quite certain that Lispers heading an Ivy League university's CS dept have already tried it out, decades ago. And if it isn't in today's Lisps, it's because it didn't work.

So what do I mean when I say that Clojure ignores Lisp? Well, it ignores the fact that individual inventions in Lisp are mostly meaningless, because Lisp is a community process.

For the larger Lisp family to adopt a feature, it has to be tried out and implemented in a dozen different dialects and implementations, over the stretch of decades. Until that happens, it didn't happen.

And there have been and are so many smart people in this community, that if you think you've got the great new idea to push Lisp farther, you're wrong.

(I'm surely not one of these dweebs that say CL is the be-all/end-all of Lisps, and we should all be using it. Hell no. Everyone should be writing new Lisps! It's one of the greatest learning experiences there is.)

Now, go ahead and have fun using Clojure if it floats your boat, but stop whining about how it's a more modern or better Lisp. That doesn't make the slightest bit of sense.

Why do you glorify doing something new and stupid, when doing good things well is what people really should be admiring. — Linus Torvalds

[Update: this is a rant. I'm not writing articles. The good people at HN have pointed out the many flaws of this rant.]

That Ragged Old Lisp

Lisp is ugly. Will always be. If you care, you're just not ready yet.

One of my big Common-Lisp-Aha!-moments was when I realized that under its ragged, shabby, muddy exterior lurked the dynamicest, funnest, object-orientedest language of all times.

Yes, the operators may be called PROGV, FMAKUNBOUND, and CDAADR. But the semantics below is just cute. PARC-cute. Industrial-strength. ANSI-standardized. ISO-standardized.

(Insert a huge hand wave here. Actually, we should have gotten rid of cons lists a long time ago, for example.)

Lisp's uglyness is like a stealth-coat, keeping it hidden from the clueless. In fact, in the Lisp I'm currently developing, I'm keeping the ugly names by design, in order to keep the wrong people out.

A great programming language can be found anywhere*. Do yourself a favor and go find the originals. And study them, until you can recite their specification by heart. Then implement them.

(* Totally false, of course. Great programming languages are few and far between.)