Let's look at the simple example...
First we just process plain pseudo-html:
CL-USER> (with-html-output-to-string (s)
(:p "a"))
"<p>a</p>"Then we mix it with Lisp code with the help of using a special symbol HTM to hint WITH-HTML-OUTPUT, where to switch:
CL-USER> (with-html-output-to-string (s)
(:ul (dolist (a '(1 2 3))
(htm (:li (str a))))))
"<ul><li>1</li><li>2</li><li>3</li></ul>"Now we try to define a macro:
CL-USER> (defmacro test (a)
`(htm (:li (str ,a))))
TEST
CL-USER> (with-html-output-to-string (s)
(:ul (dolist (a '(1 2 3))
(test a))))But the following code will cause UNDEFINED-FUNCTION error ("The function :LI is undefined.") Why? Because the translation of pseudo-html s-expressions would be already finished by the time of macroexpansion of TEST, so it will be expanded not inside
(:ul (dolist (a '(1 2 3)) ...), but inside calls to WRITE-STRING.It's definitely unlispy (not to be able to use macros anywhere you want :), so from the beginning of using CL-WHO I tried to find a work-around. (And I guess, many did the same). At least in the mailing list I turned out to be not the only one, who proposed something...
Below 3 solutions, that I've tried (in chronological order) are described. Each implements a distinct approach to combining pseudo-html forms with macros.
- Delimited macroexpansion: introduce an additional special symbol (in the lines of already present HTM, STR, ESC and FMT) -- EMB, which will instruct CL-WHO to MACROEXPAND-1 the enclosed form during input transformation.
This is the example (based on the example from CL-WHO manual):(defmacro embed-td (j)
`(:td :bgcolor (if (oddp ,j) "pink" "green")
(fmt "~@R" (1+ ,j))))
(with-html-output-to-string (*http-stream*)
(:table :border 0 :cellpadding 4
(loop for i below 25 by 5
do (htm
(:tr :align "right"
(loop for j from i below (+ i 5)
do (htm (emb (embed-td j)))))))))
I implemented it by tweaking the WITH-HTML-OUTPUT underlying input transforming function TREE-TO-TEMPLATE:(defun tree-to-template (tree)
"Transforms an HTML tree into an intermediate format - mainly a
flattened list of strings. Utility function used by TREE-TO-COMMANDS-AUX."
(loop for element in tree
nconc (cond ((or (keywordp element)
(and (listp element)
(keywordp (first element)))
(and (listp element)
(listp (first element))
(keywordp (first (first element)))))
;; normal tag
(process-tag element #'tree-to-template))
((listp element)
;; most likely a normal Lisp form (check if we
;; have nested HTM subtrees) or an EMB form
(list
(if (eql (car element) 'emb)
(replace-htm (list 'htm (macroexpand-1 (cadr element)))
#'tree-to-template)
(replace-htm element #'tree-to-template))))
(t
(if *indent*
(list +newline+ (n-spaces *indent*) element)
This is a simple and undertested solution, so it, probably, may fail in some corner cases. At least I couldn't make it reliably work, when I tried to combine it with PARENSCRIPT code (see http://common-lisp.net/pipermail/cl-who-devel/2008-July/000351.html). So it's just a direction (yet a quite possible one). But I decided to leave it aside to try the second approach... - Preliminary macroexpansion: don't modify CL-WHO, but build on top of it a macro infrastructure to expand the macros, before the whole form is passed to WITH-HTML-OUTPUT.
After thinking about a problem for some time I came to a conclusion, that it would be better to treat CL-WHO as just a good processor for static pseudo-html data, and on top of it build macros, which will allow eventually to add macroexpansion ability. For that let's consider such model: if we want to use macros, we define an "endpoint", at which they should be expanded (the function, that generates HTML after all). I called this endpoint WHO-PAGE. It will function as DEFMACRO, taking a backquote template, in which we can explicitly control evaluation of forms.(defmacro def-who-page (name (&optional stream) pseudo-html-form)
"Creates a function to generate an HTML page with the use of
WITH-HTML-OUTPUT macro, in which pseudo-html forms, constructed with
DEF-WHO-MACRO, can be embedded. If STREAM is nil/not supplied
WITH-HTML-OUTPUT-TO-STRING to a throwaway string will be used,
otherwise -- WITH-HTML-OUTPUT"
`(macrolet ((who-tmp ()
`(,@',(if stream
`(with-html-output (,stream nil :prologue t))
`(with-html-output-to-string
(,(gensym) nil :prologue t)))
,,pseudo-html-form)))
(defun ,name ()
(who-tmp))))
The definition of this HTML-generation functions uses macrolet to expand the macros before passing the body to CL-WHO. And the special macros, that will be used inside the template, could be defined like this:(defmacro def-who-macro (name (&rest args) pseudo-html-form)
"A macro for use with CL-WHO's WITH-HTML-OUTPUT."
`(defmacro ,name (,@args)
`'(htm ,,pseudo-html-form)))
But that's not all. Sometimes we will need to evaluate the arguments, passed to who-macro (to be able to accept pseudo-html forms as arguments), so a complimentary utility can be added (it's named by the PARENSCRIPT naming convention):(defmacro def-who-macro* (name (&rest args) pseudo-html-form)
"Who-macro, which evaluates its arguments (like an ordinary function,
which it is in fact.
Useful for defining syntactic constructs"
`(defun ,name (,@args)
,pseudo-html-form))
The example usage can be this:(def-who-macro title (for &optional (show-howto? t))
"A title with a possibility to show howto-link by the side"
`(:div :class "title"
(:h2 :class "inline"
"→ " ,(ie for)) " "
(when ,show-howto?
(how-to-use ,for)) (:br)))
(def-who-macro* user-page (title &key head-form onload body-form)
`(:html
(:head (:title ,(ie title))
(:link :rel "stylesheet" :type "text/css"
:href "/css/user.css")
(:script :src "/js/user.js" :type "text/javascript"
"")
,head-form)
(:body :onload ,onload
,(header-box)
(:table ,@(wide-class)
,body-form)
,(footer-box))))
I could successfully use this approach and have added some additional features to it (docstings and once-only evaluation, for example). Actually, I've built a whole dynamic web-site on it, and then faced a problem. As the code of the system grew quite complex with several layers of who-macros, SBCL started to lack resources to compile it. More specifically, upon re-compilation of the system it sometimes fell into heap dump. We tried to investigate the problem, but couldn't dig deep enough to find it's cause (or just lacked enough knowledge to properly debug it).
But the possibility of this solution indeed shows the nature of Lisp: virtually everything can be changed and all the boilerplate for that can be made transparent to the end-user. Alas it also shows, that this flexibility comes at a cost: it's pretty hard to build a robust and simple system, that will handle all the possible corner cases. - Transparent macroexpansion (idea and basic implementation by Victor Kryukov): the last approach (which I use now) was proposed in CL-WHO mailing list. It's in some sense the most obvious one: modify CL-WHO rules to accommodate macroexpansion.
Here's the implementation of this idea, that I use, which is tightly based on the one, proposed by Victor:(defparameter *who-macros* (make-array 0 :adjustable t :fill-pointer t)
"Vector of macros, that MACROEXPAND-TREE should expand.")
(defun macroexpand-tree (tree)
"Recursively expand all macro from *WHO-MACROS* list in TREE."
(apply-to-tree
(lambda (subtree)
(macroexpand-tree (macroexpand-1 subtree)))
(lambda (subtree)
(and (consp subtree)
(find (first subtree) *who-macros*)))
tree))
(defmacro def-internal-macro (name attrs &body body)
"Define internal macro, that will be added to *WHO-MACROS*
and macroexpanded during W-H-O processing.
Other macros can be defined with DEF-INTERNAL-MACRO, but a better way
would be to add additional wrapper, that will handle such issues, as
multiple evaluation of macro arguments (frequently encountered) etc."
`(eval-when (:compile-toplevel :load-toplevel :execute)
(prog1 (defmacro ,name ,attrs
,@body)
(unless (find ',name *who-macros*)
;; the earlier the macro is defined, the faster it will be found
;; (optimized for frequently used macros, like the inernal ones,
;; defined first)
(vector-push-extend ',name *who-macros*)))))
;; basic who-macros
(def-internal-macro htm (&rest rest)
"Defines macroexpasion for HTM special form."
(tree-to-commands rest '*who-stream*))
(def-internal-macro str (form &rest rest)
"Defines macroexpansion for STR special form."
(declare (ignore rest))
(let ((result (gensym)))
`(let ((,result ,form))
(when ,result (princ ,result *who-stream*)))))
(def-internal-macro esc (form &rest rest)
"Defines macroexpansion for ESC special form."
(declare (ignore rest))
(let ((result (gensym)))
`(let ((,result ,form))
(when ,form (write-string (escape-string ,result)
*who-stream*)))))
(def-internal-macro fmt (form &rest rest)
"Defines macroexpansion for FMT special form."
`(format *who-stream* ,form ,@rest))
An interesting side-effect here is, that now HTM (et al.) becomes not just a symbol, but a normal macro, so this will work out of the box:CL-USER> (with-html-output-to-string (s)
(:ul (dolist (a '(1 2 3))
(test a))))
And a user-level DEF-WHO-MACRO, that incorporates ONCE-ONLY (which is regularly needed for this domain) and docstring support can be defined like this:(defmacro def-who-macro (name attrs &body body)
"Define a macro, that will be expanded by W-H-O.
/(Its name is added to *WHO-MACROS*).
Body is expected to consist of an optional doctring and declaration,
followed by a single backquoted WHO template.
All regular, optional and keyword arguments are wrapped in ONCE-ONLY
and can't contain pseudo-html forms."
`(eval-when (:compile-toplevel :load-toplevel :execute)
(unless (find ',name *who-macros*)
(vector-push-extend ',name *who-macros*))
(defmacro ,name ,attrs
,@(when (cdr body) ; docstring
(butlast body))
,(let ((attrs (if-it (or (position '&rest attrs) (position '&body attrs))
(subseq attrs 0 it)
attrs)))
(if attrs
`(once-only ,(mapcar #`(car (as 'list _))
(remove-if #`(char= #\& (elt (format nil "~a"
(car (mklist _)))
0))
attrs))
`(htm ,,(last1 body)))
(last1 body))))))
I hope the above explanations and examples can be grasped easily. And moreover, that they will be useful for those, who are searching for the solution to the same problem. Still, if questions arise, be free to ask them.
If someone uses a different approach, I'm very interested to hear about it.
PS. Btw, Edi Weitz mentioned in the mailing list, that he is preparing a new version of the library. As he'd said, that addition of macroexpansion was on his TODO list, it will be interesting to see, what the new CL-WHO will hold...
5 comments:
Hi Vsevolod,
Any chance of uploading your patched CL-WHO? I'd like to try it out. Thanks!
Mark, sure. Which version do you need? Currently I'm using the 3rd one. It proved rather robust. Basically, all you need is to modify who.lisp with the given code. But, for clarity, the code (of the whole who.lisp and packages.lisp for version 0.11.1) can be found here: http://paste.lisp.org/display/80129. If you need other versions or have any questions, let me know...
I have not been using any of lisp library because of non-lisp way for their syntax. Now i've found exactly what i need, the library where it is easy to use s-exps for html and lisp and to mix html and lisp code. Try cl-yaclml :). Happy coding
The source code of the forked version is here: https://github.com/vseloved/cl-who
After an hour fiddling with macros, I finally stumbled upon your post. It solved my problem, thanks!
Post a Comment