Common Lisp is a bit like an onion. There are different layers of language and the inner layers date back to the first Lisp implementation from 1958. It’s grown over decades.

A beginner is now confronted with history and inconsistencies. On the positive side a goal was not to throw away useful older applications or libraries, they could be ported over the different generations of Lisp. Same for the book. If you read a CL introduction from the mid 80s it’s still mostly valid, but lacks newer stuff. It’s also the implementations: some SBCL code may date back to the early 80s, with Spice Lisp, which was started even before Common Lisp existed.

At that time PROG was a widely used control structure for imperative programming. PROG1, PROG2 and PROGN were a bit similar and indicate in the name which result is returned.

You walk in a street, where some house are a few hundred years old and some are relatively new, using different building styles.

why would evaluate the comparison? The comparison is a symbol, which a name for the function. Just use that symbol and insert it into the code.

(defmacro for ((param-name start-value end-value &optional (step1 1)) &body body)
  (let* ((func-name (gensym))
         (comparison (if (< step1 0) '<= '>=))
         (start (gensym))
         (end (gensym))
         (step (gensym))
         (k (gensym)))
    `(let ((,start ,start-value)
           (,end ,end-value)
           (,step ,step1))
       (labels ((,func-name (,param-name ,k)
                  (if (,comparison ,end ,param-name)
                      (progn (progn ,@body)
                        (,func-name (+ ,param-name ,step)
                                    (1+ ,k)))
                    ,k)))
         (,func-name ,start 0)))))

If you actually want to make sure that function does not get redefined, you need to understand that functions and variables are in different namespaces. You would need to get a function object and funcall it.

(defmacro for ((param-name start-value end-value &optional (step1 1))
               &body body)
  (let* ((func-name (gensym))
         (comp (gensym))
         (comparison (if (< step1 0) '#'<= '#'>=))
         (start (gensym))
         (end (gensym))
         (step (gensym))
         (k (gensym)))
    `(let ((,start ,start-value)
           (,comp ,comparison)
           (,end ,end-value)
           (,step ,step1))
       (labels ((,func-name (,param-name ,k)
                  (if (funcall ,comp ,end ,param-name)
                      (progn (progn ,@body)
                        (,func-name (+ ,param-name ,step)
                                    (1+ ,k)) ) ,k)))
         (,func-name ,start 0)))))

But you don’t need that.

If you want to debug macros, you need to see the expansion. Do it like this:

CL-USER 14 > (pprint
              (macroexpand
               '(for (i start end)
                  (print i))))

(LET ((#:G1788 START) (#:G1789 END) (#:G1790 1))
  (LABELS ((#:G1787 (I #:G1791)
             (IF (>= #:G1789 I)
                 (PROGN
                   (PROGN (PRINT I))
                   (#:G1787 (+ I #:G1790) (1+ #:G1791)))
               #:G1791)))
    (#:G1787 #:G1788 0)))

(defmacro for   ((param-name start-value end-value &optional (step1 1)) &body body)     (let* ((func-name (gensym))            (comp (gensym))            (comparison (if (< step1 0) '<= '>=))            (start (gensym))            (end (gensym))            (step (gensym))            (k (gensym)))       `(let ((,start ,start-value)              (,comp ,comparison)              (,end ,end-value)              (,step ,step1))         (labels ((,func-name (,param-name ,k)                    (if (,comparison ,end ,param-name)                        (progn (progn ,@body)                          (,func-name (+ ,param-name ,step)                                      (1+ ,k)) ,k))))         (,func-name ,start 1)))))

Your indentation is wrong. The last ,k is at a wrong place. Shouldn’t it be returned by the other if clause.

If you have an error and you need help, then you need to provide the necessary information:

• a reproducible example
• the error message

sure, remove the LET

(defmacro for ((param-name start-value end-value &optional (step1 1)) &body body)
  (let* ((func-name (gensym))
         (comparison (if (< step1 0) '< '>))
         (start (gensym))
         (end (gensym))
         (step (gensym))
         (k (gensym)))

    `(let ((,start ,start-value)
           (,end   ,end-value)
           (,step  ,step1))

       (labels ((,func-name (,param-name ,k)
                  (let ((new-exprs (progn ,@body)))
                    (if (,comparison ,end ,param-name)
                        (,func-name (+ ,param-name ,step) (1+ ,k))
                      ,k))))
         (,func-name ,start 1)))))

Make sure that the new form evaluates the macro arguments in the correct order. The LET enforces it.

Now you would need to think what to do about the step thing.

Can it only be a number? Can it be computed? What if the step value is negative?

How to check that the PARAM-NAME is actually a symbol?

Write some test cases.

LispWorks documents this: https://www.lispworks.com/documentation/lw80/lw/lw-sql-33.htm#LWUGRM)execute-command

You might want to check the indentation of your code. It looks not right. Do you use TAB characters in your editor? Don’t!

Besides the eval is evil, what are other pitfalls of writing code this way?

One problem could be, that your code is not working, because you have a bug in your EVAL form. Did you actually try to run it?

Why the usual gensym in macro is not interned?

Because then there is no name clash with symbols written by the macro user. If you would intern gensyms, where would you intern it? In which package Your variant does not say, whatever the current package is the symbol will be interned. If the user has a symbol there, one may get random (hard to debug) name clashes.

Besides the eval is evil, what are other pitfalls of writing code this way?

EVAL is not ‘evil’. It’s just most of the time not needed and you need to understand what it does. EVAL evaluates the code always in the global environment. EVAL also may not compile the code, so it may run interpreted, depending on the Common Lisp implementation.

allows you to skip the often tedious parsing step by using the host’s read

READ mostly gives a tree of tokens. This makes it a form of a tokenizer. After we call READ we don’t know what the tokens mean: is FOO a variable, a macro, a function, a built-in special form, a type, … This can only be determined by interpreting/compiling the token tree.

Smalltalk also has similar interactive development styles. One can build the application incrementally piece-by-piece. It has a residual graphical integrated development environment -> the development environment is a part of the program. It uses late-binding via message-passing. It starts and saves images (-> dumps of the heap contents, data and code). It is introspective & reflective (one can find out about the running program and can change it).

Smalltalk

No, the compiler does not do that. If you look into your macro here, it just puts the args start and end back in. The generated code then is executed. Remember: START and END are bound to the source forms, not computed values.

In the original macro you had the form (loop for port from start to end ...), which you tried to compute at macro-expansion time. But the values of START and END are not necessary numbers, but source forms, like variable names.

If you compile code, then the compiler expands macro forms at compile-time. If you want to generate a variable number of let bindings, then the number needs to be known at compile-time.

start and end thus can’t be variables at run-time.

(let ((start 1))
  (with-free-ports start 3 port-1))

If we compile above form, then at compile-time the value of start generally is unknown.

You compile the file compile.lisp. You don’t load it, you are compiling it.

That means: you compile the expression (ql:quickload "defstar"), but you don’t execute it. The file-compiler generates code for function calls, but does not execute them.

Then the package stuff does not know the package named “DEFSTAR”, because the thing has not been loaded.

See EVAL-WHEN:

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload "defstar"))

Above makes sure that the enclosed form is also executed during compilation.

On my Mac with ABCL 1.9.2 I can load it via Quicklisp and get for example this error:

; Caught COMPILE-FAILED-WARNING:

;   Lisp compilation failed while compiling
    #

Proprietary implementation makes it hard to inspect what’s going on in the image and optimize the code comprehensively

If I were a paying Franz customer and I would be interested in SLIME/SLY improvements, I would kindly ask them to provide it. Maybe they would then just do it or ask the customer to pay for it. That’s what technical support is for.

Second: as a Franz customer one could get the source code for much of the product. I’m not a customer, but I guess this possibility still exists.

Allegro is not the ideal basis, especially for learning

I think it can actually be the opposite. Among new Lisp users GNU Emacs is often cited as a hurdle.

Allegro CL comes with an GUI based IDE on Linux, Windows, and Web browsers. This makes it possible to use it without GNU Emacs + SLIME/SLY. I consider that to be a feature. The IDE of Allegro CL has a bunch of features: https://franz.com/support/documentation/10.1/doc/cgide.htm#menus-dialogs-1

Best: the stuff is written all in Allegro CL itself and can be reused.

CMUCL is also way behind it platform support. CMUCL is mostly limited to being a 32bit application (IIRC). SBCL OTOH supports various 64bit platforms, incl. ARM64 on Linux and macOS.

CCL is mostly only on life support. The native port to the Apple Silicon Macs isn’t under way. CCL isn’t making any progress.

Allegro … So it actually is worse than open-source impls.

I doubt that this is generally the case.

CL-USER> (java:jstatic "sqrt" "java.lang.Math" 17.0d0)
4.123105625617661d0

That’s about the so-called “modern mode”, where standard Common Lisp symbols internally have lower case names (which violates the Common Lisp standard) and the default readtable case is :preserve. In the standard it is :upcase. Also other predefined symbols have lowercase names.

https://franz.com/support/tutorials/casemode-tutorial.htm

Is that incompatible language change used by other implementations?

See CLHS 1.4.1.4.1 Case in Symbols: http://www.lispworks.com/documentation/HyperSpec/Body/01_dada.htm

• “As such, case in symbols is not, by default, significant.”

• “The symbols that correspond to Common Lisp defined names have uppercase names”

The symbol will be interned in some package. A keyword symbol is in the package KEYWORD. That’s slightly better than to use the symbol of some other package, which “pollutes” that package with that symbol. ;-)

An alternative, out of fashion because of the upper case, is to use a string “FOOBAR”.