Hello guys,

I have a question regarding the nested backquotes in macros. I wrote a macro, which creates lexical bindings for “port:ip” values:

(defun mkstr (&rest args)
  (with-output-to-string (s)
    (dolist (a args) (princ a s))))

(defun mksymb (&rest args)
  (values (intern (string-upcase (apply #'mkstr args)))))

;; my macro
(defmacro with-free-ports (start end &body body)
  (let ((range (loop for port from start to end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
    `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
       (progn ,@body))))

One sets a range of ports on localhost and these ports are bound to symbols port-1, port-2, etc…

(with-free-ports 1 3 port-1) ;; => "127.0.0.1:1"

This works fine if the start or end parameters are given as values. But if they are variables. which must be evaluated, this macro doesn’t work:

(let ((start 1))
  (with-free-ports start 3 port-1)) ;; error

In order to fix it, I made the let- bindings a part of the macro-expansion:

(defmacro with-free-ports (start end &body body)
  `(let ((range (loop for port from ,start to ,end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
     `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
	       (progn ,@body))))

but get a compilation warning that the body is never used. I assume this is because of the inner backquote.

To evaluate ,@body inside the inner backquote, I use one more comma, and the macro compiles without warnings:

(defmacro with-free-ports (start end &body body)
  `(let ((range (loop for port from ,start to ,end collect (format NIL "127.0.0.1:~a" port)))
	(n 0))
     `(let ,(mapcar #'(lambda (p) `(,(mksymb "PORT-" (incf n)) ,p)) range)
	       (progn ,,@body)))) ;; one more comma here

But it doesn’t work:

(let ((start 1))
  (with-free-ports start 3 port-1)) ;; error: port-1 is unbound

because with this ,,@body I evaluate port-1: (progn ,port-1) and this triggers the error.

I would appreciate if smbd can help me a bit and say what I am doing wrong.

Thank you.

  • dr675r@alien.topBEnglish
    1·
    1 year ago

    I will preface this by saying in general I don’t think the approach of interning symbols and binding them at runtime is a good idea. If you’ve referenced these symbols by name in the body of the form, you already knew you needed them, so why not just just use LET to create the binding? Without knowing the details of your use case it sounds like it could be an XY problem.

    However, this being Common Lisp, there is a way to do most things and the special operator PROGV is used to create dynamic variable bindings. This is not a tool I reach for often, but it does occasionally come in handy. I am not proud of this:

    (in-package #:cl-user)
(ql:quickload "alexandria")

(defmacro with-free-ports ((start end &key (prefix "PORT-") limit) &body body)
  (check-type limit (or null (integer 1 *)))
  (alexandria:with-gensyms (first-port last-port index vars values symbol-prefix)
    `(let ((,first-port ,start)
           (,last-port ,end)
           (,symbol-prefix ,prefix))
       (check-type ,first-port (integer 1 *))
       (check-type ,last-port (integer 1 *))
       (assert (>= ,last-port ,first-port) (,first-port ,last-port) "port range not ordered")
       (check-type ,symbol-prefix (or string symbol character))
       ,(when limit
          `(when (>= (- ,last-port ,first-port) ,limit)
             (error ,(format nil "WITH-FREE-PORTS limit (~A) exceeded" limit))))
       (loop named with-free-ports
             for ,index from ,first-port to ,last-port               
             collecting (alexandria:symbolicate ,symbol-prefix (princ-to-string ,index)) into ,vars
             collecting (format nil "127.0.0.1:~A" ,index) into ,values
             finally (return-from with-free-ports (progv ,vars ,values ,@body))))))

    To use it:

    (let ((start 80))
  (with-free-ports (start (+ start 5) :limit 10)
    (setf port-80 "http")
    (values port-80 port-85))) ;; => "http", "127.0.0.1:85"

    I have added what I consider the bare minimum of safety checks, such as a limit on the number of symbols it will create. You also lose help from the compiler which can’t warn you about unused variables and you may get warned that some of your variables have been assumed special (at least on LispWorks). Overall, I think its a bit of hack and not something I would put in production code.

    It is also worth noting the following loop achieves the same thing, without the hackery and is trivial to change to a vector if you’re worried about efficiency:

    (let ((start 80))
  (loop for port from start to (+ start 5)
        collecting (format nil "127.0.0.1:~A" port) into ports
        finally (progn
                  (setf (first ports) "http")
                  (return (values (first ports) (sixth ports))))))
    • xhash101@alien.topOPBEnglish
      1·
      1 year ago

      Thanks a lot!

      If you’ve referenced these symbols by name in the body of the form, you already knew you needed them,

      Yes, but I do not know how many of them. The purpose of this macro is to create a lexical environment to test a network code. Some functions take just one port as a parameter, but the others - two and more. With this macro it would be very easy to call these functions and to link their IO through the port numbers:

      (with-free-ports 0 10
  (fn-1 port-1 port-2 port-3)
  (fn-2 port-1))

      And with the first posted version of with-free-ports I achieved it. The problems appeared when I tried to bind start in run-time. From your reply and from the u/lispm comments I understood that this is not a good idea. What I do not understand, is how to use such macros inside another macros/functions, which can supply with-free-ports with start and end parameters ? Also, I cannot figure out, when exactly macro-expansions take place if I use many enclosed macros. Are they expanded all at once when I compile my code?

      Anyway, I am very grateful for your suggestions. This helps a lot to learn CL.

      • dr675r@alien.topBEnglish
        1·
        1 year ago

        Macros are expanded recursively, so if the result of expanding a macro is itself a macro then the evaluator or compiler will immediately expand it again. This is why we have MACROEXPAND-1 which expands it once, and MACROEXPAND which expands a form until it is no longer a macro form. I would suggest you MACROEXPAND-1 my answer, then repeat with MACROEXPAND to see the non-macro code that actually ends up being compiled/evaluated.

        You may find Section 3 of the HyperSpec helpful. The spec goes into a fair amount of detail about the semantics of both evaluation and compilation which should help.