Context:

I’ve used Emacs off and on for many years at this point. Throughout that time there have been periods where I really leaned in to it and tried to use it for everything, and there have been periods where I only used it for org and/or magit, etc. I’ve learned lots of things about it and I’ve forgotten lots of things about it, but I’ve never been what I would call an “expert” or even a “power user”. So, when I feel like something isn’t working well in Emacs, I almost always default to the assumption that I’m doing something wrong or misunderstanding something, etc.

So, it very well may be that I’m wrong/crazy in my recent conclusion that use-package might not be the ideal abstraction for managing Emacs packages.

With that out of the way, I’ll say that when I first saw use-package, I thought it was amazing. But, in the years that I’ve been using use-package, I never feel like my init file is “right”. Now, I’m starting to think that maybe it’s use-package that’s wrong and not me (insert Simpsons principal Skinner meme).

I don’t know how best to articulate what I mean by use-package being a “wrong abstraction”, but I’ll try by listing some examples and thoughts.

Autoloads

First of all, I feel like the way autoloads are handled with use-package is too mistake-prone. Libraries/packages typically define their own autoloads, but the use-package default is to eagerly load the package. I understand that installing a library via package.el, etc will process the autoloads for us and that manually/locally installed packages get no such benefit.

But, if we’re using use-package to also manage installing the packages for us (:ensure t), then why shouldn’t it know about the autoloads already and automagically imply a :defer t by default?

So, by default, we have to remember to either add :defer t or we have to remember that setting our own hooks, bindings, or commands will create autoloads for us.

I know that you can configure use-package to behave as though :defer t is set by default, but that’s just broken for packages that don’t have any autoloads.

It feels like maybe use-package is doing too many things. Maybe it was actually more correct in the old days to separate the installation, configuration, and actual loading of packages, rather than trying to do all three in one API.

Configuration that depends on multiple packages is ugly/inconsistent

Many packages are fairly standalone, so you can just do,

(use-package foo
    :defer t
    :config
    (setq foo-variable t))

and it’s clean and beautiful. But, sometimes we have configuration that is across multiple packages. A real-world example for me is magit and project.el. Magit actually provides project.el integration, wherein it adds magit commands to the project-switch-commands and the project-prefix-map. That’s great, but it will only run if/when the magit package is loaded.

So, my first guess at using use-package with magit was this,

(use-package magit
    :ensure t
    :defer t
    :config
    (setq magit-display-buffer-function #'magit-display-buffer-same-window-except-diff-v1))

which seems reasonable since I know that magit defines its own autoloads. However, I was confused when I’d be using Emacs and the project.el switch choices showed a magit option sometimes.

I eventually realized what was going on and realized that the solution was to immediately load magit,

(use-package magit
    :ensure t
    :config
    (setq magit-display-buffer-function #'magit-display-buffer-same-window-except-diff-v1))

but that kind of sucks because there’s no reason to load magit before I actually want to use it for anything. So, what we can do instead is to implement the project.el integration ourselves. It’s really just two commands:

(define-key project-prefix-map "m" #'magit-project-status)
(add-to-list 'project-switch-commands '(magit-project-status "Magit") t)

But, the question is: Where do we put these, and when should they be evaluated? I think that just referring to a function symbol doesn’t trigger autoloading, so I believe these configurations should happen after project.el is loaded, and that it doesn’t matter if magit is loaded or not yet.

Since, project.el is built-in to Emacs, it’s probably most reasonable to do that config in the magit use-package form, but what if project.el were another third-party package that had its own use-package form? Would we add the config in the project use-package form, or in the magit use-package form? Or, we could do something clever/hacky,

(use-package emacs
    :after project
    :requires magit
    :config
    (define-key project-prefix-map "m" #'magit-project-status)
    (add-to-list 'project-switch-commands '(magit-project-status "Magit") t))

But, if we do this a lot, then it feels like our init.el is getting just as disorganized as it was before use-package.

Conclusion

This is too rambly already. I think the point is that I’m becoming less convinced that installing/updating packages, loading them, and configuring them at the same time is the right way to think about it.

Obviously, if you know what you’re doing, you can use use-package to great success. But, I think my contention is that I’ve been familiar with Emacs for a long time, I’m a professional software developer, and I still make mistakes when editing my init file. Either I’m a little dim or the tooling here is hard to use correctly.

Am I the only one?

  • JDRiverRun@alien.topBEnglish
    1·
    1 year ago

    People seem to be missing the point that no conceivable package loading tool — use-package, general, straight, by-hand gnarly lisp using eval-after-load, etc., etc. — will solve the original problem as stated. You want a binding to appear, tying project to magit. That binding is specified at the top level in one of magit’s many lisp files. You want this binding to appear without loading magit. There is no approach which will accomplish this, other than excerpting that binding yourself, as you’ve done in the “clever/hacky” solution (though I’d put it in the project :config stanza for better organization).

    The way simple extension loading works in other applications in my experience is “fully load all configured extensions at startup”. This solves many problem (including this one), but is slow. You have the option to defer loading and much more with Emacs; the price you pay is complexity and the need to understand how the pieces fit together.

    • meedstrom@alien.topBEnglish
      1·
      1 year ago

      Maybe if Magit contained the following form?

      ;;;###autoload
(autoload #'project-switch-project "magit")

      i.e. autoloading an autoload :)

    • ragnese@alien.topOPBEnglish
      1·
      1 year ago

      People seem to be missing the point that no conceivable package loading tool — use-package, general, straight, by-hand gnarly lisp using eval-after-load, etc., etc. — will solve the original problem as stated. You want a binding to appear, tying project to magit. That binding is specified at the top level in one of magit’s many lisp files. You want this binding to appear without loading magit. There is no approach which will accomplish this, other than excerpting that binding yourself, as you’ve done in the “clever/hacky” solution

      Yes, agreed. My contention is not with the inherent complexity of the situation. Rather, my issue is related to this last part of your paragraph,

      though I’d put it in the project :config stanza for better organization

      That’s the whole problem with using use-package forms for configuration! use-package is centered around a single feature at a time, whereas many configurations involve tying multiple features together. In this case, it doesn’t matter if I put the configuration in project’s use-package form or in magit’s use-package form–in either case, one of the forms will be misleading because it will seem like all of the configurations related to that package is in that form, but it won’t be true because something about it is being configured in the other package’s form.

      That’s not the only thing, either. Another example is completions. I use vertico and corfu, but they tie in closely with a lot of built-in Emacs completion stuff, so I have the following settings,

      (customize-set-variable 'read-extended-command-predicate #'command-completion-default-include-p)
(setq completion-category-overrides '((file (styles partial-completion))))
(setq completion-styles '(flex basic))

      those settings are not part of the vertico package, but they’re being set because I’m using vertico and want it to behave a certain way. So, do they go in the vertico use-package, or do they go in an emacs or simple.el use-package, or do they just go at the top level somewhere?

      The truth is that we don’t actually configure packages in isolation, which is why I feel like use-package kind of imposes a structure that appears to make sense in a lot of cases, but does NOT actually make sense in the general case.

      • JDRiverRun@alien.topBEnglish
        1·
        1 year ago

        That’s the whole problem with using use-package forms for configuration! use-package is centered around a single feature at a time, whereas many configurations involve tying multiple features together.

        What would you propose as a better design? It seems like you hope that pretty much all users of such a mythical package system would converge on precisely the same init structure, given the same feature requirements. Given the complexity of the system, I don’t see how that’s possible. I mean not even for simple toml/yaml configs is that ever the case.

        In this case, it doesn’t matter if I put the configuration in project’s use-package form or in magit’s use-package form–in either case, one of the forms will be misleading because it will seem like all of the configurations related to that package is in that form, but it won’t be true because something about it is being configured in the other package’s form.

        You have two packages, and you’d like them to be tied together in some manner. Where should this config go so as to avoid such confusion? It has to go somewhere. The cleanest approach I know is to create a separate small package whose sole purpose is to make that integration happen, and use-package it. Some of these “tie-in” packages in fact do exist, for example:

        ;;;;; embark-consult: Tie the two together
(use-package embark-consult ...

        You could create your tie-ins as small user packages, and group them under a joint heading in your init. This is not as hard as it sounds: at base a package is just some .el file on the load-path which does (provide 'something). But even still, this requires you or a user of your init to notice this “combiner” package, and realize they need it if they want the joint functionality.

        So, do they go in the vertico use-package, or do they go in an emacs or simple.el use-package, or do they just go at the top level somewhere?

        This is kind of like asking whether socks go in the top drawer or the bottom :). Wherever you’d best remember them! Vertico makes the most sense to me. I use-package builtins all the time, so as to organize them in a way I will remember. I do all my completion category stuff in my orderless stanza, and for any “special” completion setups (e.g. eglot), do those in the relevant package stanzas. M-x find-library will show you all the things you can use-package.