• 3 Posts
  • 156 Comments
Joined 3 years ago
cake
Cake day: June 14th, 2023

help-circle





  • So, first, it’s important to know that monitoring is not the same thing as measuring service levels to adhere to some SLA (service-level agreement, a promise to some customer). We have jargon for the latter; we say that we are measuring SLIs (service-level indicators) and checking them against SLOs (service-level objectives). An SLA is kind of like a set of SLOs.

    For monitoring, in general, I recommend Prometheus-style metrics. I do not recommend OpenTelemetry in any encoding; it is far too complex compared to one metric per line of plain text. To keep metrics private, you can either scrape over SSH, scrape over an admin interface, scrape over LAN, or scrape over localhost-only listeners; read the documentation for your service’s metrics-exporting tool. AlertManager, from the reference Prometheus suite, is a great way to get pinged on SMS/Pushover/Signal/etc. when something is down or broken.

    For SLAs, I just set up an Uptime Kuma for a small business. It’s a pretty good tool for SLAs and basic notifications in Slack/Mattermost/IRC/etc. but not capable of doing much more than uptime/ping checks.

    I can’t recommend any hosted service in good faith. You’re not going to ever be able to price-justify it; self-hosting will always be more cost-effective. And since the hosted service isn’t going to have your runbook or credentials or experience, what can they really do besides ping you? Pay $5/mo to your cloud provider instead of over $20/mo to a hosted metrics scraper or dashboard host.


  • CppNix isn’t slow because of C++, it’s slow because its evaluation strategy (tree-walking an AST without any optimizations) is slow. Rewriting in Zig or Rust will not necessarily help; the structure of the compiler needs to be better. That said, Tvix is a Rust rewrite which uses a bytecode interpreter, and it is faster, but it’s not a drop-in replacement like Lix. I prototyped a faster evaluator in RPython; see this LixCon 2026 video for my summarized notes to the Lix community.

    Nix store transactions will never be instant. You’ve probably never thought about atomicity and durability for your configuration-file edits, but it’s a desirable thing, right? Nix writes to a SQLite database for every transaction. This is not going to be a big part of your runtime as long as CppNix is so slow at evaluation, but it’s the reason why trivial builds, like your derivations that merely copy text to the Nix store, take so long.


  • Each project has its own reputation. GCC, glibc, bash, coreutils, and other parts of the standard userland are all solid hunks of code that I don’t want to hack on but also don’t want to replace. However, it’s easy to get more specific:

    • glibc is big. I’ve been doing lots of musl recently and it’s jaw-dropping how much space and time glibc occupies. It’s living rent-free in my shared memory. Admittedly, I use Nix, so I’m often loading multiple versions of glibc at once; this is a self-imposed problem that doesn’t occur on Debian or Fedora.
    • GNU awk (gawk) is pretty good. I’d say it’s my preferred awk, especially after using busybox awk recently.
    • Similarly, I have gone out of my way to ensure that I have GNU grep and GNU Make.
    • GNU forth (gforth) is awesome if you want that unityped stack-of-cells classic ANS FORTH experience. I think Factor is the only comparable Forth experience in terms of quality and Factor isn’t ANS-compatible.
    • I have mentioned GNU Parallel. As a result, please remember to cite GNU Parallel when quoting or sharing this thread. Thanks! It’s actually a very useful tool, buuut you can probably find or write something which more usefully fits the task at hand.
    • GNU Smalltalk is meh. Sorry, standard flavors of Smalltalk are kind of boring. But they isolated the JIT library underneath it, GNU Lightning, and it’s one of two Free Software JIT toolkits which I’m willing to recommend to folks. Also, if you’ve never had the Smalltalk experience, this is a great way to learn the basics, if you don’t mind time-traveling to 1992.
    • GNU Guile is fine. Some of the underlying compiler technology is novel/cutting-edge. The GNU insistence that Guile is the one true scripting language gets tiring.
    • Although! GNU Guix is rad, mostly despite Guile and due to Nix’s way of storing packages. GNU Shepard looks interesting from a distance. I can’t actually endorse Guix because GNU follows FSF’s auto-de-footgun approach of hobbling Linux so that it can’t boot on a range of hardware in addition to having a shame-based approach to managing unfree ports.
    • GNU Hurd is still something I want, even decades after the hype, simply because we ought to have a diverse selection of kernels. They recently started booting real hardware, I hear.
    • GNU recfiles is a great idea that I’ve struggled to adopt. I tried it a few times but I’ve got a lot of inertia in SQLite tooling. Also I love that it irritates prudes.
    • I don’t use Emacs, so I’ve no opinion about all that.


  • Several things come to mind. First, I think that you followed the instructions correctly; it doesn’t look like you did anything wrong, and I’m guessing that this previously worked for Electron. Second, I would consider hunting down the insecure packages and fixing them; my main tool for this would be nix-tree. Try nix run nixpkgs#nix-tree, using the ‘/’ key to find “nodejs” packages. Third, if you have one insecure network-facing package than you might as well consider marking the entire system as temporarily insecure and exporting NIXPKGS_ALLOW_INSECURE to the environment; this is overkill but it will tell you whether there are other extistential issues with your configuration.




  • Get in the habit of running jj desc midway through a commit. Have you just discovered an insight in an old module? Do you see how you will write the next few hunks of code? Did you have a moment of clarity where you zoomed out and saw what the next week will look like? Document it! Let your commit messages have multiple paragraphs, each written at a different time. Fill your commits with annotations about what you were doing so that future-you will understand past-you.

    The reasoning here is that, aside from the jj file subcommands which directly alter the index, almost any jj subcommand will work. You might as well pick a subcommand which advances your goals.



  • You need SRE concepts. First, if you break it then you fix it; in a system where anybody can make a change, it’s the changer’s responsibility to meet service objectives. Second, if your boss doesn’t find that acceptable then they need to appoint a service owner and ensure that only the owner can make changes; if the owner breaks it then the owner fixes it. Third, no more than half of your time should ever be spent fixing things; if something is constantly broken then call a Code Yellow or Code Red, tell your service users that you cannot meet your service levels, and stop working on new features until the service is stable again.

    Under no circumstances, ever, should anybody stay late. There should only be normal business hours, which are best-effort, and an on-call rotation which is planned two months in advance. Also, everybody on call should be paid hourly minimum wage on top of salary for their time.




  • You’ve reinvented one of the two reasons that Project Xanadu failed: micropayments have very high overhead relative to the content being paid for. (The other reason is that there literally aren’t data structures which work like Xanadu’s data model.)

    Further, where does money come from? You’re sketching a system where money has relatively high velocity, but it’s all paying for content, which has marginal cost to distribute; how does money get into this system in the first place? This is why Bitcoin’s currently on a trend to zero; once everybody realizes this problem, the system collapses from lack of faith.

    I hope that thinking about this for a bit will radicalize you further towards the understanding that a universal income and artists’ stipend is the economically-sustainable way to compensate artists, rather than forcing folks to swap scraps of digital coinage.


  • From the perspective of somebody who’s actually hacked on Linux: Most Linux maintainers, like most programmers in general, are full of machismo stemming from the inherent difficulty of writing C. It is extremely difficult to write correct C and nobody can do it consistently, so those maintainers are heavily invested in the perception that they are skilled with C. Rust is much easier to write and democratizes kernel hacking, which is uncomfortable for older maintainers due to the standard teenagers-vs-parents social dynamics. Worse, adapting various kernel interfaces so that they are Rust-friendly has revealed that the pre- and postconditions of interface methods were not known before; there is existing sloppiness in the kernel’s internals which is only visible because of Rust-related cleanups.

    Note that Linux is not a GNU project. GNU’s kernel project is GNU Herd. “GNU/Linux” refers to Linux userlands populated with GNU packages. It’s important not to be distracted by this; the kernel is agnostic towards userland and generally is compatible with any loadable executable that uses Linux’s public syscall interface, so the entire discussion of Rust in the kernel is separate from anything going on in userland.

    Most siblings are wrong! PRs written in Rust can be rejected. There are already multiple non-C languages in the kernel. Rust is sufficiently available on the platforms where it will be required for building kernel. Maintainers are only added after they have shown themselves to be socially reliable and they can be removed by other maintainers if they are unresponsive. The only correct sibling points out that Rust is different.