lol we just unlocked a new paradigm, guaranteed we don’t hit a plateau for at least another two years. considering it looks like we’re probably already on the verge of one or two paradigm shifts on top of that, no real reason to anticipate a plateau in the immediate future regardless.
this isn’t mechanistic interpretability, it’s debugging.
except that it seems the employees who stayed are the ones least likely to do this.
just wanted to say that this is a domain i don’t have a lot of experience with and I would be very interested if you keep us updated with your findings as you explore the different options available.
learning dynamics and geometry. this definitely gets some attention, but almost always in the context of scaling. it’s a pretty interesting topic in its own right.
cool idea! one thought i had: rather than modifying the embeddings themselves to be more interpretable, you could keep the embeddings fixed and learn models (e.g. linear probes) that interrogate the embedding to produce the sub-embeddings you’re interested in. the downside to this approach is it adds an extra “sub-embedding factorization” inference step when you want to compute those scores, but otherwise i think this approach might be more reliable, flexible, and cheaper than finetuning the upstream embedding model to output embeddings with the desired properties.
causal inference, leveraging machine learning methods to either estimate causal effects, infer causal graphs, or both. if you’re not familiar with “causal inference”, start with learning about that first and then move up to ml applications within that domain after getting your feet wet with CI first. Judea Pearl’s “The Book of Why” is a good introduction to the topic.
it’s possible to “overfit” to a subset of the data. generalization error going up is a symptom of “overfitting” to the entire dataset. memorization is functionally equivalent to locally overfitting, i.e. generalization error going up in a specific neighborhood of the data. you can have a global reduction in generalization error while also having neighborhoods where generalization gets worse.