• 0 Posts
  • 2 Comments
Joined 1 year ago
cake
Cake day: November 2nd, 2023

help-circle

  • Could you use a traditional embedding, and then somehow search for a vector that represents the semantic feature you are interested in? What I mean is that, since LLMs can understand the concept of numbers, and this is a pretty fundamental part of language, presumably (but not necessarily) there is a vector in the high dimensional embedding space that represents the concept of “how many”. I’m thinking, of course, along the lines of the traditional example of “king” - “male” + “female” = “queen”, where you could, for example, define a “gender” vector based on “male”, “female” and perhaps a set of other related words.

    I’m not sure how feasible that is at all - I’m just curious if it’s something you explored or read about as you were doing this?