GBNF, a rebranding of Backus-Naur Form is a kind of Regex if you somehow made Regex more obtuse and clunky and also way less powerful. It’s like going to the dentist in text form. It is bad, and should feel bad.
HOWEVER, if you tame this vile beast of a language you can make AI respond to you in pretty much any way you like. And you should.
You can use it by pasting GBNF into SillyTavern, Oobabooga, or probably something else you might be using. First, click on the
then scroll down and paste it like so:
In Ooba, you can go to
And then
Note that not all loaders support it, I think it’s limited to llama.cpp, transformers, and _HF variants.
Then, your next messages will be formatted like you wanted. In this case, every message will be "quoted text", *action text* or multiple instances. It should be simple to understand.
Here’s that one in case you want it, I just wrote it and tested it:
root ::= (actions | quotes) (whitespace (actions | quotes))*
actions ::= "*" content "*"
quotes ::= "\"" content "\""
content ::= [^*"]+
whitespace ::= space | tab | newline
space ::= " "
tab ::= "\t"
newline ::= "\n"
Even if you don’t know Regex this language should be easy to pick up, and will allow you to make LLMs always respond in a particular format (very useful in some cases!)
You can also look at the examples.
There are websites to test BNF like this one but since it’s a badly designed, badly implemented language from hell, none of them will work and you will have to look at the console to find out why this ugly duckling of a language didn’t want to work this time. Imagine if Batch files had regular expressions, it’d probably look like this. All of that said, this is pretty fucking useful! So thanks to whoever did the heavy lifting to implement this.
You must log in or register to comment.
I’d rather stick with pydantic declaration via Langchain than something that needs to be so hand written
GBNF is 10x more readable than regex, but neither one is very human friendly.
I won’t stand for this slander for GBNF, there’s a reason why programming language grammar is defined in GBNF and not regexes.
Actually GBNF is this re-branding, BNF is the proper name (the G is Georgi Gerganov’s). There’s also a reason why languages compile to assembly but that doesn’t mean it’s user-friendly. Or Abstract Syntax Trees. There’s stuff that pretty much only applies to compilers, doesn’t mean it’s a good general-purpose solution.
Though I must imagine implementing BNF is orders of magnitude easier than implementing the monster that is extended regular expressions.
I dunno in what bubble ebnf is worse than regex. Its more general (especially as a notation), easier to parse both for humans and computers, about as easy to codegen and is used all over the place in programming language design. The only advantage regex has is brevity and being able to fit it in a one liner for quick filtered search.
GBNF is super powerful, and anyone developing software with locals LLMs should learn about how to use it. As part of my larger open source project, Swiss Army Llama, I recently made a couple very handy tools for working with GBNF grammars. You can supply either an example JSON or a Pydantic data model, and it will automatically generate the complete GBNF grammar for you reflecting the same fields. It even supports some degree of nested fields. And there is another tool for taking a complete GBNF grammar specification and validating it. You can see how I implemented these particular tools here:
https://github.com/Dicklesworthstone/swiss_army_llama/blob/main/grammar_builder.pyOr if you just want to use the tools, you can install my project:
https://github.com/Dicklesworthstone/swiss_army_llama/tree/mainAnd just find the relevant endpoints in the Swagger page, which makes it super easy to try them out.
I think this is only available on
llama.cpp. I’ve been using it for a while for simple structured outputs and am extremely happy with the results. With OpenAI’s function calling, I always had to write validators – first to make sure the output is indeed a JSON, and then another validator to make sure the JSON complies with my JSON schema.grammarmakes all of that redundant because it is 100% guaranteed to generate the desired output (including JSON).Yeah I didn’t even thought this was possible, but it makes for a much safer way to do function calling! Like, imagine the pain of protecting against all the myriad exploits vs just using this. It’s fantastic.
And yeah I can only use it in
llama.cppfor some reason too, but I got the impression_HFshould have it.
Can anyone explain me how can i use this in SillyTavern?! Because I’m new to this whole thingy.
The regex editor is something else and is about find/replace after the AI has generated stuff. GBNF is about restricting the AI to only generate specific stuff. Like imagine you want a yes/no answer and the AI is physically unable to answer anything but that. While Regex is more about “replace all instances of X with Y in the response”.
