For those of you who work with Transformers, are there any good pytorch libraries or performance benchmarks out there? Note that I am not looking for LLMs, I would just like implementations of the standard Transformer and various newer methods such as the Linear Transformer, Performer, Linformer, Reformer, etc. Ideally, I would like a library that also includes benchmarks, where various models can be compared in performance on simple to complex tasks. I would like to get into research in this space, but I don’t know where to start.

  • KingsmanVince@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    10 months ago

    You can check out HuggingFace transformers, xformers, Lucidrains implementations. For benchmarking, just test on GLUE.