Do you have a specific use case or need in mind? If you want it to remember things, you wouldn’t necessarily ‘feed it into an LLM’ but if you want it to produce output more like how you’d speak, then fine-tuning would probably be appropriate.

Depending on what you wanna do, it will have different design requirements.

In general, I’d ask what’s the desired goal first.