I’ve had an equivalent of this in a local test service for a while now and one thing I observed was it was really important to give the calling LLM information about when a given document was going to be useful. The usefulness of multiple documents skyrocketed when they were well named and I supplied a one-line description of the content in the main prompt.