Router Query Engine

• Given a query, the router will pick one of several query engines to execute a query → this gives us some dynamic query understanding capabilities.

• To build a simple router (with only two query engines) over one single document, we do the following:

  • We index the document into two vector indexes:
    • Vector Index → upon receiving a query, it returns the most similar results.
    • Summary Index → upon receiving a query, it return everything.
  • Define query engines and set metadata (this is LlamaIndex specific syntax, using other libraries would require adopting their syntax)
  • Define router query engine
    • There are several selectors in LlamaIndex that we could use here:
      • The LLM selectors use the LLM to output a JSON that is parsed, and the corresponding indexes are queried.
      • The Pydantic selectors use the OpenAI Function Calling API to produce Pydantic selection objects, rather than parsing raw JSON.

• code

TABLE OF CONTENTS

Tool Calling

• In this section, we examine how we can leverage LLMs to use tools and also infer the arguments to the function (tool).
• Tool calling enables LLMs to interact with external environments through a dynamic interface where tool calling not only helps choosing the appropriate tool but also infer necessary arguments for execution.
• In standard RAG, LLMs are mainly used for synthesis of information only.
• Tool calling adds a layer of query understanding on top of a RAG pipeline, enable users to ask complex queries and get back more precise results.