LlamaIndex

LlamaIndex is a framework for building retrieval and agent applications over your data. Bayle connects through its OpenAILike class.

Prompt to your agentCopies this page's setup instructions, to paste into your coding agent.

Show the prompt

Help me set this up to use Bayle AI, an OpenAI-compatible API. Full guide: https://docs.bayle.ai/guides/sdks/llamaindex/

Goal: use Bayle models in my LlamaIndex (https://www.llamaindex.ai) code with OpenAILike.

1. Install the package if `python -c "from llama_index.llms.openai_like import OpenAILike"` fails:
   pip install llama-index-llms-openai-like
2. Pick a model and read its real context size:
   curl -s "https://api.bayle.ai/models?type=text" \
     | jq -r '.data[] | "\(.id)  \(.model_spec.availableContextTokens)"'
   Show me the list, let me choose, and use that id as MODEL_ID and its size as context_window.
3. In my code, build the LLM like this:
   import os
   from llama_index.llms.openai_like import OpenAILike

   llm = OpenAILike(
       model="MODEL_ID",
       api_base="https://api.bayle.ai",
       api_key=os.environ["BAYLE_API_KEY"],
       is_chat_model=True,
       context_window=128000,
   )
   Replace context_window with the chosen model's real size; the default of 3,900 tokens makes LlamaIndex truncate far more than needed. `is_chat_model=True` is required, because the default False calls the legacy /completions endpoint, which Bayle does not serve. If the model supports function calling and I use agents, also add is_function_calling_model=True.
4. Verify by running:
   print(llm.complete("In one sentence, what is an embedding?"))
   and show me the output.

Rules:
1. The base URL is https://api.bayle.ai exactly as written. Append no path to it.
2. Read the key from the BAYLE_API_KEY environment variable. Never write the key itself into a file you create, a command, a commit or the chat. Where the tool keeps the key in its own settings screen, tell me where to paste it. If BAYLE_API_KEY is not set, stop and ask me to create a key at https://chat.bayle.ai/developers and export it.
3. Take model ids only from GET https://api.bayle.ai/models (no key needed). Never guess one.
4. Finish with one real test request, show me the result and list the files you changed.

Install

pip install llama-index-llms-openai-like

See the official OpenAILike reference for installation and every option.

Use

Don't have a Bayle key yet?Get API key

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="MODEL_ID",
    api_base="https://api.bayle.ai",
    api_key=os.environ["BAYLE_API_KEY"],
    is_chat_model=True,
    context_window=128000,
)

print(llm.complete("In one sentence, what is an embedding?"))

• is_chat_model=True is required. It defaults to False, which calls the legacy /completions endpoint that Bayle doesn’t serve.
• Set context_window to the model’s real size. The default is 3,900 tokens, which makes LlamaIndex truncate far more than it needs to. The size is in model_spec.availableContextTokens on GET /models.
• Add is_function_calling_model=True for a model that supports tools, to use it in agents.
• The parameter is api_base, not base_url.

Pick a model

curl -s "https://api.bayle.ai/models?type=text" \
  | jq -r '.data[] | "\(.id)  \(.model_spec.availableContextTokens)"'