Overview
Instead of calling a tool 50 times (e.g., web_search, grep, update_row), the LLM calls batch once with 50 inputs. Ray executes them in parallel and returns one structured result.
Usage
from rayai import tool, BatchTool
@tool(desc="Look up information about a topic")
def lookup_topic(topic: str) -> str:
return f"Information about {topic}"
@tool(desc="Translate text")
def translate(text: str, target_lang: str) -> str:
return f"Translated '{text}' to {target_lang}"
# Create batch tool with your tools
batch_tool = BatchTool(tools=[lookup_topic, translate])
# LLM calls it like:
result = batch_tool(
tool_name="lookup_topic",
tool_inputs=[
{"topic": "Python"},
{"topic": "Rust"},
{"topic": "Go"}
]
)
# Returns:
# {
# "results": ["Information about Python", "Information about Rust", "Information about Go"],
# "errors": [None, None, None],
# "tool_name": "lookup_topic",
# "count": 3
# }
With LangChain Agent
from langchain.agents import create_agent
from rayai import tool, BatchTool
from rayai.adapters import RayToolWrapper, AgentFramework
@tool(desc="Search Wikipedia")
def search_wiki(query: str) -> str:
# Your implementation
return f"Results for {query}"
# Create batch tool
batch_tool = BatchTool(tools=[search_wiki])
# Convert to LangChain format
wrapper = RayToolWrapper(framework=AgentFramework.LANGCHAIN)
[lc_batch_tool] = wrapper.wrap_tools([batch_tool])
# Create LangChain agent (requires OPENAI_API_KEY env var)
agent = create_agent(model="openai:gpt-4o-mini", tools=[lc_batch_tool])
| Field | Type | Description |
|---|
tool_name | str | Name of tool to call |
tool_inputs | list[dict[str, Any]] | List of input dicts |
| Field | Type | Description |
|---|
results | list[Any] | Results (None if error) |
errors | list[str | None] | Error messages (None if success) |
tool_name | str | Name of tool that was called |
count | int | Number of inputs processed |
API Reference
| Parameter | Type | Description |
|---|
tools | list[Callable | RayTool] | None | List of tools to register |
name | str | Tool name (default: “batch”) |
description | str | Custom description |
validate_inputs | bool | Validate inputs against schema (default: True) |
BatchTool handles partial failures gracefully. If some inputs fail, others still return results. Check the errors list to see which inputs failed.
