Merlin Agent Builder

The Merlin agent builder helps you design, configure, and run AI agents directly inside Tray. Agents combine AI models with your data and workflows, so they can act with context from your business systems.

What is an AI Agent?

Copy

An AI (Artificial Intelligence) agent is an autonomous system that can understand its environment's context, make decisions, and take actions to achieve specific goals. Unlike simple chatbots that only respond to questions, AI agents can:

• Understand context from multiple data sources

• Reason about problems using available information

• Take concrete actions through connected systems

• Learn and adapt their responses based on outcomes

Think of it this way: While a chatbot might tell you "Your order status is unknown," an AI agent can check your order system, payment processor, and shipping provider, then either give you a complete status update or automatically resolve any issues it finds.

Components of an Agent on Tray

Copy

If you're familiar with building on the Tray platform, you can build an agent project that consists of:

• Agent scope - what you want your agent to do

  • Defines your agent's goals and behavior. This is critical for guiding the agent to successfully complete tasks. You provide relevant information to outline the agent's goals, specific instructions, and the style in which you want the agent to respond.

• Data sources - what information it can access

  • Connect your existing systems (Google Drive, Confluence, databases, etc.) to create a searchable knowledge base that your agent references when it needs context to make informed decisions.

• Workflows as tools - what actions it can take

  • Let your agent perform tasks by running chosen workflows, everything from running data queries to creating tickets to sending notifications. We provide prebuilt tools to help you get started, but you can also build your own bespoke tools using Tray's visual workflow builder.

• AI model - the brain that makes decisions

  • The LLM (Large Language Model) that powers decision-making. Your choice of model depends on your specific needs:

    • Task complexity - simple queries vs. complex reasoning

    • Response speed - fast responses vs. thorough analysis

    • Cost considerations - usage-based pricing varies by model

    • Context requirements - how much information the model needs to remember

• Agent workflow - how it all works together

  • This specialized Tray workflow acts as the orchestrator, using agent connectors to coordinate everything. It follows your scope guidelines, leverages your chosen AI model to understand requests and make decisions, searches your knowledge base for relevant information, and executes the appropriate tools to complete tasks.

Each Tray agent is a Tray project where these components work together to deliver intelligent automation that understands your business context and takes real actions across your tech stack.

Prerequisites

Copy

Workspace permissions required:

• Agent Builder feature enabled on your workspace (contact your workspace admin if you don't see the Agent Builder option)

• Project Editor role with the workspace

Billing considerations:

• Tray native AI model is included with your plan

• Data sources and tools consume standard workflow tasks from your plan allocation

• Bring Your Own (BYO) AI models (OpenAI, Anthropic, etc.) require separate billing with those providers

Getting started

Copy

1. Create your agent project On the Projects page, click "Add project" and select "Merlin Agent Builder". Choose from these options:

   1. Knowledge agent

      - Pre-configured for information retrieval and Q&A

   2. ITSM agent

      - Set up for IT service management tasks

   3. Support ticket agent

      - Designed for customer support workflows

   4. Blank agent

      - Start from scratch with full customization

   We recommend starting with one of the pre-configured agents ("agent accelerators") to understand how agents work before building custom ones. After selecting your agent type, enter a project name to create your agent.

   

2. Configure your agent Your agent dashboard contains everything you need to set up:

   • Agent scope

     - If using an accelerator, the default prompt is ready for testing. You can customize it later based on your specific needs.

   • Data sources

     - Connect a test data source - e.g. add a test folder from Google Drive with sample documents so your agent has knowledge to reference when answering questions. More information on Adding a data source

   • Tools

     - All agents include Knowledge Search Tool by default. Add more tools from templates or create custom ones as your use case requires More information on Tools

3. Test your agent Navigate to the "Test" tab to interact with your agent like your end users would. Ask questions about its capabilities or request information from your connected documents. Make sure any data sources have finished syncing before testing - you'll see the sync status in your data sources list.

4. Deploy your agent Your agent is automatically available via API. Find the deployment URL in "Interaction Channels" - it follows the format https://<id>-api.trayapp.io/ai-agent. Use this endpoint to integrate your agent into applications. For more details, see Interaction channels

   You can continue adding tools, data sources, and refining your agent scope as you expand functionality.

Configuring your agent

Copy

When you open an agent project, you'll see the agent dashboard.

This is your control center for:

Agent scope

Copy

Agent scope defines what your agent knows and how it behaves. Think of it as the agent's job description and personality guide.

If you're using one of the pre-configured agents accelerators (Knowledge agent, ITSM agent, or Support ticket agent), they come with predefined scope tailored for their specific use cases. This gives you a working starting point that you can customize based on your needs.

Be specific about the role:

1
You are a customer support agent for [Company Name], specializing in billing and subscription questions.

Define capabilities and limitations:

1
You can help with: 
2
- Billing questions and payment issues 
3
- Subscription upgrades/downgrades 
4
- Account access problems 
5

6
You should escalate to human agents: 
7
- Technical product issues 
8
- Refund requests over $100 
9
- Angry or frustrated customers

Set the tone:

1
Always be professional, helpful, and empathetic. Use clear, non-technical language.

Best practices for agent scope

Copy

Key prompting tips:

• Set behavioral expectations - Professional tone, structured responses, etc.

• Define limitations clearly - What the agent can't do helps users set appropriate expectations

• Use examples - Include sample interactions showing desired behavior: "When asked about pricing, respond with: 'I'll look up our current pricing information for you...'"

• Prioritize tasks - Tell the agent what to do first: "Always search the knowledge base before providing any factual information"

• Handle uncertainty - Specify what to do when information is unclear: "If you're unsure, ask clarifying questions rather than guessing"

• Set error handling - Define fallback behaviors: "If a tool fails, explain the issue and suggest alternative approaches"

• Include persona details - Describe the agent's personality: "Be friendly but professional, patient with technical questions, and proactive in offering help"

• Specify output format - Request structured responses: "For troubleshooting, use numbered steps. For explanations, use clear headings and bullet points"

• Keep it focused - While comprehensive, avoid overly complex instructions that might confuse the AI model. Aim for clear, actionable guidance that helps the agent serve your specific use case effectively.

AI model

Copy

Tray native model (default)

Copy

• Included with your plan

• Max context window: up to 1 million tokens

• Rate limited

• Best for: Getting started, testing

Bring Your Own Model options

Copy

You can connect your own API keys from third-party AI providers like OpenAI or AWS Bedrock to use their models instead of Tray's default model. Currently supported:

• AWS Bedrock

• OpenAI

• Azure AI

• Google Gemini

Setup steps for BYO models

1. Click "Change Model" in the AI Model section

2. Select your provider (OpenAI, Anthropic, etc.)

3. Click "Create New Authentication" or select existing

4. Enter your API key from the provider

5. Choose the specific model version

6. Click "Save"

Advanced settings

• Context window (token limit): Default is 32k tokens

  • Smaller windows (8k-16k) may be faster but limit conversation memory

  • Standard windows (64k-128k) handle typical conversations and documents well

  • Large windows (256k-500k) support extensive analysis and long conversations

  • Maximum supported: up to 1 million tokens for processing entire books, codebases, or complex multi-document workflows

• Temperature: Controls response creativity (0.0 = focused, 1.0 = creative). It controls how predictable versus varied the responses will be - lower values (0.0) make responses more consistent and factual, while higher values (1.0) make responses more varied and creative but potentially less reliable.

Recommended defaults

• For customer support: Tray native or GPT-4, temperature 0.3

• For content creation: Claude-3-Sonnet, temperature 0.7

• For data analysis: GPT-4-turbo, temperature 0.1

Data sources

Copy

Data sources provide knowledge to your agent without making live API calls for every query. This creates a searchable knowledge base that your agent can reference instantly.

Overview

Copy

1. Connect to a system (Google Drive, Confluence, etc.)

2. Tray syncs and processes your documents

3. Content is stored in a vector table (searchable Knowledge Base Core)

4. Agent searches this Knowledge Base to answer questions

Behind the scenes: adding a data source creates a sub-project with sync workflows. Data is stored in a vector table that enables semantic search, this is also handled by a specialized sub-project. All synced data goes into a single vector table per agent.

Supported data sources

Copy

• Google Drive - Documents, spreadsheets, presentations

• Gmail - Email threads and attachments

• Confluence - Wiki pages and documentation

• Sharepoint - Documents and sites

• Custom integrations - Via API connectors

Adding a data source

Copy

Step 1: Add and configure your source

1. In the Data sources tab, click "Add Data Source"

2. Select from available data source types

3. Authenticate with the selected service

4. Configure sync settings. Depending on the chosen Data source type, you will have to configure additional settings, e.g.:

• Folder / Confluence space selection: Choose what to sync (start narrow)

• Gmail Labels / Gmail Search Query / Jira JQL Query: Select appropriate search for items

• Include/exclude patterns

Step 2: Initial sync

• First sync may take several minutes for large datasets

• Check the sync status in the Data sources list for progress updates

• Embedded data will be saved in the vector table once processed

Best practices for data sources

Copy

• Start small: Sync one folder or space first to test

• Organize source content: Well-structured documents work better

• Use descriptive filenames: Helps the agent find relevant content

• Update regularly: Enable automatic sync for changing content

Troubleshooting sync issues

Copy

1. Check the data syncing sub-project workflows by navigating to Data source assets. You can access them by right clicking the three dot menu in the data sources list

2. Once in the data source sub-project, review execution logs for error messages

3. Verify permissions on source system

4. Test with a single document first

Configuration changes: Modifying sync settings may require re-processing all documents.

Tools

Copy

Tools are workflows that your agent can use to take actions. They're the "hands" of your agent - turning conversation into real business impact.

Tools from template

Copy

Tray offers pre-built tools for common actions:

• Send Email - Compose and send emails

• Create Jira Ticket - Log issues or requests

• Update Salesforce Record - Modify CRM data

• Slack Message - Send team notifications

• Google Calendar Event - Schedule meetings

For the full list, please check out our template library.

To add a tool from template:

1. In the Tools tab, click "Add tool" and choose "From Template"

2. Browse available tools by category

3. Click "Use template" on your choice

4. Configure required authentications (if applicable)

Important! The newly added tool will be automatically enabled and visible to the agent. If you need to do more configuration before enabling the tool, you can disable it.

Custom Tools

Copy

To add a custom tool:

1. In the Tools tab, click "Add tool" and choose "Custom Tool"

2. Enter a descriptive name for your tool

3. Write a comprehensive tool description

   This step is critical - without a clear description, your agent will never select this tool when deciding how to respond to requests. Your description should include:

   • when the agent should use this tool

   • what inputs the tool requires

   • what the expected outcome will be

   • why limitations or constraints

4. Click on the newly created tool to customize the underlying workflow

Important! The newly added tool will be automatically enabled and visible to the agent. If you need to do more configuration before enabling the tool, you can disable it.

Good descriptions

1
"Use this tool to send email notifications to customers about order status updates. Required: customer email, order number, status message."
2

3
"Create a Jira ticket for technical issues that need developer attention. Include problem description, affected systems, and priority level."
4

5
"Send a Slack message to the #alerts channel when urgent issues need immediate team attention."

Bad descriptions

1
"Sends emails" (too vague) 
2

3
"Use when needed" (no clear trigger) 
4

5
"Handles tickets" (unclear purpose)

Disabling tools

Copy

If you have tools in your agent project that you don't want to delete - perhaps they're still in development or temporarily not needed - you can disable them instead. When a tool is disabled, the agent won't see it or consider it when selecting tools to complete a task.

To disable a tool:

1. Go to your Tools list

2. Click the three-dot menu next to the tool you want to disable

3. Click "Disable tool"

Best practices for tools

Copy

• Start with templates before building custom tools

• Test extensively in different scenarios

• Use comprehensive descriptions that explain the action

• Handle errors gracefully with fallback options

• Limit tool count - too many options confuse the agent

Interaction channels

Copy

Interaction channels are how users connect with your agent. Currently supported:

• API endpoint

Coming soon:

• Slack app integration - Deploy agents directly into Slack channels

• Teams integration - Microsoft Teams bot deployment

API endpoint

Copy

1. Go to "Interaction Channels" tab

2. Copy the endpoint URL

3. Use standard HTTP POST requests

The response format is determined by your tool's workflow response schema. To preview the expected format, navigate to the tool's workflow and examine the response schema in the "Trigger Event Reply" step.

Integration examples:

• Embed in your helpdesk system

• Connect to chatbots or voice systems

• Integrate with mobile apps

• Power internal tools and dashboards

Testing

Copy

The built-in test environment lets you interact with your agent safely before deployment.

Using the test chat

Copy

1. Click the "Testing" tab

2. Type messages as if you were a real user

3. Watch how your agent responds

4. Check if tools are being used correctly

Example test scenarios

Copy

• Basic conversation - e.g. "Hello, what can you help with?"

• Knowledge questions - ask about content from your data sources

• Tool activation - request actions your tools can perform

• Edge cases - unclear requests, requests outside scope

• Error handling - what happens when tools fail?

Test agent responses show:

• The actual reply your users will see

• Whether knowledge base was searched

• Which tools were considered or used

• Processing time and token usage

Use logs to get additional insights into agent responses.

Test limitations

Copy

• No persistent history, each test session is independent

• Test conversations aren't saved

Logs

Copy

Agent interactions and executions are logged to help you monitor performance and debug issues. Unlike standard workflow logs that show input/output for each step, agent logs provide detailed insight into the AI decision-making process.

Log structure

Copy

Each agent request generates a detailed log containing:

• Prompt - The exact user prompt or query sent to the agent

• Reasoning - Detailed breakdown of how the agent processed the request:

  • Select tool - Shows the tool selection process, including:

    • What data was analyzed to make the decision

    • Which tool was selected and why

    • Parameters passed to the selected tool

  • Execute tool - The complete output returned by the executed tool

• Response - The final human-readable response sent back to the user