# Fine-Tuning AI Models with Enterprise Data

Learn how to fine-tune AI models using data extracted from enterprise systems like Jira, ServiceNow, or Zendesk to create specialized support and knowledge agents.

## Overview

Fine-tuning allows you to create specialized AI models that understand your organization's specific domain knowledge, terminology, and problem-solving approaches. By training models on historical support tickets, documentation, or domain-specific conversations, you can build AI assistants that provide more accurate, contextual responses.

This guide demonstrates how to build a complete fine-tuning pipeline using Tray workflows that:

* Extracts data from enterprise systems (Jira Service Desk, Zendesk, ServiceNow, etc.)
* Processes and transforms raw data into training format
* Uses AI to intelligently extract question-answer pairs
* Uploads training data to OpenAI's fine-tuning API
* Monitors fine-tuning job progress and notifies on completion

## Use Cases

Fine-tuning is particularly valuable for:

* **IT Support Agents**: Train models on historical ITSM tickets to handle common technical issues
* **Customer Support**: Create support bots that understand your product-specific terminology and solutions
* **Knowledge Management**: Build agents that can answer questions using your organization's internal documentation style
* **Domain Expertise**: Develop specialized assistants for legal, medical, financial, or other regulated industries
* **Multi-language Support**: Fine-tune models to better handle company-specific translations and localized content

## Architecture Overview

The solution consists of **three interconnected workflows** that orchestrate the entire fine-tuning process:

![Three-workflow architecture overview](https://tray.ai/documentation/images/platform/artificial-intelligence/ai-use-cases/fine-tuning-models/architecture-overview.png)

### Workflow 1: Fine Tune ITSM Support Data (Main Orchestrator)

**Purpose**: End-to-end orchestration of the fine-tuning process

**Key Steps**:

1. Manual trigger initiates the process
2. Calls "Generate Training Data" workflow and receives the generated JSONL filename
3. Retrieves the training file from Tray file storage
4. Uploads file to OpenAI Files API
5. Starts the fine-tuning job with target model
6. Polls for job completion every 10 seconds
7. Sends email notification when complete

![Main orchestration workflow canvas](https://tray.ai/documentation/images/platform/artificial-intelligence/ai-use-cases/fine-tuning-models/orchestration-workflow.png)

### Workflow 2: Generate Training Data

**Purpose**: Extract and process source data into JSONL format

**Key Steps**:

1. Callable trigger receives parameters from orchestrator
2. Pagination loop handles Jira's paginated API responses
3. For each issue, calls "Process Jira Issue" workflow
4. Filters for resolved issues only (`solved === true`)
5. Appends training examples to JSONL file
6. Returns filename to orchestrator workflow

![Data generation workflow with pagination](https://tray.ai/documentation/images/platform/artificial-intelligence/ai-use-cases/fine-tuning-models/data-generation-workflow.png)

### Workflow 3: Process Jira Issue

**Purpose**: Convert individual records into fine-tuning format using AI

**Key Steps**:

1. Receives `issue_id` parameter from caller
2. Fetches full issue details from Jira
3. Extracts relevant fields (description, status, comments) using JSON transformer
4. Sends to OpenAI with specialized extraction prompt
5. Uses function calling to enforce structured JSON output
6. Returns formatted training data and resolution status

![Issue processing workflow with AI extraction](https://tray.ai/documentation/images/platform/artificial-intelligence/ai-use-cases/fine-tuning-models/issue-processing-workflow.png)

## Prerequisites

> **Required Components:** Before building the fine-tuning pipeline, ensure you have:- **OpenAI API Account**: With fine-tuning access enabled (check your organization's tier)
> - **Data Source Authentication**: Jira Cloud, Zendesk, ServiceNow, or other ITSM system configured in Tray
> - **Email Connector**: Configured for completion notifications
> - **Tray File System Access**: Available in all Tray accounts
> - **Minimum Training Data**: At least 50-100 resolved tickets with clear Q\&A conversations (OpenAI recommendation)

> **Warning:** Fine-tuning costs vary based on model and training data size. Monitor your OpenAI usage dashboard during fine-tuning jobs. A typical job with 100 examples may cost between $10-50 depending on token count and model selection.**Current pricing** (as of 2025): Check OpenAI's pricing page for `gpt-4.1-2025-04-14` fine-tuning rates.

## Understanding JSONL Format for Fine-Tuning

OpenAI's fine-tuning API requires training data in **JSONL (JSON Lines)** format, where each line is a complete JSON object representing one training example:

```json
{"messages": [{"role": "user", "content": "How do I reset my password?"}, {"role": "assistant", "content": "To reset your password, go to Settings &gt; Security &gt; Change Password. Enter your current password, then your new password twice to confirm."}]}
{"messages": [{"role": "user", "content": "Why is my API returning 401 errors?"}, {"role": "assistant", "content": "A 401 error indicates authentication failure. Check that your API key is valid and hasn't expired. You can generate a new key in the Admin console under API Access."}]}
{"messages": [{"role": "user", "content": "Can I export my data to CSV?"}, {"role": "assistant", "content": "Yes, use the Export feature under Reports. Select your date range and fields, then click Export to CSV. The file will be emailed to you within a few minutes."}]}
```

### Key Format Requirements

* **One training example per line**: Each line must be a complete, valid JSON object
* **Messages array**: Contains exactly 2 messages for basic fine-tuning (user question + assistant response)
* **Role specification**: `"role"` must be either `"user"` or `"assistant"`
* **Content field**: `"content"` contains the actual text
* **No trailing commas**: Unlike JSON arrays, JSONL files don't have commas between lines
* **Character escaping**: Special characters must be properly escaped (use `&lt;` and `&gt;` for angle brackets in email addresses)

## Setting Up the Workflows

### Step 1: Configure the Main Orchestration Workflow

Create a new workflow named "Fine Tune ITSM Support Data" with the following configuration:

### Trigger Configuration

**Manual Trigger**:

* Add a Manual Trigger to initiate the process on-demand
* Optionally add input parameters for:
  * `project_key`: Jira project to process (e.g., "SUPPORT")
  * `model`: Target fine-tuning model (default: `gpt-4.1-2025-04-14`)
  * `notification_email`: Email for completion notifications

### Call Workflow Step

**Call Workflow Connector**:

* Select "Generate Training Data" as the workflow to call
* Enable "Wait for response" option
* Map any input parameters (project key, date ranges, filters)
* Store the response in a variable: `$.steps.call-workflow-1.response.filename`

### File Retrieval

**File System Connector - Get File**:

* Operation: "Get file"
* File path: Map from Call Workflow response `$.steps.call-workflow-1.response.filename`
* This retrieves the generated JSONL file from Tray's file storage

### OpenAI Upload

**HTTP Client Connector**:

* Method: `POST`
* URL: `https://api.openai.com/v1/files`
* Headers:
  * `Authorization`: `Bearer {your_openai_api_key}`
  * `Content-Type`: `multipart/form-data`
* Body (form-data):
  * `file`: Map from File System step output `$.steps.file-system-1.file`
  * `purpose`: `fine-tune` (literal string)
* Store response: The `id` field contains the uploaded file ID

### Step 2: Start the Fine-Tuning Job

### Job Initialization

**OpenAI Raw HTTP Connector**:

* Method: `POST`
* Endpoint: `https://api.openai.com/v1/fine_tuning/jobs`
* Authentication: Use OpenAI authentication configured in Tray
* Body (JSON):

```json
{
  "training_file": "$.steps.http-client-1.body.id",
  "model": "gpt-4.1-2025-04-14"
}
```

* Store the response job ID: `$.steps.openai-raw-1.body.id`

> **Tip:** You can add optional parameters like `hyperparameters` to customize the fine-tuning process. See OpenAI's fine-tuning documentation for available options.

### Status Polling Loop

**Loop Forever Connector**:

* Creates an infinite loop for polling job status
* Contains the following steps inside the loop:

1. **OpenAI Raw HTTP GET Request**:
   * Method: `GET`
   * Endpoint: `https://api.openai.com/v1/fine_tuning/jobs/{job_id}`
   * Map job\_id from: `$.steps.openai-raw-1.body.id`

2. **Boolean Condition Connector**:
   * Check if status is in terminal states:
   * Condition: `$.steps.openai-raw-2.body.status === 'succeeded' || $.steps.openai-raw-2.body.status === 'failed' || $.steps.openai-raw-2.body.status === 'cancelled'`

3. **Break Loop Connector** (conditional):
   * Only executes if Boolean returns `true`
   * Exits the polling loop when job reaches terminal state

4. **Delay Connector**:
   * Duration: 10 seconds (10000 milliseconds)
   * Prevents excessive API calls and rate limiting

![Polling loop configuration](https://tray.ai/documentation/images/platform/artificial-intelligence/ai-use-cases/fine-tuning-models/polling-loop.png)

> **Info:** **Why polling?** OpenAI fine-tuning jobs don't provide webhooks for completion notifications. Polling every 10 seconds is a balanced approach that provides timely notifications without hitting rate limits. For jobs with larger datasets (1000+ examples), consider increasing the polling interval to 30-60 seconds.

### Step 3: Build the Data Generation Workflow

Create a workflow named "Generate Training Data" with a Callable Trigger:

### Pagination Setup

**Storage Connector - Initialize**:

* Create variable: `next_page_token`
* Initial value: `null`

**Loop Forever Connector**:

* Contains pagination logic to fetch all issues

Inside the loop:

1. **Jira Get Issues** (or equivalent for your system):
   * Project: Map from trigger input
   * Max results: 10 (batch size)
   * Start at: Use `next_page_token` from storage

2. **Storage Connector - Set**:
   * Update `next_page_token` with response value

3. **Boolean Condition**:
   * Check if `$.steps.jira-1.isLast === true`

4. **Break Loop** (conditional):
   * Execute when no more pages remain

### Issue Processing

**Loop List Connector**:

* Loop through: `$.steps.jira-1.issues`

Inside the loop:

1. **Call Workflow Connector**:
   * Workflow: "Process Jira Issue"
   * Pass parameter: `issue_id` from current loop item
   * Wait for response

2. **Boolean Condition**:
   * Check if: `$.steps.call-workflow-2.response.solved === true`

3. **File System - Append to File** (conditional):
   * Only execute for resolved issues
   * File path: `{execution_uuid}.jsonl`
   * Content: `$.steps.call-workflow-2.response.training_data`
   * Format: Ensure each record is on a new line

### Response

**Callable Response Connector**:

* Return object:

```json
{
  "filename": "{execution_uuid}.jsonl",
  "total_examples": "$.steps.loop-list-1.iterations",
  "resolved_count": "count_of_resolved_issues"
}
```

### Step 4: Create the Issue Processing Workflow

Create a workflow named "Process Jira Issue" with a Callable Trigger that accepts `issue_id`:

### Data Extraction

**Jira Get Issue Connector**:

* Issue ID: Map from trigger `$.trigger.issue_id`

**JSON Transformer (JSONata)**:

* Expression:

```jsonata
{
  "issue": $.fields.description,
  "status": $.fields.status.name,
  "comments": $.fields.comment.comments.body
}
```

* This extracts only the relevant fields needed for AI processing

### AI Extraction

**OpenAI Chat Completion Connector**:

* Model: `gpt-4`
* Temperature: `0.28` (low temperature for consistent extraction)
* Messages:
  * System message: See full system prompt in accordion below
  * User message: Pass the transformed JSON from previous step
* **Tool Choice**: Set to `save_training_data` (forces function calling)
* Tools: Define the function schema (see accordion below)

### Full System Prompt for AI Extraction

```
# System Prompt: Jira Service Desk Ticket to Fine-tuning Data Converter

You are a specialized AI assistant that converts Jira Service Desk support tickets into training data for OpenAI model fine-tuning.

## Your Task

Analyze the provided Jira ticket and its comments to extract:
1. The user's original question or problem
2. The assistant's response that resolved the issue
3. Whether the problem was successfully solved

## Instructions

### 1. Extract User Question
- Identify the core problem or question from the ticket description and initial comments
- Combine multiple user messages if they clarify or expand on the original issue
- Keep the question clear, concise, and self-contained
- Remove any ticket metadata, formatting artifacts, or irrelevant context

### 2. Extract Assistant Response
- Identify the response from support staff that successfully resolved the issue
- Include only the solution, not intermediate troubleshooting steps unless they're essential
- Ensure the response is complete and actionable
- Remove any ticket-specific references (ticket numbers, internal links, etc.)
- Maintain technical accuracy and clarity

### 3. Determine Resolution Status
- Set `solved` to `true` if:
  - The ticket was marked as resolved/closed
  - The user confirmed the solution worked
  - The solution appears complete and the user stopped responding
- Set `solved` to `false` if:
  - The ticket is still open
  - The user reported the solution didn't work
  - The conversation was abandoned without resolution
  - The issue was escalated or transferred

## Quality Guidelines

- **Clarity**: Ensure both question and answer are understandable without ticket context
- **Completeness**: The assistant response should fully address the user question
- **Generalization**: Remove company-specific or ticket-specific details where possible
- **Accuracy**: Preserve technical details and solution steps accurately
- **Brevity**: Be concise but don't omit critical information

## Edge Cases

- **Multiple issues in one ticket**: Create training data for the primary resolved issue
- **Partial solutions**: Only include if the main problem was solved (solved=true)
- **Back-and-forth conversations**: Consolidate into one coherent Q&A pair
- **Unclear resolution**: Mark as solved=false and do your best to extract useful training data
```

### Function Schema for Structured Output

```json
{
  "name": "save_training_data",
  "description": "Saves the extracted Jira ticket conversation as fine-tuning training data in JSON format along with resolution status",
  "parameters": {
    "type": "object",
    "required": ["training_data", "solved"],
    "properties": {
      "training_data": {
        "type": "object",
        "description": "The formatted conversation data for OpenAI fine-tuning",
        "required": ["messages"],
        "properties": {
          "messages": {
            "type": "array",
            "description": "Array of message objects containing the user question and assistant response",
            "items": {
              "type": "object",
              "required": ["role", "content"],
              "properties": {
                "role": {
                  "type": "string",
                  "enum": ["user", "assistant"],
                  "description": "The role of the message sender"
                },
                "content": {
                  "type": "string",
                  "description": "The content of the message"
                }
              }
            },
            "minItems": 2,
            "maxItems": 2
          }
        }
      },
      "solved": {
        "type": "boolean",
        "description": "Indicates whether the user's problem was successfully resolved"
      }
    }
  }
}
```

### Parse and Return

**Object Helpers - Parse JSON**:

* Input: `$.steps.openai-1.choices[0].message.tool_calls[0].function.arguments`
* This parses the function call arguments into a usable object

**Callable Response Connector**:

* Return:

```json
{
  "training_data": "$.steps.object-helpers-1.result.training_data",
  "solved": "$.steps.object-helpers-1.result.solved"
}
```

## Customizing for Different Data Sources

The solution architecture is designed to be adaptable to various enterprise systems:

### Adapting for Zendesk

Replace the Jira-specific steps with Zendesk equivalents:

* **Trigger**: Use Zendesk's "Search Tickets" operation with status filter `solved`
* **Pagination**: Zendesk uses `next_page` URL in responses
* **Field Mapping**: Adjust JSON transformer:

```jsonata
{
  "issue": $.description,
  "status": $.status,
  "comments": $.comments[type='Comment'].body
}
```

### Adapting for ServiceNow

* **API Endpoint**: Use ServiceNow's Table API for incidents
* **Query**: Filter for `incident_state=6` (Resolved) and `incident_state=7` (Closed)
* **Field Mapping**:

```jsonata
{
  "issue": $.short_description & " " & $.description,
  "status": $.incident_state,
  "comments": $.comments
}
```

### Adapting for Salesforce Cases

* **SOQL Query**: `SELECT Description, Status, Comments__c FROM Case WHERE Status = 'Closed' AND IsSolution = true`
* **Field Mapping**:

```jsonata
{
  "issue": $.Description,
  "status": $.Status,
  "comments": $.Comments__c
}
```

### Adapting for Custom Knowledge Bases

For documentation or knowledge base content:

* **Source**: Confluence, Notion, or custom documentation platforms
* **Data Structure**: Map article title to "user question" and article body to "assistant response"
* **Filtering**: Only include verified, published content
* **System Prompt Modification**: Adjust to focus on extracting key points rather than Q\&A pairs

## Monitoring and Validation

### Understanding Fine-Tuning Job Statuses

OpenAI fine-tuning jobs progress through several states:

| Status             | Description                                | Action Required                      |
| ------------------ | ------------------------------------------ | ------------------------------------ |
| `validating_files` | Initial validation of training data format | Wait                                 |
| `queued`           | Job is queued for processing               | Wait                                 |
| `running`          | Fine-tuning is actively in progress        | Wait                                 |
| `succeeded`        | Job completed successfully                 | Use the fine-tuned model             |
| `failed`           | Job encountered an error                   | Check error details, fix data, retry |
| `cancelled`        | Job was manually cancelled                 | Review and restart if needed         |

### Validating Training Data Quality

Before uploading to OpenAI, validate your training data:

> **Quality Checks:** **Automated Validation**:- Minimum examples: At least 50 training examples (OpenAI minimum is 10, but 50+ yields better results)
> - Format validation: Each line is valid JSON with required `messages` array
> - Character encoding: All text is UTF-8 encoded
> - Token count: Each example is under the model's context window (typically 4096-8192 tokens)**Manual Spot Checks**:- Questions are clear and self-contained
> - Answers are complete and accurate
> - No PII or sensitive data included
> - Examples represent diverse scenarios

### Adding Validation Steps to Your Workflow

Between data generation and OpenAI upload, add validation:

**Script Connector (Validation)**:

```javascript
const lines = inputs.file_content.split('\n').filter(l => l.trim());

const validation = {
  total_lines: lines.length,
  valid_json: 0,
  invalid_json: 0,
  errors: []
};

lines.forEach((line, idx) => {
  try {
    const obj = JSON.parse(line);
    if (!obj.messages || obj.messages.length !== 2) {
      validation.errors.push(`Line ${idx + 1}: Invalid messages array`);
    } else {
      validation.valid_json++;
    }
  } catch (e) {
    validation.invalid_json++;
    validation.errors.push(`Line ${idx + 1}: ${e.message}`);
  }
});

return validation;
```

## Best Practices

### Data Quality Filtering

> **Check:** **High-Quality Training Data Characteristics**:- Issues are fully resolved (status is "Closed" or "Resolved")
> - Clear user question with complete assistant response
> - Conversations are relevant to your target use case
> - No duplicate or near-duplicate examples
> - Balanced representation of issue types
> - Technical accuracy verified

### Optimal Training Data Volumes

OpenAI's recommendations for fine-tuning:

* **Minimum**: 10 examples (absolute minimum)
* **Recommended**: 50-100 examples for basic tasks
* **Ideal**: 200-500 examples for complex domains
* **Maximum**: No hard limit, but diminishing returns after 1000+ examples

> **Tip:** Start small! Fine-tune with 100 carefully curated examples first. Evaluate the results, then incrementally add more training data if needed. Quality always trumps quantity.

### System Prompt Engineering

The system prompt for AI extraction is critical:

**Key Elements**:

* Clear role definition ("You are a specialized assistant...")
* Specific task description (extract question, answer, status)
* Output format requirements (JSON structure via function calling)
* Quality guidelines (clarity, completeness, accuracy)
* Edge case handling (multiple issues, unclear resolution)

**Iteration Strategy**:

1. Test with 5-10 sample tickets manually
2. Review extracted Q\&A pairs for quality
3. Refine prompt based on common issues
4. Re-test and iterate
5. Only then run at scale

### Handling Edge Cases

**Multiple Issues in One Ticket**:

* Use AI to identify the primary issue
* Create separate training examples if multiple distinct problems were solved

**Unresolved or Partially Resolved Tickets**:

* Filter out with `solved === false`
* Don't include in training data unless specifically training for escalation scenarios

**PII and Sensitive Data**:

* Add PII redaction step before AI extraction
* Use [Merlin Guardian](https://tray.ai/documentation/connectors/artificial-intelligence/merlin-guardian) to automatically mask sensitive information (names, emails, SSNs, etc.)
* The masked data is safe for AI processing, and can be unmasked after extraction if needed
* Consider using synthetic or anonymized data for demonstration purposes

**Non-English Content**:

* Fine-tune separate models for each language
* Or use translation services before extraction (less ideal)
* Ensure your system prompt specifies the expected language

## Troubleshooting

### Common Issues and Solutions

### Fine-Tuning Job Fails with 'Invalid Training File' Error

**Symptoms**: Job status immediately changes to `failed` with error about training file format.

**Causes**:

* JSONL format is incorrect (not one JSON object per line)
* Missing required fields in training data
* Invalid character encoding

**Solutions**:

1. Validate JSONL format: Each line must be complete, valid JSON
2. Check for trailing commas or formatting issues
3. Ensure `messages` array contains exactly 2 items with correct roles
4. Verify UTF-8 encoding without BOM (Byte Order Mark)
5. Test with a small sample file first (5-10 examples)

**Example of common mistake**:

```json
// WRONG - Array format
[
  {"messages": [...]},
  {"messages": [...]}
]

// CORRECT - JSONL format (no commas between lines)
{"messages": [...]}
{"messages": [...]}
```

### AI Extraction Returns Inconsistent Results

**Symptoms**: Some tickets are processed correctly, others return malformed data or wrong content.

**Causes**:

* System prompt is too vague or ambiguous
* Temperature setting is too high (more randomness)
* Training examples in prompt don't cover edge cases
* Function schema doesn't enforce strict validation

**Solutions**:

1. Lower temperature to 0.2 or 0.28 for more consistent extraction
2. Use `tool_choice` with specific function name to force structured output
3. Add more specific instructions in system prompt for edge cases
4. Include example scenarios in the prompt (few-shot learning)
5. Test with diverse ticket types before scaling

### Polling Loop Never Completes

**Symptoms**: Workflow keeps polling indefinitely, never breaking the loop.

**Causes**:

* Boolean condition is incorrectly configured
* Job ID is not being correctly referenced
* API authentication is failing silently
* Job actually failed but status check is not working

**Solutions**:

1. Debug the polling step: Check what status value is actually returned
2. Verify job ID mapping: Ensure `$.steps.openai-raw-1.body.id` is correct
3. Test the Boolean condition separately with known status values
4. Add debug logging to see actual status values during polling
5. Check OpenAI dashboard directly to verify job status
6. Add a timeout mechanism: Break loop after X iterations (e.g., 360 for 1 hour)

### Too Many Low-Quality Training Examples

**Symptoms**: Fine-tuned model doesn't perform better than base model, or gives inconsistent results.

**Causes**:

* Including unresolved tickets in training data
* AI extraction is not filtering properly
* Source tickets don't have clear resolutions
* Too many duplicate or similar examples

**Solutions**:

1. Add stricter filtering: Only include tickets with confirmed resolution
2. Implement similarity detection to remove duplicates
3. Add quality scoring in the processing workflow
4. Manually review a sample of extracted examples
5. Consider adding a human-in-the-loop review step for critical industries

**Quality Filtering Example**:

```javascript
// Add to Process Jira Issue workflow
const qualityScore = calculateQuality({
  hasUserQuestion: inputs.issue && inputs.issue.length > 20,
  hasAssistantResponse: inputs.comments && inputs.comments.length > 50,
  isResolved: inputs.status === 'Resolved' || inputs.status === 'Closed',
  notDuplicate: !isDuplicate(inputs.issue)
});

// Only proceed if quality score is high
if (qualityScore &lt; 0.7) {
  return { solved: false, reason: 'Low quality score' };
}
```

### Training File Upload Fails

**Symptoms**: HTTP Client step returns error when uploading file to OpenAI.

**Causes**:

* Invalid OpenAI API key
* File is too large (exceeds API limits)
* Incorrect content-type header
* Network or authentication issues

**Solutions**:

1. Verify API key is valid and has fine-tuning permissions
2. Check file size: OpenAI has limits (typically 512MB for JSONL files)
3. Ensure `Content-Type: multipart/form-data` is set correctly
4. Use Tray's built-in OpenAI connector instead of raw HTTP Client if available
5. Test file upload manually using curl or Postman first
6. Split large training files into smaller batches if needed

## Cost Optimization

### Estimating Fine-Tuning Costs

OpenAI charges based on:

* **Training tokens**: Number of tokens in your training data × number of epochs
* **Base model**: `gpt-4.1-2025-04-14` costs more than `gpt-3.5-turbo`
* **Usage fees**: Fine-tuned model usage charges (input + output tokens)

**Example calculation** (using hypothetical rates):

* 100 training examples
* Average 200 tokens per example = 20,000 tokens total
* 3 epochs (default) = 60,000 training tokens
* At $0.008 per 1K tokens = \~$0.48 for training

> **Tip:** Use the cheaper `gpt-3.5-turbo` for initial testing and prototyping. Only upgrade to `gpt-4.1-2025-04-14` if you need the performance improvement. The cheaper model often performs well for domain-specific tasks.

### Reducing Costs

**Strategies**:

1. **Curate data carefully**: 100 high-quality examples > 500 mediocre ones
2. **Reduce epochs**: Set `hyperparameters.n_epochs` to 2 instead of default 3
3. **Filter aggressively**: Only include truly resolved, clear examples
4. **Test with small batches**: Start with 50 examples, evaluate, then scale
5. **Use base models when sufficient**: Don't fine-tune if prompt engineering achieves good results

## Next Steps

Once your fine-tuning job succeeds:

1. **Test the Fine-Tuned Model**:
   * Use the model ID from the job response (e.g., `ft:gpt-4.1-2025-04-14:your-org:custom-suffix:job-id`)
   * Create a test workflow with OpenAI Chat Completion using your fine-tuned model
   * Compare responses to the base model on sample questions

2. **Deploy in Production**:
   * Integrate the fine-tuned model into your support chat workflows
   * Build a callable workflow that uses the model for answering user questions
   * Add fallback logic to use base model if fine-tuned model is unavailable

3. **Iterate and Improve**:
   * Collect feedback on model responses
   * Identify areas where the model underperforms
   * Generate additional training data for weak areas
   * Re-run fine-tuning with expanded dataset

4. **Monitor Performance**:
   * Track model usage and costs in OpenAI dashboard
   * Implement logging to capture model responses
   * Set up alerting for unexpected behavior or costs

## Additional Resources

* [OpenAI Fine-Tuning Documentation](https://platform.openai.com/docs/guides/fine-tuning)
* [Tray OpenAI Connector Documentation](https://tray.ai/documentation/connectors/artificial-intelligence/openai)
* [Building AI Knowledge Agents](https://tray.ai/documentation/platform/artificial-intelligence/intelligent-automations)
* [Vector Storage for RAG Pipelines](https://tray.ai/documentation/platform/artificial-intelligence/vector-storage)
* [Data Protection and Security](https://tray.ai/documentation/platform/enterprise-core/security-compliance/data-protection-commitment)

> **Note:** This documentation provides a comprehensive template for fine-tuning AI models with Tray. Adapt the workflows and configurations to match your specific data sources, organizational requirements, and use cases.
