Skip to content
Scrapfly logo

Connectors / Data and analytics · HTTP Connector

Automate Scrapfly with Tray.ai

Run a batch of scrapes and start a crawl in Scrapfly, and start a workflow when Scrapfly sends “Crawler started”.

What can you do with Scrapfly on Tray.ai?

Scrapfly is a web scraping platform with APIs for fetching pages through managed proxies and browsers, taking screenshots, extracting structured data and crawling whole sites. Tray.ai connects to Scrapfly through the HTTP Connector, so your workflows can call the Scrapfly API alongside the rest of your stack.

  • When Scrapfly sends “Crawler finished”, load the crawled pages into your search index.
  • Alert the data team in your team chat when Scrapfly sends “Crawler error”.
  • Capture a page screenshot of your pricing page after each release and file it for compliance records.
  • Extract data with a prompt from supplier pages and update product details in your catalog.
Connects through
HTTP Connector
Authentication
An API key in the query string or an API key in a request header
Base URL
https://api.scrapfly.io

Set it up

Connect Scrapfly to Tray.ai

Each step uses Scrapfly's own API details, from its documentation.

Step 1

Save Scrapfly's API key

Save the key as a Tray.ai authentication and add it to the query string from $.auth, so it never sits in the workflow itself.

Step 2

Point the HTTP Client at the API

Add an HTTP Client step and use https://api.scrapfly.io as the base URL. Or paste a cURL example from Scrapfly's API docs, and Tray.ai fills in the method, URL, and headers.

Step 3

Start with “Get crawl status”

It's one of 11 documented actions, under Crawls. The response comes back as data the next step can map into your other apps.

Step 4

Start workflows from Scrapfly events

Give Scrapfly the URL of a Tray.ai webhook trigger. “Crawler started” can then start a workflow, and so can 5 other events.

Workflows to build with Scrapfly

Each one uses actions and events from Scrapfly's own API documentation.

Load Scrapfly crawl contents into Snowflake

When

Crawler finished

Then

  1. Get crawl contents for the finished crawl
  2. Load the page content into a table in Snowflake

With Snowflake

Alert engineering in Slack when a Scrapfly crawl errors

When

Crawler error

Then

  1. Get crawl status for the crawl
  2. Post the error details to the data channel in Slack

With Slack

Enrich new Salesforce accounts from their website

When

An account is created in Salesforce

Then

  1. Scrape a web page for the account's website
  2. Extract data with a prompt to get industry and size
  3. Update the account in Salesforce

With Salesforce

Apps you can build on Scrapfly data

Internal tools your team can build with an AI assistant, each one using Scrapfly calls from its API documentation.

Crawl health monitor

Logs crawler starts, finishes, errors and failed URLs as they happen, with the status of each crawl.

Crawl content review

Lists the URLs a crawl visited and pulls structured data from their contents with a template.

Give your AI agents Scrapfly tools

Build an MCP server on Scrapfly in Agent Gateway and choose which actions your agents can call. Every call runs with access control and an audit trail. How Agent Gateway works

  • Scrape a web page
  • Start a crawl
  • Extract data with a template
  • Run a batch of scrapes
  • Capture a page screenshot
  • Get crawl status
  • List crawled URLs
  • Get crawl contents

What the Scrapfly API covers

3 resources and 11 documented actions, each one a call the HTTP Connector can make from a Tray.ai workflow. Connecting it takes a few clicks: here's how.

Scraping

  • Scrape a web page
  • Run a batch of scrapes
  • Capture a page screenshot

Crawls

  • Start a crawl
  • Get crawl status
  • List crawled URLs
  • Get crawl contents
  • Cancel a crawl

Extraction

  • Extract data with a template
  • Extract data with a prompt
  • Extract data with a model

Events Scrapfly can send to a Tray.ai webhook trigger

  • Crawler started
  • Crawler finished
  • Crawler cancelled
  • Crawler error
  • URL visited
  • URL failed

Checked against Scrapfly's own documentation on 2026-10-22

scrapfly.io/docs/scrape-api/getting-started

Frequently asked questions

What can a Tray.ai workflow do in Scrapfly?

+

The Scrapfly API documents 11 actions across 3 resources (Scraping, Crawls, Extraction). For example: Scrape a web page; Start a crawl; Extract data with a template. A workflow can make any documented call through the HTTP Connector.

How does Tray.ai authenticate with Scrapfly?

+

The Scrapfly API uses an API key in the query string and an API key in a request header. You save it once as a Tray.ai authentication, and the HTTP Connector adds it to every request.

Which Scrapfly events can start a Tray.ai workflow?

+

Scrapfly sends 6 events to a webhook URL, for example: Crawler started; Crawler finished; Crawler cancelled; Crawler error; URL visited; URL failed. A Tray.ai webhook trigger receives them and starts a workflow.

See Scrapfly working against your stack.

We'll walk through a tailored demo with your systems plugged in.

Great — a couple more details

This helps us connect you with the right person. Optional, and you can skip it.