Paginate through Stripe Records

Workflow

Sales

Beginner

Use Template

This is a 'Workflow' template which means that it is a single standalone workflow.

Some workflow templates can be modified to work with other workflow templates - e.g. to convert a data sync between two services from uni-directional to bi-directional

Overview

Copy

This template demonstrates how to pull records from Stripe and process them in batches using pagination. Pagination is a standard method required when retrieving large datasets (e.g., transactions, invoices, customers) from Stripe using the Stripe API.

When fetching large amounts of data, Stripe returns a has_more flag and a starting_after cursor, which are used to retrieve the next set of records. A pagination loop continues pulling records in batches until has_more is false.

For example, if there are 10,000 transactions, they can be retrieved in 10 batches of 1,000 rather than making 10,000 individual API calls, which would be inefficient and could exceed rate limits.

End Result

Copy

The retrieved Stripe data will be available in structured batches for further processing. A placeholder script step is included in this workflow to demonstrate where additional logic can be applied to handle the results.

Prerequisites

Copy

Before using this workflow, ensure the following:

• You have a Stripe account with API access.

• You can authenticate as an API user with sufficient permissions to fetch the necessary records.

Getting Live

Copy

To configure this workflow:

1. Authenticate to Stripe

   • Add the Stripe API authentication to the relevant connector step within the workflow.

   • Ensure the authentication has access to the required records.

2. Identify a Trigger

   • If this is a standalone process, use a Manual Trigger to start the workflow on demand.

   • If it should run automatically at intervals, use a Scheduled Trigger (e.g., daily or hourly).

   • If it should start based on an event in another system, use a Webhook Trigger or a Service Trigger if available.

3. Define the Data to Retrieve

   • In the List Invoices (Stripe-1) step, specify the fields and filters to refine the dataset.

   • If retrieving different record types (e.g., invoices, customers), replace the List Invoices operation with the appropriate API call.

   • If using a raw HTTP request, update the pagination logic accordingly to handle Stripe’s starting_after cursor.

4. Processing the Retrieved Data

   • If the process must run quickly, consider using a Call/Callable Workflow to handle batch processing efficiently.

   • Run the workflow manually by clicking Run workflow in the builder's bottom-left corner.

Workflow Logic

Copy

The workflow follows this logic:

1. A loop runs until all records have been retrieved.

2. Within the loop:

   • The starting_after cursor from the last batch is retrieved (if it's the first run, this is empty).

   • The Stripe List Invoices operation fetches a batch of records using the cursor.

   • The batch is processed or stored for further use.

3. A boolean condition checks if has_more is true from the most recent Stripe API response.

   • If false, the loop stops.

   • If true, the loop continues with the updated starting_after value.

4. The

   starting_after value is saved for the next iteration.

Implementation Notes

Copy

• The Read Me script step is a placeholder. Replace it with logic specific to your use case.

• Potential ways to use the fetched data:

  • Store each batch in a database or CSV file.

  • Send the records to another system (e.g., CRM, BI tool).

  • Use callable workflows for parallel processing to handle large volumes efficiently.