Integration versioning without breaking customers

On Wednesday, the team ships a new version of the Salesforce integration with a required setting for sync direction. New customers see it in setup and pick an option. The 380 customers who set up before it existed have no value for it, and every one of their syncs fails at 2 p.m. Support spends Thursday sending the same email, and engineering spends it working out how to go back to a version nobody exported.

Every embedded integration changes after launch, with new fields, new settings and fixes. An internal integration has one user. A customer integration has hundreds, each with their own settings, and a change reaches all of them at once.

What is integration versioning?

Integration versioning is how a software company releases changes to an integration that many customers run. Each new version is tested the way a customer would use it. Before publishing, the team knows whether the change needs anything from existing customers. Changes that do go out in waves, and every published version is kept so going back is quick.

Why one change breaks every customer

1. Versions are edited live. There’s no separate environment, so the change and the test both happen in production.
2. Nobody checks whether the change needs the customer. A new connection, a new permission or a new required setting needs every existing customer to act, and the release treats it like a bug fix.
3. New settings have no default. A setting that could have had a default blocks every existing customer instead.
4. Everyone gets the change at once. A mistake reaches four hundred customers in the same minute.
5. Old versions aren’t kept. Going back means rebuilding from memory.

What it costs

A bad release hits every customer on the same afternoon, and support takes the calls. Teams that have lived through one start shipping less often, so customers wait longer for the fixes and improvements they asked for.

What to fix first

1. Keep development and production separate. Move versions between them by export and import, and never edit the live version.
2. Test as a customer. Run the new version through the real setup screens, in the development environment, with a test account.
3. Sort every change before publishing. If it adds a connection, changes a permission or adds a required setting, existing customers must upgrade. Everything else can go to everyone.
4. Give every new setting a default. A default turns a change that needs every customer into one that needs none.
5. Release changes that need the customer in waves. Start with a group chosen on purpose, watch failure rates, then move on to the rest.
6. Keep every version, with what changed. Record the date, the change, whether it needs the customer and how to go back.

The guide to rolling out integration changes to every customer has the prompts for each step.

What it looks like at Cvent

Cvent moved more than half of its nearly 1,000 legacy integrations, across more than 575 clients, from its old platform to Tray.ai in six months. With those integrations on Tray.ai, it now rolls out new integrations faster.

Where it fits

A release is one of the moments a working integration is most likely to break, so error alerts need to show failure rates between waves. A change that adds a permission also sends every existing customer back to the connect screen, which is why it always needs a wave of its own.

For the full life of an embedded integration, from choosing what to build to tracking who uses it, see the beginner’s guide to embedded integrations.