Azure Blob Storage

1.0

Azure Blob Storage is a feature of Microsoft Azure. It allows users to store large amounts of unstructured data on Microsoft's data storage platform.,

Overview

Copy

Azure Blob storage is a service for storing large amounts of unstructured object data, such as text or binary data. Common uses of Blob storage include: Serving images or documents directly to a browser. Storing files for distributed access.

When using the Azure Blob Storage connector, the first thing you will need to do is go to your Tray.io account page, and select the workflow you wish to work on. Once in the workflow builder itself, search and drag the Azure Blob Storage connector from the connectors panel (on the left hand side) onto your workflow.

With the new Azure Blob Storage connector step highlighted, in the properties panel on the right, click on the Authenticate tab and 'Add new authentication' (located under the 'Authentication' field).

This will result in a Tray.io authentication pop-up window. The first page will ask you to name your authentication, and state which type of authentication you wish to create ('Personal' or 'Organisational').

As you can see, the next page asks you for your 'Instance URL', 'Directory ID', 'Client ID', and 'Client Secret', credentials. Also please note that the scopes have to be ticked in order for your authentication to work as expected.

In order to get these fields, you will have to create an integration application in Azure.

Copy

1. First sign in to the Azure portal.

2. If you have access to multiple tenants, use the Directory + subscription filter in the top menu to select the tenant in which you want to register an application.

3. Search for and select Azure Active Directory.

1. Under Manage, select App registrations, then New registration.

1. Enter a name for your application and the redirect URL and select Register to complete the initial app registration.

When registration completes, the Azure portal displays the app registration's Overview pane, which includes its Application (client) ID. Also referred to as just Client ID, this value uniquely identifies your application in the Microsoft identity platform. Please copy your 'Client ID' (Application ID) and 'Directory ID' (Tenant ID) to add them in the Tray authentication pop-up.

Next, we have to configure the Azure Application:

Copy

1. Select your application in 'App registrations' in the Azure portal.

1. Under Manage, select 'API permissions', then click on the 'Add permission' button and choose 'Azure Storage' box.

1. Next tick the 'user_impersonation' check box and click the 'Add permissions' button.

1. Notice your Azure Storage user impersonation was added.

Get you 'Client Secret':

Copy

1. Select your application in App registrations in the Azure portal.

2. Select 'Certificates & secrets' and then 'New client secret'.

1. Add a description for your client secret, select a duration, then select 'Add'.

1. Record the secret's value for use in your client application code - it's never displayed again after you leave this page.

Configure RABC role for the user:

Copy

1. Select your Subscription from Microsoft Azure resources.

1. Select the subscription you need.

1. Choose 'Access control (IAM)', click on the 'Add' button, then select 'Add role assignment'.

1. Then choose 'Storage Blob Data Owner' from the dropdown list and search for the user you want to give permissions to and click 'Save'.

The 'Instance URL' can be found in the API documentation on every endpoint page. 'myaccount' has to be changed with the name of the storage you want to retrieve the blobs from.

Copy

Once you have added these fields to your Tray.io authentication popup window, click on the 'Create authentication' button. Go back to your settings authentication field (within the workflow builder properties panel), and select the recently added authentication from the dropdown options now available.

Your connector authentication setup should now be complete.

Available Operations

Copy

The examples below show two of the available connector operations in use.

Please see the Full Operations Reference at the end of this page for details on all available operations for this connector.

Note on Operations Usage

Copy

Using the Raw HTTP Request ('Universal Operation')

Copy

As of version 1.0, you can effectively create your own operations.

This is a very powerful feature which you can put to use when there is an endpoint in Azure Blob Storage which is not used by any of our operations.

To use this you will first of all need to research the endpoint in the Azure Blob Storage API documentation v1.0, to find the exact format that Azure Blob Storage will be expecting the endpoint to be passed in.

Note that you will only need to add the suffix to the endpoint, as the base URL will be automatically set (the base URL is picked up from the value you entered when you created your authentication).

The base URL for Azure Blob Storage is: https://test-storage.blob.core.windows.net.

For example, say that the 'List Blobs' operation did not exist in our Azure Blob Storage connector, and you wanted to use this endpoint, you would use the Azure Blob Storage API docs to find the relevant endpoint - which in this case is a GET request called: /mycontainertest?restype=container&comp=list.

More details can be found here.

As you can see there is also the option to include a query parameter, should you wish to do so. Note that the 'x-ms-version' header is required in order to make a successful request. So if you know what your method, endpoint and details of your query parameters are, you can get the Blob storage information with the following settings:

Method: GET

Endpoint: /mycontainertest?restype=container&comp=list

Query Parameter: Key: prefix Value: a

Final Example outcome being: https://test-storage.blob.core.windows.net/mycontainertest?restype=container&comp=list

Example Usage

Copy

Below is an example of a way in which you could potentially use the Azure Blob Storage connector, to get the existing Blobs from your storage account one by one.

The steps will be as follows:

1. Setup using a manual trigger and List Blobs operation.

2. Add a Loop connector to iterate through the blobs found.

3. Get all the available data for each blob, individually.

The final outcome should look like this:

1 - Setup Trigger & List Blobs operation.

Copy

Once you have clicked 'Create new workflow' from your main Tray.io dashboard named it, select the Manual trigger from the trigger options available:

After you have been redirected to the Tray.io workflow dashboard, from the connectors panel on the left, add a Azure Blob Storage connector to your second step. Set the operation to 'List Blobs'.

Feel free to re-name your steps as you go along to make things clearer for yourself and other users.

When run, this workflow will list all the blobs and all the snapshots available within the Azure Blob Storage under your credentials.

2 - Add a Loop connector

Copy

Next, search for the Loop collection connector within your connector panel, and drag it into your workflow as your next step. Set your operations to 'Loop list'.

The Loop Collection connector allows you to iterate through a list of results. In this example, we will use it to iterate through the data found within the previous Azure Blob Storage connector step.

In order to specify the list you want to loop through, start by using the 'List' mapping icon (found next to the list input field, within the properties panel) to generate the connector-snake.

While hovering over the 'List Blobs' step (with the tail end of the connector-snake), select Blob from the list of output properties displayed. This will auto-populate a jsonpath within your 'List' input field, and update the type selector to jsonpath.

For more clarification on the pathways you have available, open the Debug panel to view your step's Input and Output.

When run, this workflow will iterate through the list of blobs provided by the earlier step.

3 - Download Blob

Copy

The last step is to drag a Azure Blob Storage connector inside of the Loop Collection step itself. Set the operation to 'Download Blob'. As you can see, the 'Container name' and the 'Blob name' fields are required.

You can also check the Debug panel for more details on the provided output:

When run, the workflow will now iterate through the list of blobs provided, and get all the details available for each individual one.