Transformations API
17 minute read
RudderStack’s Transformations API allows you to create, read, update and delete transformations and libraries programmatically by making HTTP calls.
This guide describes the various API operations, related request and response structures, and error codes associated with this API.
Prerequisites
- Generate a workspace-level Service Access Token with the following permissions to authenticate the API:
- For testing/personal use cases only: Generate a Personal Access Token with Read-Write role
RudderStack recommends using a workspace-level Service Access Token for authentication.
Any action authenticated by a Personal Access Token will break if the user is removed from the organization or a breaking change is made to their permissions.
Token permissions for legacy RBAC system
If you are on the legacy Permissions Management (RBAC) system, your workspace-level Service Access Token should have minimum Admin permissions with Grant edit access toggled on under Transformations.
See this documentation for more information on generating the token.
Authentication
The Transformations API uses Bearer authentication. Pass your workspace-level Service Access Token or Personal Access Token from the Prerequisites as the Bearer token:
Authorization: Bearer <SERVICE_ACCESS_TOKEN>
Base URL
Use the base URL for your API requests depending on your region:
Transformations
RudderStack transformations are responsible for converting received event data into a suitable destination-specific format. All the transformation code is written in JavaScript.
We also support user-specific transformations for real-time operations, such as aggregation and sampling.
Transformations help you to create a user-defined code that allow you to route your events in a manner that is suitable for your destinations.
Transformer payload
| Field | Type | Presence | Description |
|---|---|---|---|
name | String | Required | Sets the transformation name. |
language | String | Required | Language of the transformation code. Acceptable values are javascript and pythonfaas. |
description | String | Optional | Sets the transformation description. |
code | String | Optional | User-defined code that maps event data to destinations as defined by the user |
codeVersion | String | Optional | This is a number value always set to version “1” for API calls. |
createdAt | Date | Optional | The timestamp of the transformer when it is created |
updatedAt | Date | Optional | The timestamp of the transformer when it is updated |
versionId | String | Optional | Maintains a version of transformer every time it is updated |
workspaceId | Object | Optional | Workspace ID on which this transformation is created |
destinations | Array | Optional | List of all Destination IDs to which your transformation is connected |
Create a transformation
Create an unpublished transformation.
When you create a transformation but do not publish it, that is, when publish = false, RudderStack creates revisions for the transformation, but it is not available to incoming event traffic and cannot connect to destinations.
When you wish to make the transformation live, see Publish a transformation.
Query parameters:
falsetrue, publishes your transformer to the latest version; code is made live for incoming traffic.Example request:
In this example, publish is false, which is the default setting for the parameter. When unpublished, RudderStack only creates revisions for the transformation, meaning that you cannot connect destinations to the transformation and it cannot be used for incoming event traffic.
Request body:
destinationIds that you wish to connect with this transformation. You can connect only if publish is set to true.javascript and pythonfaas.Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2LnbcGgKON5BbHHuyYVesZ24uqu",
"versionId": "2LnbcImcBqOTkm4FFVCpIakptZJ",
"name": "Get metdata",
"description": "Gets the metadata for an event",
"code": "export function transformEvent(message, metadata) { const met = metadata(message); return met; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2023-02-16T01:11:11.586Z",
"updatedAt": "2023-02-16T01:11:11.586Z"
}
Events JSON: When passing events in the request body, format the events in JSON
{
"events": [
{
"anonymousId": "8d872292709c6fbe",
"channel": "mobile",
"context": {
"traits": {
"address": {
"city": "Kolkata",
"country": "India",
"postalcode": "700096",
"state": "West bengal",
"street": "Park Street"
}
}
},
"properties": {
"revenue": "30",
"currency": "USD",
"quantity": "5",
"price": "58.0"
}
}
]
}
Publish a transformation
Publish your transformation. This request is the same as Create transformation except that you need to include ?publish=true in the query, which will allow you to connect destinations to the transformation and make it available to incoming traffic.
When you publish a transformation, we maintain two copies of the transformer: one is published and the other is used for revisions. The published version can be connected to destinations and its code is made live for incoming traffic.
Query parameters:
falsetrue, publishes your transformer to the latest version; code is made live for incoming traffic.Request body:
destinationIds that you wish to connect with this transformation. You can connect only if publish is set to true.javascript and pythonfaas.Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2LpkmqnLEuUziYZcfRCdNLDQxDk",
"versionId": "2LpkmrHiyW4gXovOzhvtzbhdZ23",
"name": "Cool transformation",
"description": "A test description",
"code": "export function transformEvent(event) { return event; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2023-02-16T19:26:13.263Z",
"updatedAt": "2023-02-16T19:26:13.263Z",
"destinations": []
}
List all transformations
List all published transformations for a workspace.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"transformations": [
{
"id": "sedrftg",
"versionId": "edrtv",
"name": "new Transformations-2",
"description": "",
"code": "export function transformEvent(event) { return event; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T04:48:27.288Z",
"updatedAt": "2021-03-04T04:48:27.288Z",
"destinations": []
},
{
"id": "xcgvhcfdx",
"versionId": "dtvbyutbvc",
"name": "Update Transformations and Publish",
"description": "",
"code": "function transformEvent(event) { return event; } ",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T10:07:25.513Z",
"updatedAt": "2021-03-04T10:07:25.513Z",
"destinations": []
}
]
}
Retrieve a single transformation
Retrieve a published transformations from an ID.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "swderftgy",
"versionId": "edftgyhu",
"name": "new Transformations-2",
"description": "",
"code": "export function transformEvent(event) { return event; } ",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T04:48:27.288Z",
"updatedAt": "2021-03-04T04:48:27.288Z",
"destinations": []
}
Update and publish a transformation
Updating a transformation creates a new revision and sets it as published if the publish flag is set is true, and its code becomes live for upcoming traffic. If the publish flag is false , it only creates a new revision for that transformation.
You cannot update the language used to write the transformation code, that is, a JavaScript transformation cannot be converted to Python and vice versa.
Example request:
Example response:
{
"id": "2C8Vk2wj8qkofy00YzJbvJOGeqa",
"versionId": "2MTBAAFdGs29S080tsvyg8gbsUj",
"name": "Updated transformation JS 2",
"description": "Updated description",
"code": "export function transformEvent(event) { return event; }\n",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2023-03-02T18:23:30.580Z",
"updatedAt": "2023-03-02T18:23:30.580Z"
}
Delete a transformation
Delete a published transformation by ID. Note that RudderStack never deletes a transformation revision.
Example request:
Example response:
HTTP/1.1 200 OK
List transformation versions
List all transformation versions for a given transformation ID.
Example request:
Query parameters:
ascasc for ascending or desc for descending. By default, it sets the order as ascending on createdAt.Example response:
{
"TransformationVersions": [
{
"id": "1pIYoILGZTNYZP4YYkeyNIKlitl",
"versionId": "1pIYoLfEzcMK3D3M1ihjqI02wnx",
"name": "Update Transformations and Publish",
"description": "",
"code": "export function transformEvent(event) { return event; }\n",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T10:07:24.562Z",
"updatedAt": "2021-03-04T10:07:24.562Z"
},
{
"id": "1pIYoILGZTNYZP4YYkeyNIKlitl",
"versionId": "1pIhxFXd7NR7XDA914rLAn5f7wq",
"name": "Update Transformations and Publish",
"description": "Hey I am updated again",
"code": "export default function cube(x) { return x * x * x ; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T11:22:36.102Z",
"updatedAt": "2021-03-08T04:22:42.646Z"
}
]
}
Retrieve a single transformation version
Get a single transformation revision.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "1pIYoILGZTNYZP4YYkeyNIKlitl",
"versionId": "1pIYoLfEzcMK3D3M1ihjqI02wnx",
"name": "Update Transformations and Publish",
"description": "Updated sample transformation ready to be published",
"code": "export function transformEvent(event) { return event; }\n",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2021-03-04T10:07:24.562Z",
"updatedAt": "2021-03-04T10:07:24.562Z"
}
Destination connections
Use the endpoints in this section to connect a transformation to a destination, update the connection settings, or disconnect it.
Note that:
- You need to publish a transformation before connecting it to a destination.
- A destination can have only one connected transformation at any given time. Connecting a transformation to a destination replaces any transformation already connected to it.
Connect a transformation to a destination
Connect a transformation to a destination by specifying the transformation ID in the path and the destination ID in the request body.
Path parameters:
Request body:
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2C8Vk2wj8qkofy00YzJbvJOGeqa",
"versionId": "2MTBAAFdGs29S080tsvyg8gbsUj",
"name": "Cool transformation",
"description": "A test description",
"code": "export function transformEvent(event) { return event; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2023-03-02T18:23:30.580Z",
"updatedAt": "2023-03-02T18:23:30.580Z",
"destinations": [
{
"id": "2C8YtptB4KF2eL3KRi9mCFkY3BF",
"name": "My destination",
"enabled": true
}
]
}
RudderStack returns a 400 Bad Request response if the transformation is not published, with the below error:
{
"message": "Transformation is not published"
}
Disconnect a transformation from a destination
Disconnect a transformation from a destination by specifying the transformation ID in the path and the destination ID in the request body.
Path parameters:
Request body:
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2C8Vk2wj8qkofy00YzJbvJOGeqa",
"versionId": "2MTBAAFdGs29S080tsvyg8gbsUj",
"name": "Cool transformation",
"description": "A test description",
"code": "export function transformEvent(event) { return event; }",
"codeVersion": "1",
"language": "javascript",
"createdAt": "2023-03-02T18:23:30.580Z",
"updatedAt": "2023-03-02T18:23:30.580Z"
}
Libraries
Libraries are JavaScript code that you can write and export to be used in your transformations. They give you the flexibility for reusing and maintaining different versions of the transformation code.
Suppose you write an aggregation function. You can easily export them and use it within different transformations just by importing that module by the library name.
Libraries payload
Create a library
Create a library and get its object as a response.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2M1HnI40CGbHb4FxjjRBj1aFRZK",
"versionId": "2M1HnJoHwL4zXE91pEwZaAHQp8F",
"name": "cool library",
"description": "cool description",
"code": "export function add(a,b) {return a+b; } export function sub(a,b) {return a-b; }",
"language": "javascript",
"createdAt": "2023-02-20T21:25:33.380Z",
"updatedAt": "2023-02-20T21:25:33.380Z",
"importName": "coolLibrary"
}
Query parameters:
falsetrue, publishes your transformer to the latest version; code is made live for incoming traffic.Request body:
javascript and pythonfaas.List all libraries
Get all published libraries.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"libraries": [
{
"id": "1pHx15j5rXvmmQUIMBaQdIyrpr2",
"versionId": "1pHxdlGL8IyoP7WfvRil4Qs88cp",
"name": "Get Cube",
"description": "First Library using apiCall",
"code": "export default function cube(x) { return x * x ; }",
"language": "javascript",
"createdAt": "2021-03-04T05:01:46.985Z",
"updatedAt": "2021-03-04T05:01:47.141Z",
"importName": "getCube"
},
{
"id": "1pT7933tHRBPlEMIZt5Zi3VIht1",
"versionId": "1pT793mcqQkcyHdqwXkxHmtgMMg",
"name": "User Defined Library",
"description": "Get User context",
"code": " export default function cube(x) { return x * x * x; }",
"language": "javascript",
"createdAt": "2021-03-08T03:47:51.512Z",
"updatedAt": "2021-03-08T03:47:51.512Z",
"importName": "userDefinedLibrary"
}
]
}
Retrieve a library by ID
Get a single published library by ID.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "1pT7933tHRBPlEMIZt5Zi3VIht1",
"versionId": "1pT793mcqQkcyHdqwXkxHmtgMMg",
"name": "User Defined Library",
"description": "Get User context",
"code": " export default function cube(x) { return x * x * x; }",
"language": "javascript",
"createdAt": "2021-03-08T03:47:51.512Z",
"updatedAt": "2021-03-08T03:47:51.512Z",
"importName": "userDefinedLibrary"
}
List all library versions
Get all library revisions for a library ID.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"libraryVersions": [
{
"id": "1pT7933tHRBPlEMIZt5Zi3VIht1",
"versionId": "1pT793mcqQkcyHdqwXkxHmtgMMg",
"name": "userDefinedLibrary",
"description": "Get User context",
"code": "export default function cube(x) { return x * x * x; }",
"language": "javascript",
"createdAt": "2021-03-08T03:47:51.686Z",
"updatedAt": "2021-03-08T03:47:51.686Z",
"isPublished": false
},
{
"id": "1pT7933tHRBPlEMIZt5Zi3VIht1",
"versionId": "1pT8KDAD66mQxnaUQxJpNs9qLFn",
"name": "userDefinedLibrary",
"description": "Get Divisible by 2",
"code": "export default function cube(x) { return 2 * x; }",
"language": "javascript",
"createdAt": "2021-03-08T03:57:33.738Z",
"updatedAt": "2021-03-08T03:57:33.738Z",
"isPublished": true
}
]
}
Query parameters:
ascasc for ascending or desc for descending. By default, it sets the order as ascending on createdAt.Retrieve a single library version
Get a single library revision.
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "1pT7933tHRBPlEMIZt5Zi3VIht1",
"versionId": "1pT8KDAD66mQxnaUQxJpNs9qLFn",
"name": "userDefinedLibrary",
"description": "Get Divisible by 2",
"code": "export default function cube(x) { return 2 * x; }",
"language": "javascript",
"createdAt": "2021-03-08T03:57:33.738Z",
"updatedAt": "2021-03-08T03:57:33.738Z",
"isPublished": false
}
Update and publish a library
This request lets you update the code and description of the transformation library by specifying its ID. To publish the library, set the publish flag to true. If the publish flag is false , it only creates a new version of that library.
Note that:
- You cannot change the name of the library using this request.
- You cannot update the language used to write the library code, that is, a JavaScript library cannot be converted to Python and vice versa.
Request body:
Example request:
Example response:
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
{
"id": "2MTDFOdoL9qQxFFnhl6oB23oAQ2",
"versionId": "2MTDbAFBqzB7Wg8mWyWNBIk5DOU",
"name": "Sample Transformation Library JS 2",
"description": "Updated library description",
"code": "export function sum(a, b) { return a + b; }",
"language": "javascript",
"createdAt": "2023-03-02T18:42:53.821Z",
"updatedAt": "2023-03-02T18:42:53.821Z",
"importName": "sampleTransformationLibraryJs2"
}
Delete a library
Delete a library by ID.
Example request:
Example response:
HTTP/1.1 200 OK
Publish API
As an end user you can create a transformer/library and perform several edits on it. Note that publishing is optional at create.
If you perform some edits on this version of transformer, RudderStack takes your latest update as the published version, creates a copy of the older version, and saves it as revisions. Let’s assume that after creating some 7 to 8 such revisions of your transformer, you finally decide to use the second or third version of the transformer.
This is where the RudderStack Publish API comes into play.
Publish a transformation or library
Publish any transformation revisions or library revisions.
Example request:
Request body:
versionIds that you wish to publish.versionIds that you wish to publish.One of above transformations or libraries must be present to make a successful publish call.
A few things to note:
- You can choose to publish some revisions transformer without the libraries.
- You can choose to publish some revisions libraries without the transformers.
- You can publish both library and transformation revisions.
Whenever you call thepublishAPI, we run tests in our server to make sure you won’t save any transformation/libraries code that can lead to any exceptions. In case if your publish is failing, make sure to check your transformation code and the libraries that it is referring to.
Example response:
{
"published": true
}
