Add data processing
Last updated
Last updated
Some times, when exposing a trigger to another system, the message sent is not what the Start step of the machine workflow expects, and neither of the two can or want to be adapted to the other.
In such a case you can add Data Processing to you integration, that transform the incoming message from, and/or outgoing message to, the external system.
On the integration click the tab Data Processing and within that tab click either Input or Output sub-tab, depending on if you want to add data processing to the incoming or returning message.
Input Data Processing
Click the toggle to On.
Now in the dropdown under Kind of transformation you have some options:
None
Liquid - Liquid is a declarative "templating" language highly suitable for transforming Json to Json. (for more on the Liquid template language: https://shopify.github.io/liquid/ )
XSLT - XSLT (Extensible Stylesheet Language Transformations) is a language originally designed for transforming XML documents into other XML documents.
Select for example Liquid if that suits your incoming message format.
In the top text box that appears called Example input write or paste an example of a typical incoming message that covers as much of the message structure as possible.
Now scroll down and make the next text box called Transformation visible.
If you are very apt in writing Liquid template transformations, or have one ready, write or paste that in to this text box.
If you want some help you have two options.
Copy the target structure
Let AI suggest a transformation
You can select the entire text in the Target structure text box and paste it to the Transformation text box. This will initialize the Liquid transformation with a scaffold template that will create a message looking like what the machine workflow Start step expects but without mapped values from the incoming message.
Now the task is to replace the occurrences of "x" in the scaffold Liquid template with references to the incoming message structure.
Place the cursor on one of the "x" occurrences (and delete the x character) and now type two opening/left curly braces ( {{ ) and press ctrl-space to bring up the auto complete drop down.
The drop down lists data fields of the incoming message structure. Choose the one that is appropriate for the scaffold field you are working with. Make sure the expression ends with two closing/right curly braces ( }} ).
Repeat this for all the scaffold "x" occurrences.
If you need you can of course also do more advanced Liquid template transformations using control flows and filter operators. But that is up to you to explore.
At any time during you transformation construction you can try the transformation at hand on the Example input message by clicking the button TestTransformation. The result of the transformation will appear in the text box Result under Output and is automatically compared with the text in the text box Target structure under Output.
Another way to start creating a transformation is to use the functionality Let AI suggest a transformation which uses a Large Languge Model AI to figure out how the Liquid template should look like to go from the Example input to the Target structure.
Often just the names of the keys in the key-value pairs are sufficient for the AI functionality to make a good enough extrapolation.
Start by trying it and click the button Let AI suggest a transformation.
It produces something like the example above.
The Let AI suggest a transformation functionality works better the more information you provide. So start by replacing the value placeholders ( "x" ) in the Target structure with values that correspond to the value in the Example input.
So what if there are some fine grained details in the value mappning?
For example, let's assume the following regarding some value fields in the Target structure:
CreatedDate should be of the format m/d/yyyy
Street1 should have the street number
Street2 should have the street name
We amend the text in the Target structure accordingly so signal this details. Use the information from the Example input and change the values according to your specific logic as you enter them to the Target structure.
Now click Let AI suggest a transformation again.
Voila!
The Let AI suggest a transformation functionality with its Large Language Model understod (in this example) perfectly how the Example input should be further transformed to create the specified Target structure.
It has added a date format filter on the mapping of CreatedDate.
It has added split and first/last filters on the mapping of BillingStreet to Street1 and Street2.
Click the button Test Transformation to verify the functionality of the transformation.
Note: Remember that this functionality is as with that of many Large Language Model AIs. It merely makes a kind of statistical extrapolation from the text that is provided. It will far from always produce exactly correct results. But most of the time it is at least quite close, if not being correct. Considering it a way to get started.
Now let's assume you added one or more query parameters to an Integration of type webhook (on the Trigger Type tab) as in the example below.
Now you want the values passed as the query parameters in the http call to be passed on further to the machine workflow.
You can inject these values to the message passed on to the machine workflow by using the transformation template. Currently this only works for transformations of type Liquid.
At the place in the Liquid template where you want a certain query parameter injected you type a two left curly braces {{ and a context pop-up will appear with all the variables available, both from the incoming message structure in the text area above the transformation template, but also one special variable called Trigger_parameters.
If you select that option, (by pointer device or clicking enter) a special string parameters. will be injected. After the punctuation dot you can once again get a contextual pop-up (if it does not appear, try typing ctrl-space with the cursor directly after the punctuation dot).
You will now see all the query parameters configured for the webhook integration and can select the one you want.
Whatever is passed as value for that query parameter during an http post to the webhook (or the default value if nothing is passed) will be injected in the transformation message and then sent to the machine workflow.
