Step Functions, JSONata and Comments
Learn how JSONata's comments can improve your AWS Step Functions.
My favourite AWS PreInvent 2024 announcement was JSONata support in Step Functions.
I quickly converted a lot of my Step Functions to use the new JSONata syntax. I was able to eliminate a lot of data transformation steps. The old JSONPath implementation was frustratingly limited. Everything had to be merged into a global object that was carried from state to state. Extracting data from results involved 3 separate JSONPath queries. Massaging data resulted in additional steps (and cost).
Rather than rant about the past, today I want to focus on an underrated feature of the JSONata support in Step Functions.
Comments
The best documentation is self documenting code.
— No one debugging a production issue at 3am
Most Some of the code we write made sense on the day it was written. A year later it looks more like Egyptian heliographics than something we’ve written. This is worse when someone else wrote the code.
Step Functions uses JSON for defining the statement machine. JSON doesn’t allow inline comments. Javascript supports comments, but JSON explicitly excludes them. This makes it difficult to document step functions. That is until now!
JSONata supports C style comments. We can use these comments to document Step Functions. While this doesn’t completely solve the documentation problem, it does make it possible to add some inline comments. A light sprinkling of comments can help your future self and team mates. You can even get a bit imaginative. Take this example:
{
"Comment": "Workflow showing how to use JSONata for comments",
"QueryLanguage": "JSONata",
"StartAt": "GetTicket",
"States": {
"GetTicket": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Output": "{% $states.result.Payload %}",
"Arguments": {
"FunctionName": "arn:aws:lambda:us-east-1:012345678910:function:picofun_zendesk_get_api_v2_tickets_ticket_id:$LATEST",
"Payload": {
"path": {
"ticket_id": "{% $states.input.Ticket.ticket_id %}"
}
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 5,
"BackoffRate": 2,
"JitterStrategy": "FULL"
}
],
"Next": "GetGroupMappings",
"Assign": {
"Ticket": {
"ticket_id": "{% $states.input.Ticket.ticket_id %}",
"body": "{% $states.result.Payload.body.ticket.description ~> $lowercase() /* lowercase is more efficient */ %}",
"group_id": "{% $states.result.Payload.body.ticket.group_id %}",
"opened": "{% ($toMillis($states.result.Payload.body.ticket.created_at) / 1000) /* convert to unix timestamp */ %}"
}
}
},
"GetGroupMappings": {
"Type": "Task",
"Arguments": {
"Name": "arn:aws:ssm:us-east-1:012345678910:parameter/groups"
},
"Resource": "arn:aws:states:::aws-sdk:ssm:getParameter",
"Assign": {
"MappedGroups": "{% $parse($states.result.Parameter.Value) /* decode json string */ %}"
},
"Next": "RecordTicket"
},
"RecordTicket": {
"Type": "Task",
"Arguments": {
"ResourceArn": "arn:aws:rds:us-east-1:012345678910:cluster:mydb",
"SecretArn": "arn:aws:secretsmanager:us-east-1:012345678910:secret:rds!cluster-810496ad-8958-4aff-9bb0-ce4b7d84bc3c-w86PpW",
"Database": "ticket",
"Sql": "INSERT INTO table (id, body, group_id, opened) VALUES (:id, :body, :group_id, :opened)",
"Parameters": [
{
"Name": "group_id",
"Value": {
"LongValue": "{% $number([$lookup($MappedGroups, $string($Ticket.group_id)), $Ticket.group_id][0]) /* See https://github.com/jsonata-js/jsonata/issues/732#issuecomment-2612124834 */ %}"
}
},
{
"Name": "id",
"Value": {
"LongValue": "{% $number($Ticket.ticket_id) %}"
}
},
{
"Name": "opened",
"Value": {
"LongValue": "{% $number($Ticket.opened) %}"
}
},
{
"Name": "body",
"Value": {
"StringValue": "{% $Ticket.body %}"
}
}
]
},
"Resource": "arn:aws:states:::aws-sdk:rdsdata:executeStatement",
"Retry": [
{
"ErrorEquals": [
"RdsData.DatabaseResumingException",
"RdsData.DatabaseUnavailableException",
"RdsData.RdsDataException"
],
"IntervalSeconds": 5,
"MaxAttempts": 3,
"BackoffRate": 2
}
],
"End": true
}
}
}
The inline comments make it easier to understand what is supposed to happen in this step.
Your future self may just thank you one day for that extra {% /* comment */ %} you dropped into the state definition file.
Need Help?
If you want to adopt Proactive Ops, but you're not sure where to start, get in touch! I am happy to help get you.
Proactive Ops is produced on the unceeded territory of the Ngunnawal people. We acknowledge the Traditional Owners and pay respect to Elders past and present.