Identifying the issue with passing AWS event payload in Step Function

DevOps AWS
#1

Can anyone guess what am I doing wrong that the input of the “Pass” state in the Step Function State Machine is empty when it is triggered by being a target of a Rule that passes an AWS event in EventBridge?

The event Rule triggers and starts the “Pass” state, but does not pass the payload of the event to it.

In this Sfn, I want to make a decision based on a value passed by the event to the Pass step, but I do not receive the payload. I’m using the “Matched event” to do no transformation. I have also tried with transformation, but got same result.

I appreciate any help.

#2

Any ideas also what to check to debug?

#3

what’s the output from start-glue-crawler?

#4

You can try to tinker with this, it’s kind of helpful, but also kind of … not https://console.aws.amazon.com/states/home?region=us-east-1#/simulator

#5

This might be helpful https://docs.aws.amazon.com/step-functions/latest/dg/input-output-resultpath.html

#6

Thank you Erica

The output of the previous step is also an empty object {}
which makes sense

So, the logic I am looking for is that at the start-glue-crawler step, it executes Crawler (async) and detaches from the Step Function

Then when EventBridge catches a crawler “finished” event, it triggers the next step which is a “Pass”

Right now, the input of the “Pass” step is the same as the output of the “start-glue-crawler” step, not the payload of the event that EventBridge sends

I even tried the Merge option, but still I only see the “Empty object” payload

I’m not sure what should I do to receve the event’s payload instead of the empy object passed from the previos step in Sfn

#7

I donno if it is EventBridge that does not send the event payload (which based on the config I shared, it should in theory at least - not sure how to debug it though) or if it is the “Pass” step in Sfn that does not Merge the event payload with the previous step, and if so, what should I change for that to happen?

#8

I’ve also tried different ResultPaths:
• $ => to pass the the whole object => which is an empty object - not the payload of the event
• $.detail etc. (this fails because the object is empty)

#9

I realized that “Pass” does not “wait for the event & it directly passes the empty object to the next step.

Also, Crawler does not support TaskToken, so I had to use this approach:
https://aws.amazon.com/blogs/compute/orchestrating-aws-glue-crawlers-using-aws-step-functions/

#10

Glad you were able to get it figured out! I had a similar issue with managing inputs/outputs in StepFunctions, but I couldn’t spend too long on it yesterday to see if I could help.