Hello, if I was looking to start a Kubernetes batch job when a file was uploaded to S3 - would the best solution be to
• Setup an eventbridge rule to monitor S3 uploads for a bucket
• Kick off a step functions job on upload
• Step functions start and monitor Kubernetes job
I think you want S3 event trigger => SQS => Lambda and then do whatever you need in there. I can’t fathom why you would then trigger a k8s batch job when you have lambda at that point, but you could start the batch job from lambda.
Thanks. I need to execute jar files that have been built elsewhere. These jar files will interact with other services and finish when they have a response back and have done some data transformation.
So I thought containerising those jar files might be the easiest approach.
Nothing inherently would stop you from executing those in lambda that I’m aware of. lambda can run binaries. If the action is going to take longer than 15minutes you can use codebuild to run the container for up to 8 hours. For longer you can use ECS Fargate scheduled task. There isn’t anything about what you are doing that fundamentally would be better with k8s, but again, that’s up to you.
That’s interesting I didn’t think of using CodeBuild, that would certainly reduce operational overhead of a cluster just to run odd jobs. I guess using AWS Batch would be a more complicated solution than CodeBuild too.
And use either lambda or step functions/eventbridge to kick off the CodeBuild job.
Possibly wrap the codebuild job in codepipeline which can allegedly receive s3 events to kick the build off 
