Task scheduling with Kafka

Hello! I want to build a service that when called can schedule a Task for a point in the future. I.e. “at 5AM 05/09/2022” do a POST at /api/example". What’s the best way to do this?

Maybe vote for my feature request :joy:
https://github.com/kubernetes/kubernetes/issues/75817

I have a custom solution to it but it’s not public… maybe I can publish it hmmh

If you want a ready made solution now, use Argo Workflows and the ‘suspend’ template

It’s a pretty big hammer though, maybe not worth it if you don’t need workflows in general

Basically you would make a Workflow which uses suspend to sleep until the correct time, then schedules a Job which does the POST

Yeah, installing Argo is probably not worth it

My solution so far is to schedule a Chron Job and then delete the job once it completes

But im not sure if it’s the easiest way

My custom solution was based on metacontroller

But it was for a specific special case… not sure if it would make an elegant generic delayed Job scheduler

Hmmm! I just thought of a way to solve this… 3 years of “more kubernetes” seems to help

Instead of a CronJob, just make Jobs… but set paralellism in them to zero (default is 1). that way the Job will not be scheduled.

And annotate the Job with your custom annotation timestamp when to schedule it

Then have a CronJob that runs periodically, scans the Jobs with your annotation and patches paralellism to 1 … :smile:

Ok it’s friday evening and my brain is exhausted, that’s the best I could come up with

https://kubernetes.io/blog/2021/04/12/introducing-suspended-jobs/

Maybe I will cook up a controller and put it on github

you can test https://github.com/vaizki/k8s-jobber

Was it any help? :slightly_smiling_face: