Monitoring the Number of Objects within an S3 bucket path

Hi friends, does anyone have any suggestions on how we might be able to monitor the Number of Objects within an s3 bucket path. For example i would like to track the number of objects within my-data-lake/bronze/source1 then displaying it in Grafana. Currently i can only get the number of objects per bucket.

You can list the objects at a path and then count the number of objects
https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html|https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html

That’s the CLI doc but you could do it with SDK too

Ye im aware of that command. But building something to plot the number of objects overtime is a little more involved than i first thought

I had one idea to create a custom cloudwatch metrics and a lambda function that runs every 5 mins to update it :confused:

There also s3 inventory but that runs once a day only

There’s a cloud watch example here but also a note that it didn’t work for everyone?
https://stackoverflow.com/questions/2862617/how-can-i-tell-how-many-objects-ive-stored-in-an-s3-bucket|https://stackoverflow.com/questions/2862617/how-can-i-tell-how-many-objects-ive-stored-in-an-s3-bucket

the 3rd option i came across was aws athena and glue crawler

I mean if it’s an application that’s uploading the files you could track it from the application metrics

Instead of looking up what’s actually in the bucket

Anyway good luck I think I’ve shared all of the knowledge I possibly can on the subject :joy_cat:

Yeah no problem, thanks for the suggestion