Hi friends, does anyone have any suggestions on how we might be able to monitor the Number of Objects within an s3 bucket path. For example i would like to track the number of objects within my-data-lake/bronze/source1 then displaying it in Grafana. Currently i can only get the number of objects per bucket.
You can list the objects at a path and then count the number of objects
https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html|https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html
That’s the CLI doc but you could do it with SDK too
Ye im aware of that command. But building something to plot the number of objects overtime is a little more involved than i first thought
I had one idea to create a custom cloudwatch metrics and a lambda function that runs every 5 mins to update it 
There also s3 inventory but that runs once a day only
There’s a cloud watch example here but also a note that it didn’t work for everyone?
https://stackoverflow.com/questions/2862617/how-can-i-tell-how-many-objects-ive-stored-in-an-s3-bucket|https://stackoverflow.com/questions/2862617/how-can-i-tell-how-many-objects-ive-stored-in-an-s3-bucket
the 3rd option i came across was aws athena and glue crawler
I mean if it’s an application that’s uploading the files you could track it from the application metrics
Instead of looking up what’s actually in the bucket
Anyway good luck I think I’ve shared all of the knowledge I possibly can on the subject 
Yeah no problem, thanks for the suggestion
