Monitoring GKE clusters with Managed Prometheus

Hi folks! Does anyone have useful information about how to Monitor the GKE cluster with Managed Prometheus? I would like to create useful dashboards like in Grafana for GCP…

Or maybe another way to have a centralised Grafana for multiple GKE clusters, the main idea is to use Managed Prometheus for all our clusters to scrape metrics. Thanks in advance

Are your GKE clusters in same project or spread across multiple GCP projects?

The first approach should be only in the same project at least

Right now, I’m confused after reading all the GCP Documentation. It looks complex.

This is the closest link i found which is a mix of managed and self hosted prometheus you need to install exporters https://cloud.google.com/stackdriver/docs/managed-prometheus/best-practices/ingest-and-query

With managed prometheus i am not sure if each of the clusters would get its own datastore / Monarch

From the above link If you use only managed or self-deployed collection, then the diagram is still applicable; just ignore the collection style you aren't using:

Not to give negative feedback but I have never been a fan of managed offerings with GCP be it Managed Istio aka Anthos Service Mesh (ASM) or Managed Prometheus I prefer the OSS versions :slightly_smiling_face:

I agree with you. I did a POC using OSS Prometheus Operator

But now I’m trying to build the same with Managed Prometheus to compare with the previous implementation

Because there are pros and cons always…

Yes it is a vendor lock-in also

Yeah, but we don’t have a big team with expert skills about that so we may use the magic of GPC whenever we can…

I found right here an interesting doc

https://cloud.google.com/stackdriver/docs/managed-prometheus/best-practices/multi-tenant

I’m looking for something like that

Isn’t is as simple as pointing all the prometheus proxy services (from different GKE clusters) to the same Grafana endpoint / centralized Grafana?

I’m going to play a little bit :wink:

Good luck and let me know :slightly_smiling_face:

Thank you so much. I will share everything once I finish