Setting up Datadog on GKE Autopilot encountering data collection errors

Monitoring and Observability Datadog
#1

Hi, I am trying to setup datadog on a gke autopilot.
After the install (I following this webpage) https://www.datadoghq.com/blog/gke-autopilot-monitoring/

I get these errors and datadog doesn’t collect any events.

process-agent 2023-12-11 11:01:15 UTC | PROCESS | INFO | (pkg/workloadmeta/store.go:524 in startCandidates) | workloadmeta collector “docker” could not start. error: component workloadmeta-docker is disabled: Agent is not running on Docker │
│ trace-agent 2023-12-11 11:01:16 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ trace-agent 2023-12-11 11:01:17 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ trace-agent 2023-12-11 11:01:18 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/transaction/transaction.go:365 in internalProcess) | API Key invalid, dropping transaction for https://orchestrator.datadoghq.com/api/v2/orch │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | ERROR | (pkg/process/runner/runner.go:479 in readResponseStatuses) | [pod] Invalid response from https://orchestrator.datadoghq.com: 403 → %!s() │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | INFO | (pkg/process/runner/runner.go:230 in logCheckDuration) | Finished pod check #2 in 807.57102ms │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | ERROR | (comp/forwarder/defaultforwarder/transaction/transaction.go:365 in internalProcess) | API Key invalid, dropping transaction for https://orchestrator.datadoghq.com/api/v2/orchmanif │
│ process-agent 2023-12-11 11:01:19 UTC | PROCESS | ERROR | (pkg/process/runner/runner.go:479 in readResponseStatuses) | [pod_manifest] Invalid response from https://orchestrator.datadoghq.com: 403 → %!s() │
│ trace-agent 2023-12-11 11:01:19 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ trace-agent 2023-12-11 11:01:20 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│ trace-agent 2023-12-11 11:01:21 UTC | TRACE | INFO | (pkg/config/remote/client.go:301 in pollLoop) | retrying the first update of remote-config state (could not acquire agent auth token: unable to read authentication token file: open /etc/datadog-agent/auth_token: no such file or directory) │
│

Can anyone help?

#2

For those who suffer from this problem (GKE autopilot + datadog integration)

Check if your datadog-values.yaml has datadog.site field. My datadog is provisioned on AP1 (tokyo), so I had to add values for this.

  apiKey: <API-KEY>
  appKey: <APP-KEY>
  clusterName: <CLUSTER>
  site: ap1.datadoghq.com  # This line is not specified in the document, but NEEDED```