Duplicate API logs in Datadog

Any idea on how to stop duplicating API logs in datadog. Currently request to my api are logged twice.

The subscription field is 2 different values

What do you mean with respect to subscription field

The solution is to step sending the log twice most likely :stuck_out_tongue:

I mean it has 2 logs for one request

Is the app emitting logs twice? Are there two datadog agents collecting them independently? Are you collecting logs from two different proxies that look identical?

Yea, you’re likely somehow producing the log once, and you’ve set things up such that you’re consuming it twice

Consuming in-container app logs that also get mirrored to STDOUT, as well as consuming STDOUT is one common scenario

Each log event should, I think, have some form of provenance info on it

I.e. which agent(s) sent/emitted the event

And via which channels

Is the app emitting logs twice? Yes
Are there two datadog agents collecting them independently? I can see 2 subscription values
subscription projects/aws-account-id/subscriptions/datadog
subscription projects/aws-account-id/subscriptions/loadbalancer-datadog

Are those like cloudwatch events or something?

Can you share an actual instance of the event, with all/most of the metadata fields?

You should hopefully have some understanding of where you’re gathering logs from

One way or another, if you don’t want duplicate log events, you’ll either need to figure out a way to filter in the log pipelines, unsubscribe from a particular event stream/channel that has duplicates, or tell datadog agents to stop consuming events from something

There should be a shitload of tags on the log events

Container_name, pod_name if in k8s

Basically, DataDog will do what you tell it

If you tell it to listen to something, it’s going to listen to it and send the events