Upgrade to OTel v0.116.0 #2314

ptodev · 2024-12-24T15:28:07Z

PR Description

Upgrading to the latest version of OTel.

Community components

Pinging community component owners:

otelcol.exporter.splunkhec: @adlotsof, @PatMis16.
otelcol.exporter.datadog: @polyrain.

Please feel free to open a PR to update the community components :) There have been a few minor changes to the upstream code. You could either merge to my PR, or wait for my PR to be merged and then to merge yours to main. It is not necessary to update the community components for v1.6, since the upstream changes are not that big. If you decide to not update them at all with the latest changes, that's ok too.

Which issue(s) this PR fixes

Fixes #2255
Fixes #2243

PR Checklist

CHANGELOG.md updated
Documentation added
Tests updated
Config converters updated

github-actions · 2024-12-24T15:29:37Z

💻 Deploy preview available: https://deploy-preview-alloy-2314-zb444pucvq-vp.a.run.app/docs/alloy/latest/

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

The forked Beyla version contains an updated OTel dependency.

ptodev · 2024-12-30T13:24:39Z

go.mod

+//TODO: Do not merge this. Wait for upstream to upgrade the main branch, or to release a new version.
+replace github.com/grafana/beyla => github.com/grafana/beyla v1.9.1-0.20241230130037-7083b65bf473


Unfortunately, we need to upgrade to a new version of Beyla as part of this OTel upgrade. It'll have to contain an upgrade to OTel 0.116.

There are also lint and build issues due to Beyla dependencies.

The non-test package needs to have access to the gate, so that when it's called from non-test code it can register the feature gate with certainty.

clayton-cornell · 2025-01-02T20:49:34Z

docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md

+
 > **EXPERIMENTAL**: Metrics support in `otelcol.exporter.loadbalancing` is an [experimental][] feature.
 > Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.
 > The `stability.level` flag must be set to `experimental` to use the feature.


Suggested change

> **EXPERIMENTAL**: Metrics support in `otelcol.exporter.loadbalancing` is an [experimental][] feature.

> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.

> The `stability.level` flag must be set to `experimental` to use the feature.

[blocks]: #blocks

> **EXPERIMENTAL**: Metrics support in `otelcol.exporter.loadbalancing` is an [experimental][] feature.

> Experimental features are subject to frequent breaking changes, and may be removed with no equivalent replacement.

> The `stability.level` flag must be set to `experimental` to use the feature.

[experimental]: https://grafana.com/docs/release-life-cycle/

Missing the blocks link definition, and when we included the custom experimental text we forgot to add the link definition for the release life cycle.

clayton-cornell · 2025-01-02T20:51:26Z

docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md

+* The ones under `protocol > otlp`. This is useful for temporary problems with a specific backend, like transient network issues.
+* The ones top-level ones for `otelcol.exporter.loadbalancing` itself. 


Suggested change

* The ones under `protocol > otlp`. This is useful for temporary problems with a specific backend, like transient network issues.

* The ones top-level ones for `otelcol.exporter.loadbalancing` itself.

* The queue and retry blocks under `protocol > otlp`. This is useful for temporary problems with a specific backend, like transient network issues.

* The top-level queue and retry blocks for `otelcol.exporter.loadbalancing`.

clayton-cornell · 2025-01-02T20:51:40Z

docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md

+* The ones under `protocol > otlp`. This is useful for temporary problems with a specific backend, like transient network issues.
+* The ones top-level ones for `otelcol.exporter.loadbalancing` itself. 
+  Those configuration options provide capability to re-route data into a new set of healthy backends.
+  This are useful for highly elastic environments like Kubernetes, 


Suggested change

This are useful for highly elastic environments like Kubernetes,

This is useful for highly elastic environments like Kubernetes,

clayton-cornell · 2025-01-02T20:53:06Z

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

@@ -56,6 +58,17 @@ data without any of the well-known IP attributes. If the Deployment {{< param "P
 {{< param "PRODUCT_NAME" >}}s deployed as DaemonSet, then some of those attributes might be missing. As a workaround,
 you can configure the DaemonSet {{< param "PRODUCT_NAME" >}}s with `passthrough` set to `true`.

+By default, `otelcol.processor.k8sattributes` will be ready as soon as it starts, even if no metadata has been fetched yet.


Suggested change

By default, `otelcol.processor.k8sattributes` will be ready as soon as it starts, even if no metadata has been fetched yet.

By default, `otelcol.processor.k8sattributes` is ready as soon as it starts, even if no metadata has been fetched yet.

clayton-cornell · 2025-01-02T20:54:13Z

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

+If telemetry is sent to this processor before the metadata is synced, there will be no metadata to enrich the telemetry with.
+
+To wait for the metadata to be synced before `otelcol.processor.k8sattributes` is ready, set the `wait_for_metadata` option to `true`.
+Then the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked. 


Suggested change

Then the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked.

Then, the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked.

clayton-cornell · 2025-01-02T20:54:55Z

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

+To wait for the metadata to be synced before `otelcol.processor.k8sattributes` is ready, set the `wait_for_metadata` option to `true`.
+Then the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked. 
+If the metadata cannot be synced by the time the `metadata_sync_timeout` duration is reached, 
+`otelcol.processor.k8sattributes` will become unhealthy and will fail to start.


Suggested change

`otelcol.processor.k8sattributes` will become unhealthy and will fail to start.

`otelcol.processor.k8sattributes` will become unhealthy and fail to start.

clayton-cornell · 2025-01-02T20:55:29Z

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

+`otelcol.processor.k8sattributes` will become unhealthy and will fail to start.
+
+If `otelcol.processor.k8sattributes` is unhealthy, other {{< param "PRODUCT_NAME" >}} components will still be able to start.
+However, they may not be able to send telemetry to `otelcol.processor.k8sattributes`.


Suggested change

However, they may not be able to send telemetry to `otelcol.processor.k8sattributes`.

However, they may be unable to send telemetry to `otelcol.processor.k8sattributes`.

clayton-cornell · 2025-01-02T21:00:14Z

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md

@@ -143,12 +156,30 @@ The `annotation` block configures how to extract Kubernetes annotations.

 {{< docs/shared lookup="reference/components/extract-field-block.md" source="alloy" version="<ALLOY_VERSION>" >}}

+{{< admonition type="warning" >}}


Suggested change

{{< admonition type="warning" >}}

{{< admonition type="caution" >}}

clayton-cornell · 2025-01-02T21:04:22Z

docs/sources/reference/components/otelcol/otelcol.processor.span.md

+This example will add the same new `"documentId"="12345678"` attribute as the previous example.
+However, it will now result in an unchanged span name (/api/v1/document/12345678/update).


Suggested change

This example will add the same new `"documentId"="12345678"` attribute as the previous example.

However, it will now result in an unchanged span name (/api/v1/document/12345678/update).

This example adds the same new `"documentId"="12345678"` attribute as the previous example.

However, the span name is unchanged (/api/v1/document/12345678/update).

clayton-cornell · 2025-01-02T21:04:44Z

docs/sources/reference/components/otelcol/otelcol.processor.tail_sampling.md


 `decision_wait` determines the number of batches to maintain on a channel. Its value must convert to a number of seconds greater than zero.

 `num_traces` determines the buffer size of the trace delete channel which is composed of trace ids. Increasing the number will increase the memory usage of the component while decreasing the number will lower the maximum amount of traces kept in memory.

 `expected_new_traces_per_sec` determines the initial slice sizing of the current batch. A larger number will use more memory but be more efficient when adding traces to the batch.

-`decision_cache` requires a key `sampled_cache_size` with a value that indicates the number of trace IDs to keep in the cache. When `sampled_cache_size` is set to `0`, the cache is inactive. When you use `decision_cache`, make sure you set `sampled_cache_size` to a value much higher than `num_traces` so that decisions for trace IDs are kept longer than the span data for the trace.
+`decision_cache` can contain two keys:
+- `sampled_cache_size`: Configures the amount of trace IDs to be kept in an LRU cache,


Suggested change

- `sampled_cache_size`: Configures the amount of trace IDs to be kept in an LRU cache,

- `sampled_cache_size`: Configures the number of trace IDs to be kept in an LRU cache,

clayton-cornell · 2025-01-02T21:05:02Z

docs/sources/reference/components/otelcol/otelcol.processor.tail_sampling.md

+- `sampled_cache_size`: Configures the amount of trace IDs to be kept in an LRU cache,
+  persisting the "keep" decisions for traces that may have already been released from memory. 
+  By default, the size is 0 and the cache is inactive.
+- `non_sampled_cache_size`: Configures amount of trace IDs to be kept in an LRU cache,


Suggested change

- `non_sampled_cache_size`: Configures amount of trace IDs to be kept in an LRU cache,

- `non_sampled_cache_size`: Configures number of trace IDs to be kept in an LRU cache,

clayton-cornell · 2025-01-02T21:05:35Z

docs/sources/reference/components/otelcol/otelcol.processor.tail_sampling.md

+  By default, the size is 0 and the cache is inactive.
+
+You may want to vary the size of the `decision_cache` depending on how many "keep" vs "drop" decisions you expect from your policies. 
+For example, you may allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.


Suggested change

For example, you may allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.

For example, you can allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.

clayton-cornell · 2025-01-02T21:05:52Z

docs/sources/reference/components/otelcol/otelcol.processor.tail_sampling.md

+
+You may want to vary the size of the `decision_cache` depending on how many "keep" vs "drop" decisions you expect from your policies. 
+For example, you may allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.
+Additionally, when using `decision_cache`, configure it with a much higher value than `num_traces` so decisions for trace IDs are kept longer than the span data for the trace.


Suggested change

Additionally, when using `decision_cache`, configure it with a much higher value than `num_traces` so decisions for trace IDs are kept longer than the span data for the trace.

Additionally, when you use `decision_cache`, configure it with a much higher value than `num_traces` so decisions for trace IDs are kept longer than the span data for the trace.

Upgrade to OTel v0.116.0

ed0b0fd

ptodev requested review from clayton-cornell and a team as code owners December 24, 2024 15:28

ptodev commented Dec 24, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md Show resolved Hide resolved

dehaansa reviewed Dec 25, 2024

View reviewed changes

docs/sources/reference/components/otelcol/otelcol.processor.k8sattributes.md Outdated Show resolved Hide resolved

ptodev added 2 commits December 30, 2024 12:24

Fix link in otelcol.processor.k8sattributes docs

8f394da

Update to a forked Beyla.

e4dfef1

The forked Beyla version contains an updated OTel dependency.

ptodev mentioned this pull request Dec 30, 2024

Upgrade to OTel v0.116.0 grafana/beyla#1484

Open

ptodev commented Dec 30, 2024

View reviewed changes

ptodev added 2 commits January 1, 2025 22:35

Register feature gate in the non-test package.

0a46b6e

The non-test package needs to have access to the gate, so that when it's called from non-test code it can register the feature gate with certainty.

Set a default ReporterPeriod in otelcol.exporter.datadog

66cd98f

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Jan 2, 2025

clayton-cornell reviewed Jan 2, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to OTel v0.116.0 #2314

Upgrade to OTel v0.116.0 #2314

ptodev commented Dec 24, 2024 •

edited

Loading

github-actions bot commented Dec 24, 2024 •

edited

Loading

ptodev Dec 30, 2024

ptodev Jan 1, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

clayton-cornell Jan 2, 2025

		//TODO: Do not merge this. Wait for upstream to upgrade the main branch, or to release a new version.
		replace github.com/grafana/beyla => github.com/grafana/beyla v1.9.1-0.20241230130037-7083b65bf473

		* The ones under `protocol > otlp`. This is useful for temporary problems with a specific backend, like transient network issues.
		* The ones top-level ones for `otelcol.exporter.loadbalancing` itself.

	This are useful for highly elastic environments like Kubernetes,
	This is useful for highly elastic environments like Kubernetes,

	By default, `otelcol.processor.k8sattributes` will be ready as soon as it starts, even if no metadata has been fetched yet.
	By default, `otelcol.processor.k8sattributes` is ready as soon as it starts, even if no metadata has been fetched yet.

	Then the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked.
	Then, the processor will not be ready until the metadata is fully synced. As a result, the start-up of {{< param "PRODUCT_NAME" >}} will be blocked.

	`otelcol.processor.k8sattributes` will become unhealthy and will fail to start.
	`otelcol.processor.k8sattributes` will become unhealthy and fail to start.

	However, they may not be able to send telemetry to `otelcol.processor.k8sattributes`.
	However, they may be unable to send telemetry to `otelcol.processor.k8sattributes`.

		@@ -143,12 +156,30 @@ The `annotation` block configures how to extract Kubernetes annotations.

		{{< docs/shared lookup="reference/components/extract-field-block.md" source="alloy" version="<ALLOY_VERSION>" >}}

		{{< admonition type="warning" >}}

	{{< admonition type="warning" >}}
	{{< admonition type="caution" >}}

		This example will add the same new `"documentId"="12345678"` attribute as the previous example.
		However, it will now result in an unchanged span name (/api/v1/document/12345678/update).

	- `sampled_cache_size`: Configures the amount of trace IDs to be kept in an LRU cache,
	- `sampled_cache_size`: Configures the number of trace IDs to be kept in an LRU cache,

	- `non_sampled_cache_size`: Configures amount of trace IDs to be kept in an LRU cache,
	- `non_sampled_cache_size`: Configures number of trace IDs to be kept in an LRU cache,

	For example, you may allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.
	For example, you can allocate a larger `non_sampled_cache_size` if you expect most traces to be dropped.

	Additionally, when using `decision_cache`, configure it with a much higher value than `num_traces` so decisions for trace IDs are kept longer than the span data for the trace.
	Additionally, when you use `decision_cache`, configure it with a much higher value than `num_traces` so decisions for trace IDs are kept longer than the span data for the trace.

Upgrade to OTel v0.116.0 #2314

Are you sure you want to change the base?

Upgrade to OTel v0.116.0 #2314

Conversation

ptodev commented Dec 24, 2024 • edited Loading

PR Description

Community components

Which issue(s) this PR fixes

PR Checklist

github-actions bot commented Dec 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ptodev commented Dec 24, 2024 •

edited

Loading

github-actions bot commented Dec 24, 2024 •

edited

Loading