I like the vision and the effect - even though we do not have much
experiences with OTEL, I saw what it can do and I like what it "provides".
And I found it very appealing when I saw it in "action" during some talks.
Maybe it would be useful to see a somewhat realistic example of what can be
done
As part of AIP-49, tasks logs are actually included, as span events.
It can be turned on/off but when it's enabled, the content of the task log
will be included as span events and emitted.
Since task logs can be quite a large size, depending on the OTEL backend,
the log contents can be truncated as
Hi Kaxil,
The OTEL provider would provide hooks for the developers to be able to
create spans. Also, if the provider is running on top of Airflow that does
not have the OTEL feature enabled/available, it will emit traces on the DAG
run level, meaning task instance runs associated with the running
Howard,
I am intrigued by this, but unclear on what this would actually look like
and what benefits it would add.
Specifically, I believe that AIP-49 adds support for OTEL emission of
metrics and traces, but NOT task logs from Airflow.
I am probably being dense here, but I don't quite understand
@howard: What sort of Operators or Hooks are you planning for the OTEL
provider?
I am favour of deeper integration for OTEL and Airflow but I don't know
what Operators, Hooks or other things will be part of the provider.
Regards,
Kaxil
On Wed, 24 Jul 2024 at 01:00, Jarek Potiuk wrote:
> +1. I
+1. I like the idea of how it will add a possibility to customize OTEL
metrics and spans possibly. With Airflow 2.10 I would also love to see some
guidelines and description and maybe some kind of simple How-TO on how you
can make "more" use of OTEL - for example users could use
auto-instrumentatio
Hi Apache Airflow Community,
I hope this message finds you well.
I am writing to propose the addition of a new provider to Apache Airflow
for OpenTelemetry (https://opentelemetry.io). OpenTelemetry is an emerging
standard for instrumentation of services and applications, and recently has
matured
