taranlu-houzz commented on PR #64067:
URL: https://github.com/apache/airflow/pull/64067#issuecomment-4166648799
Hi @Subham-KRLX, I just wanted to double check that this PR will fully
address the issue of logs not showing up in container stdout when using
LocalExecutor, which is what I was trying to fix via: #63960.
This is a Claude summary of what was tried locally:
```
Attempted to apply this fix (#64067) locally via airflowLocalSettings
(Helm ConfigMap for /opt/airflow/config/airflow_local_settings.py) on Airflow
3.1.6 + LocalExecutor.
Two issues encountered:
1. The PR branch imports ElasticsearchRemoteLogIO which doesn't exist in
apache-airflow-providers-elasticsearch==6.4.2 (bundled with 3.1.6) — immediate
ImportError at
startup.
2. After working around that by using the 3.1.6-native equivalent (setting
remote_logging=True, elasticsearch.host, write_stdout=True, write_to_es=False),
Airflow
started and tasks ran successfully — but task logs still did not appear in
container stdout.
Root cause: write_stdout=True makes the task subprocess write to its
stdout, but with LocalExecutor the supervisor connects subprocess stdout to an
internal pipe and
doesn't forward it to the container's stdout unless
subprocess_logs_to_stdout=True is passed to supervise(). The containerized
executor sets this flag; LocalExecutor
does not. This fix alone is not sufficient for LocalExecutor users who
need task logs on stdout.
Workaround that does work: patching supervise() directly via
airflowLocalSettings:
import functools
import airflow.sdk.execution_time.supervisor as _sup
_orig_supervise = _sup.supervise
@functools.wraps(_orig_supervise)
def _patched_supervise(*args, **kwargs):
kwargs.setdefault("subprocess_logs_to_stdout", True)
return _orig_supervise(*args, **kwargs)
_sup.supervise = _patched_supervise
The proper fix would be for LocalExecutor._execute_work to pass
subprocess_logs_to_stdout=True to supervise(), or for a config option to be
exposed for it.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
