Hi all,
I am resending this message again in case you missed it. (Bcc'ing
contributors who had recent activities on SparkRunner and SparkReceiverIO)
Particularly, for users or contributors of "*SparkRunner"* and "
*SparkReceiverIO"*, please take a look and feel free to share your ideas or
concer
That is not guaranteed to work. The backend classes may sit inside any
staged JARs from the users. For example, users can make a uber jar of their
pipeline and its dependency including the backend classes. Then the class
path will only has that uber jar, and we cannot tell what’s inside without
dec
What about scanning the classpath and removing any known logging backends
so that ours is the only one present?
On Wed, Feb 5, 2025 at 11:37 AM Shunping Huang
wrote:
> I agree that, ideally, users should delegate control of the logging
> backend to the runner or SDK harness to ensure logs are co
I agree that, ideally, users should delegate control of the logging backend
to the runner or SDK harness to ensure logs are correctly captured.
Dataflow, as you know, has a mechanism for collecting logs written by the
JUL backend (via the default slf4j-jdk14 bridge) and sending them to Cloud
Loggin
Hi all,
(Sorry Kenn, I didn't mean to interrupt the flow of your previous
conversation. I was just finishing this email when yours came through...)
I need some opinion from you all regarding Spark and the SLF4J 2.x upgrade.
Here is some background.
- Spark used to have compile dependency on
Thanks for the very nice document. Logging has been a source of complexity
since the beginning of Beam, and upgrading to SLF4J2 is getting urgent.
I do have big picture questions/concerns:
- For non-portable runners/pipelines, shouldn't the runner totally control
logging backends so that it can
Hi everyone,
I put together a short doc to summarize the existing logging
infrastructure(dependencies) in Beam Java and outline a plan to improve it.
Basically, we are on the path towards slf4j 2.x.
https://docs.google.com/document/d/1IkbiM4m8D-aB3NYI1aErFZHt6M7BQ-8eCULh284Davs/edit?usp=sharing
