Hey,
I'm getting this error:
apache_beam.io.filesystem.BeamIOError: Match operation failed with
exceptions {'s3://fiverr-data-science-de
v/beam_poc/beam/wc/input.txt': BeamIOError("exists() operation failed with
exceptions {'s3://fiverr-data-sc
ience-dev/beam_poc/beam/wc/input.txt': ValueError('Mus
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ical.marudot.com//iCal Event Maker
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:America/Chicago
TZURL:http://tzurl.org/zoneinfo-outlook/America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T0200
+d...@beam.apache.org
Digging more into the code it looks more like a bug that was introduced in
a recent PR (https://github.com/apache/beam/pull/13180). It seems that
pipeline_options is absent when using external workers and it causes the
S3IO to fail in an assertion in its constructor.
I wond
Hi,
We plan to use UDF on our sql. We want to achieve some kind of
filtering based on internal states. We want to update that internal state
with a separate async thread in UDF. Before implementing that thing I want
to get your options. Is there any limitation for UDF to have multi-thread
implemen
The problem that I can think of is maybe before the async call is
completed, the UDF life cycle has reached to the end.
-Rui
On Wed, Feb 10, 2021 at 12:34 PM Talat Uyarer
wrote:
> Hi,
>
> We plan to use UDF on our sql. We want to achieve some kind of
> filtering based on internal states. We wa
Does beam create UDF function for every bundle or in setup of pipeline ?
I will keep internal state in memory. The Async thread will update that in
memory state based on an interval such as every hour etc. If beam keeps UDF
instance more than one bundle it is ok for me.
On Wed, Feb 10, 2021, 12:
I am still awaiting my gift voucher :)
On Wed, 10 Feb 2021 at 16:51, Carlos Camacho
wrote:
> Hi everyone,
> *Thank you for helping us choose a date and time for our User Experience
> Research Findings Readout for Apache Beam.*
>
> The winner option is *Thursday, February 11th at 11:00 AMCST / 6
Thanks Rui to remind me lifecycle of UDF. LOoks liek there is no any
lifecycle. I checked the code looks like we create UDF's instance for each
message:
org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.runtime.SqlFunctions.isTrue(new
> com.paloaltonetworks.cortex.streamcompute.functions.M
