Maneendra created FLINK-36765:
-
Summary: How to Handle Multi-Type Maps in Avro Schema with Flink
Table API?
Key: FLINK-36765
URL: https://issues.apache.org/jira/browse/FLINK-36765
Project: Flink
xuyang created FLINK-36396:
--
Summary: Remove dependency flink-java in all flink-table-api-xxx
module
Key: FLINK-36396
URL: https://issues.apache.org/jira/browse/FLINK-36396
Project: Flink
Issue
xuyang created FLINK-36334:
--
Summary: Remove the Disable markers from the tests in the various
TypeSerializerUpgradeTestBase subclasses that have been moved from the
flink-table-api-scala module
Key: FLINK-36334
URL
bear.xiong created FLINK-35741:
--
Summary: It' tell me 'This is a bug. Please file an issue.' when I
executing the Flink Table API
Key: FLINK-35741
URL: https://issues.apache.org/jira/bro
haojie created FLINK-33987:
--
Summary: Flink Table API Support file extention or suffix
Key: FLINK-33987
URL: https://issues.apache.org/jira/browse/FLINK-33987
Project: Flink
Issue Type: New Feature
Prabhu Joseph created FLINK-33536:
-
Summary: Flink Table API CSV streaming sink throws "IOException:
Stream closed"
Key: FLINK-33536
URL: https://issues.apache.org/jira/browse/FLINK-33536
Jaya Ananthram created FLINK-28513:
--
Summary: Flink Table API CSV streaming sink throws
SerializedThrowable exception
Key: FLINK-28513
URL: https://issues.apache.org/jira/browse/FLINK-28513
Project
Sergey Nuyanzin created FLINK-27698:
---
Summary: [JUnit5 Migration] Module: flink-table-api-java-bridge
Key: FLINK-27698
URL: https://issues.apache.org/jira/browse/FLINK-27698
Project: Flink
Sergey Nuyanzin created FLINK-27673:
---
Summary: [JUnit5 Migration] Module: flink-table-api-scala
Key: FLINK-27673
URL: https://issues.apache.org/jira/browse/FLINK-27673
Project: Flink
Issue
Sergey Nuyanzin created FLINK-27323:
---
Summary: [JUnit5 Migration] Module: flink-table-api-java
Key: FLINK-27323
URL: https://issues.apache.org/jira/browse/FLINK-27323
Project: Flink
Issue
I think what you need is something like:
table.executeInsert("MySink").*await()*
On Tue, Mar 8, 2022 at 8:24 PM Adesh Dsilva wrote:
> Hi,
>
> I wrote a simple program using Flink Table API.
> There is a Table source reading from an avro file, after doing some
> operat
Hi,
I wrote a simple program using Flink Table API.
There is a Table source reading from an avro file, after doing some operations
the result is stored using a sink csv table using `executeInsert()`
The program works fine and creates a csv file however the flink command does
not wait for job
Francesco Guardiani created FLINK-25229:
---
Summary: Introduce flink-table-api-bridge-common
Key: FLINK-25229
URL: https://issues.apache.org/jira/browse/FLINK-25229
Project: Flink
Issue
James Kim created FLINK-24285:
-
Summary: Flink Table API Could not find any format factory for
identifier 'csv' in the classpath.
Key: FLINK-24285
URL: https://issues.apache.org/jira/browse/F
e cpu time to the GC threads.
Thank you~
Xintong Song
On Fri, Mar 26, 2021 at 6:48 AM Sivaraman Venkataraman, Aswin Ram <
aswin.ram.sivaraman.venkatara...@sap.com> wrote:
> Hi Everyone,
> Hope you are doing well. We are currently using Flink Table API (Flink
> Version-1.12.0)
Hi Everyone,
Hope you are doing well. We are currently using Flink Table API (Flink
Version-1.12.0) to stream data from Kafka and store it in Google Cloud Storage.
The file format we are using to store data is Parquet. Initially the Flink job
worked perfectly fine and we were able to stream
Hi all,
The voting time for FLIP-36 has passed. I'm closing the vote now.
There were 3 binding votes:
- Aljoscha (binding)
- Timo (binding)
- Becket (binding)
There were no disapproving votes.
Thus, FLIP-36 has been accepted.
Thanks everyone for joining the discussion and giving feedback!
Bes
+1 (binding)
Best,
Aljoscha
apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
> >>
> >> [2]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
> >>
> >>
> >
>
>
nk-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
tes.
Best,
Xuannan
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
FLIP-36%3A+Support+Interactive+Programming+in+Flink
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
Hi everyone,
@Aljoscha, I have updated the Per-Job mode Section of the FLIP.
It seems that people involved in the discussion have reach a consensus. If
there are no more comments, I would like to start the voting thread tomorrow.
Best,
Xuannan
On Sep 15, 2020, 6:18 PM +0800, Aljoscha Krettek ,
On 15.09.20 10:54, Xuannan Su wrote:
One way of solving this is to let the CatalogManager probe the existence of the
IntermediateResult so that the planner can decide if the cache table should be
used.
That could be a reasonable solution, yes.
Best,
Aljoscha
Hi Aljoscha,
I thought about relying on the failover mechanism to re-execute the whole graph
when the cache doesn’t exist. The only concern I have is that every job that
uses the cache table in the per-job cluster will have to go through the
following process,
job submit -> job fail because of
On 15.09.20 07:00, Xuannan Su wrote:
Thanks for your comment. I agree that we should not introduce tight coupling
with PipelineExecutor to the execution environment. With that in mind, to
distinguish the per-job and session mode, we can introduce a new method, naming
isPerJobModeExecutor, in t
Hi Aljoscha,
Thanks for your comment. I agree that we should not introduce tight coupling
with PipelineExecutor to the execution environment. With that in mind, to
distinguish the per-job and session mode, we can introduce a new method, naming
isPerJobModeExecutor, in the PipelineExecutorFactor
On 10.09.20 09:00, Xuannan Su wrote:
How do you imagine that? Where do you distinguish between per-job and
session mode?
The StreamExecutionEnvironment can distinguish between per-job and session mode
by the type of the PipelineExecutor, i.e, AbstractJobClusterExecutor vs
AbstractSessionCluste
-forth
communication to determine existing cluster partitions and modify the
job graph. Or even further down the stack, because as far as I know,
`PipelineExecutor` works with `StreamGraph`.
`flink-table-api-java` has no dependency to `flink-runtime`. This has
been done on purpose.
2) API
I st
> > 1b) The FLIP states: "The default table service stores the metadata in
> > > the client (e.g. TableEnvironment)"
> > >
> > > `TableEnvironment` is not a client. Similar to `Table`, it is an API
> > > class that just delegates to other session co
Leonard Xu created FLINK-19148:
--
Summary: Table crashed in Flink Table API & SQL Docs
Key: FLINK-19148
URL: https://issues.apache.org/jira/browse/FLINK-19148
Project: Flink
Issue Type:
tion to determine existing cluster partitions and modify the
job graph. Or even further down the stack, because as far as I know,
`PipelineExecutor` works with `StreamGraph`.
`flink-table-api-java` has no dependency to `flink-runtime`. This has
been done on purpose.
2) API
I still see the rejected
for a back-and-forth
> communication to determine existing cluster partitions and modify the
> job graph. Or even further down the stack, because as far as I know,
> `PipelineExecutor` works with `StreamGraph`.
>
> `flink-table-api-java` has no dependency to `flink-runtime`. Thi
fit for a back-and-forth
communication to determine existing cluster partitions and modify the
job graph. Or even further down the stack, because as far as I know,
`PipelineExecutor` works with `StreamGraph`.
`flink-table-api-java` has no dependency to `flink-runtime`. This has
been done on
> > > > the logic dealing with cached tables can be orthogonal to all of these.
> > > > Hence I expect you could have a more detailed description here.
> > > >
> > > > 3. What's the effect of calling TableEnvironment.close()? You already
> &g
here.
> > >
> > > 3. What's the effect of calling TableEnvironment.close()? You already
> > > explained this would drop all caches this table env has,
> > > could you also explain where other functionality still works for this
> > table
>
Robert Metzger created FLINK-18697:
--
Summary: Adding flink-table-api-java-bridge_2.11 to a Flink job
kills the IDE logging
Key: FLINK-18697
URL: https://issues.apache.org/jira/browse/FLINK-18697
ty still works for this
> table
> > env? Like can use still create/drop tables/databases/function
> > through this table env? What happens to the catalog and all temporary
> > objects of this table env?
> >
> > One minor comment: I noticed you used some not existing A
bles/databases/function
> through this table env? What happens to the catalog and all temporary
> objects of this table env?
>
> One minor comment: I noticed you used some not existing API in the examples
> you gave, like table.collect(), which is a little
> misleading.
>
> B
which is a little
misleading.
Best,
Kurt
On Thu, Jul 9, 2020 at 4:00 PM Xuannan Su wrote:
> Hi folks,
>
> I'd like to revive the discussion about FLIP-36 Support Interactive
> Programming in Flink Table API
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-
Hi folks,
I'd like to revive the discussion about FLIP-36 Support Interactive Programming
in Flink Table API
https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
The FLIP proposes to add support for interactive programming in Flink Table
Aihua Li created FLINK-17907:
Summary: flink-table-api-java: Compilation failure
Key: FLINK-17907
URL: https://issues.apache.org/jira/browse/FLINK-17907
Project: Flink
Issue Type: Bug
Hi folks,
As the feature freeze of Flink 1.11 has passed and the release branch is
cut, I'd like to revive this discussion thread of FLIP-36[1]. A quick
summary of FLIP-36:
The FLIP proposes to add support for interactive programming in Flink Table
API. Specifically, it let users cach
roduced to avoid recomputation of an intermediate
table to support interactive programming in Flink Table API. And I think
the materialized view needs more discussion and certainly deserves a whole
new FLIP.
Please let me know your thought.
Best,
Xuannan
On Wed, Apr 29, 2020 at 3:53 PM Xuannan
t; > Hi all,
> > >
> > > I'd like to start the vote for FLIP-36[1], which has been discussed in
> > > thread[2].
> > >
> > > The vote will be open for 72h, until May 3, 2020, 07:00 AM UTC, unless
> > > there's an objection.
> > >
> > > Best,
> > > Xuannan
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
> > > [2]
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
> > >
> >
> >
>
y 3, 2020, 07:00 AM UTC, unless
> > there's an objection.
> >
> > Best,
> > Xuannan
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
> > [2]
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
> >
>
>
36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
rogramming+in+Flink
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-36-Support-Interactive-Programming-in-Flink-Table-API-td40592.html
following three cases: 1. the user explicitly call the
>> > > > invalidateCache() method 2. the TableEnvironment is closed 3.
>> failure
>> > > > happens on the TM. When that happens, the intermeidate result will
>> not
>> > be
>> > >
the intermeidate result will
> not
> > be
> > > > available unless it is re-generated.
> > > >
> > > > 3. In the "semantic of cache() method" section, the description "The
> > > > > semantic of the *cache() *method is a little
ion "The
> > > > semantic of the *cache() *method is a little different depending on
> > > whether
> > > > auto caching is enabled or not." seems not explained.
> > > >
> > >
> > > This line is actually outdated and sh
not." seems not explained.
> > >
> >
> > This line is actually outdated and should be removed, as we are not
> adding
> > the auto caching functionality in this FLIP. Auto caching will be added
> in
> > the future, and the semantic of cac
in this FLIP. Auto caching will be added in
> the future, and the semantic of cache() when auto caching is enabled will
> be discussed in detail by a new FLIP. I will remove the descriptor to avoid
> further confusion.
>
>
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> &
when auto caching is enabled will
be discussed in detail by a new FLIP. I will remove the descriptor to avoid
further confusion.
> Thanks,
>
> Jiangjie (Becket) Qin
>
>
>
> On Wed, Apr 22, 2020 at 4:00 PM Xuannan Su wrote:
>
> > Hi folks,
> >
> &g
iangjie (Becket) Qin
On Wed, Apr 22, 2020 at 4:00 PM Xuannan Su wrote:
> Hi folks,
>
> I'd like to start the discussion about FLIP-36 Support Interactive
> Programming in Flink Table API
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Pr
Hi folks,
I'd like to start the discussion about FLIP-36 Support Interactive
Programming in Flink Table API
https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
The FLIP proposes to add support for interactive programming in Flink Tabl
Kurt Young created FLINK-16642:
--
Summary: CSV TableSource / TableSink shouldn't be in
flink-table-api-java-bridge package
Key: FLINK-16642
URL: https://issues.apache.org/jira/browse/FLINK-16642
Pr
Jark Wu created FLINK-16146:
---
Summary: Improve end-to-end usability of Flink Table API & SQL
Key: FLINK-16146
URL: https://issues.apache.org/jira/browse/FLINK-16146
Project: Flink
Issue
Dawid Wysakowicz created FLINK-15947:
Summary: Finish moving scala expression DSL to
flink-table-api-scala
Key: FLINK-15947
URL: https://issues.apache.org/jira/browse/FLINK-15947
Project: Flink
Hi, all
sorry to resend a email with correct title .
Found a fatal bug starting from Flink 1.6, which cause Flink Table API can
not correctly extract table schema .
Jira
https://issues.apache.org/jira/projects/FLINK/issues/FLINK-13603?filter=allopenissues
there is a change on Flink-core
Hi, all
Found a fatal bug starting from Flink 1.6, which cause Flink Table API can
not correctly extract table schema .
Jira
https://issues.apache.org/jira/projects/FLINK/issues/FLINK-13603?filter=allopenissues
there is a change on Flink-core -> RowTypeInfo -> hashcode , this chang
Yu Du created FLINK-13603:
-
Summary: Flink Table ApI not working with nested Json schema
starting From 1.6.x
Key: FLINK-13603
URL: https://issues.apache.org/jira/browse/FLINK-13603
Project: Flink
sunjincheng created FLINK-13470:
---
Summary: Enhancements to Flink Table API for blink planner
Key: FLINK-13470
URL: https://issues.apache.org/jira/browse/FLINK-13470
Project: Flink
Issue Type
Timo Walther created FLINK-13045:
Summary: Move Scala expression DSL to flink-table-api-scala
Key: FLINK-13045
URL: https://issues.apache.org/jira/browse/FLINK-13045
Project: Flink
Issue
Timo Walther created FLINK-13028:
Summary: Move expression resolver to flink-table-api-java
Key: FLINK-13028
URL: https://issues.apache.org/jira/browse/FLINK-13028
Project: Flink
Issue Type
PM Becket Qin wrote:
> >
> >> Thanks Piotr, for the +1 and all the patient discussion :)
> >>
> >> On Wed, Mar 13, 2019 at 3:53 PM Piotr Nowojski
> >> wrote:
> >>
> >>> Hi Becket,
> >>>
> >>> Thank you for driving the
wrote:
>>
>>> Hi Becket,
>>>
>>> Thank you for driving the effort and writing down the detailed proposal.
>>> To me this FLIP looks good and it has +1 from me.
>>>
>>> Piotr Nowojski
>>>
>>> > On 12 Mar 2019
sunjincheng created FLINK-12308:
---
Summary: Support python language in Flink Table API
Key: FLINK-12308
URL: https://issues.apache.org/jira/browse/FLINK-12308
Project: Flink
Issue Type: New
t; Hi folks,
>> >
>> > We would like to start the discussion thread about FLIP-36 support
>> > interactive programming in Flink Table API.
>> >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+i
t;
> > On 12 Mar 2019, at 13:21, Becket Qin wrote:
> >
> > Hi folks,
> >
> > We would like to start the discussion thread about FLIP-36 support
> > interactive programming in Flink Table API.
> >
> >
> https://cwiki.apache.org/confluence/display/FLI
FLIP-36 support
> interactive programming in Flink Table API.
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
>
> There has been an extended discussion[1] in the mailing list. To quick
> recap, we propose to add capability of cac
Hi folks,
We would like to start the discussion thread about FLIP-36 support
interactive programming in Flink Table API.
https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
There has been an extended discussion[1] in the mailing list. To quick
Timo Walther created FLINK-11451:
Summary: Move *QueryConfig and TableDescriptor to
flink-table-api-java
Key: FLINK-11451
URL: https://issues.apache.org/jira/browse/FLINK-11451
Project: Flink
Hey Becket,
+1 From my side
Piotrek
> On 14 Jan 2019, at 14:43, Becket Qin wrote:
>
> Hi Seth,
>
> Thanks for the feedback. Re-caching makes sense to me. Piotr and I had some
> offline discussion and we generally reached consensus on the following API:
>
> {
> /**
>* Cache this table to
Hi Seth,
Thanks for the feedback. Re-caching makes sense to me. Piotr and I had some
offline discussion and we generally reached consensus on the following API:
{
/**
* Cache this table to builtin table service or the specified customized
table service.
*
* This method provides a hi
I spoke to Piotr a little bit offline and I wanted to comment with a summary of
our discussion and what I believe is most intuitive cache model from a users
perspective.
(I am making up some class names here, not looking to bike shed feel free to
change the names how ever you see fit).
A cac
Hi Piotr,
1. `env.getCacheService().releaseCacheFor(cachedT);` vs
`cachedT.releaseCache();`
It doesn't matter which signature we provide. To those who write the
function, "releasing the cache" is not a "side effect", it is exactly what
they wanted. Even if they know that they may be releasing some
Hi,
I know that it still can have side effects and that’s why I wrote:
> Something like this might be a better (not perfect, but just a bit better):
My point was that this:
void foo(Table t) {
val cachedT = t.cache();
...
env.getCacheService().releaseCacheFor(cachedT);
}
Should communicate
Just to clarify, when I say foo() like below, I assume that foo() must have
a way to release its own cache, so it must have access to table env.
void foo(Table t) {
...
t.cache(); // create cache for t
...
env.getCacheService().releaseCacheFor(t); // release cache for t
}
Thanks,
Jiangji
Hi Piotr,
I don't think it is feasible to ask every third party library to have
method signature with CacheService as an argument.
And even that signature does not really solve the problem. Imagine function
foo() looks like following:
void foo(Table t) {
...
t.cache(); // create cache for t
Hi,
I think that introducing ref counting could be confusing and it will be error
prone, since Flink-table’s users are not used to closing/releasing resources. I
was more objecting placing the `uncache()`/`dropCache()`/`releaseCache()`
(releaseCache sounds best to me) as a method in the “Table”
Hi Piotr,
You are right. There might be two intuitive meanings when users call
'a.uncache()', namely:
1. release the resource
2. Do not use cache for the next operation.
Case (1) would likely be the dominant use case. So I would suggest we
dedicate uncache() method to case (1), i.e. for resource
Hi Becket,
With `uncache` there are probably two features that we can think about:
a)
Physically dropping the cached table from the storage, freeing up the resources
b)
Hinting the optimizer to not cache the reads for the next query/table
a) Has the issue as I wrote before, that it seemed to
Hi Piotr,
Thanks for the proposal and detailed explanation. I like the idea of
returning a new hinted Table without modifying the original table. This
also leave the room for users to benefit from future implicit caching.
Just to make sure I get the full picture. In your proposal, there will also
Hi Becket!
After further thinking I tend to agree that my previous proposal (*Option 2*)
indeed might not be if would in the future introduce automatic caching. However
I would like to propose a slightly modified version of it:
*Option 4*
Adding `cache()` method with following signature:
Tabl
Happy New Year, everybody!
I would like to resume this discussion thread. At this point, We have
agreed on the first step goal of interactive programming. The open
discussion is the exact API. More specifically, what should *cache()*
method return and what is the semantic. There are three options:
Hi Piotrek,
1. Regarding optimization.
Sure there are many cases that the decision is hard to make. But that does
not make it any easier for the users to make those decisions. I imagine 99%
of the users would just naively use cache. I am not saying we can optimize
in all the cases. But as long as
Hi,
Thanks for the quick answer :)
Re 1.
I generally agree with you, however couple of points:
a) the problem with using automatic caching is bigger, because you will have to
decide, how do you compare IO vs CPU costs and if you pick wrong, additional IO
costs might be enormous or even can cr
Hi Piotrek,
Not sure if you noticed, in my last email, I was proposing `CacheHandle
cache()` to avoid the potential side effect due to function calls.
Let's look at the disagreement in your reply one by one.
1. Optimization chances
Optimization is never a trivial work. This is exactly why we s
Hi Becket,
> Regarding the chance of optimization, it might not be that rare. Some very
> simple statistics could already help in many cases. For example, simply
> maintaining max and min of each fields can already eliminate some
> unnecessary table scan (potentially scanning the cached table) if
Hi Becket,
Introducing CacheHandle seems too complicated. That means users have to
maintain Handler properly.
And since cache is just a hint for optimizer, why not just return Table
itself for cache method. This hint info should be kept in Table I believe.
So how about adding method cache and un
Hi Till and Piotrek,
Thanks for the clarification. That solves quite a few confusion. My
understanding of how cache works is same as what Till describe. i.e.
cache() is a hint to Flink, but it is not guaranteed that cache always
exist and it might be recomputed from its lineage.
Is this the core
Hi Becket,
I was aiming at semantics similar to 1. I actually thought that `cache()`
would tell the system to materialize the intermediate result so that
subsequent queries don't need to reprocess it. This means that the usage of
the cached table in this example
{
val cachedTable = a.cache()
va
Hi Becket,
> {
> val cachedTable = a.cache()
> val b = cachedTable.select(...)
> val c = a.select(...)
> }
>
> Semantic 1. b uses cachedTable as user demanded so. c uses original DAG as
> user demanded so. In this case, the optimizer has no chance to optimize.
> Semantic 2. b uses cachedTable
Another potential concern for semantic 3 is that. In the future, we may add
automatic caching to Flink. e.g. cache the intermediate results at the
shuffle boundary. If our semantic is that reference to the original table
means skipping cache, those users may not be able to benefit from the
implicit
Hi Piotrek,
Thanks for the reply. Thought about it again, I might have misunderstood
your proposal in earlier emails. Returning a CachedTable might not be a bad
idea.
I was more concerned about the semantic and its intuitiveness when a
CachedTable is returned. i..e, if cache() returns CachedTable
Hi Becket,
Sorry for not responding long time.
Regarding case1.
There wouldn’t be no “a.unCache()” method, but I would expect only
`cachedTableA1.dropCache()`. Dropping `cachedTableA1` wouldn’t affect
`cachedTableA2`. Just as in any other database dropping modifying one
independent table/mate
Hi Till,
It is true that after the first job submission, there will be no ambiguity
in terms of whether a cached table is used or not. That is the same for the
cache() without returning a CachedTable.
Conceptually one could think of cache() as introducing a caching operator
> from which you need
Hi，
All the recent discussions are focused on whether there is a problem if
cache() not return a Table.
It seems that returning a Table explicitly is more clear (and safe?).
So whether there are any problems if cache() returns a Table? @Becket
Best,
Jark
On Tue, 4 Dec 2018 at 22:27, Till Rohrm
It's true that b, c, d and e will all read from the original DAG that
generates a. But all subsequent operators (when running multiple queries)
which reference cachedTableA should not need to reproduce `a` but directly
consume the intermediate result.
Conceptually one could think of cache() as int
1 - 100 of 188 matches
Mail list logo
