Hi.
> Let me give you a concrete example. The Gateway may use a Generic catalog
> implementation backed by a Database (where table properties would be
stored
> in a DB). This would be a common setup in different Flink services.
> In this model the catalog itself needs DB credentials while the runn
Hey!
> Actually, a job requires same credentials to read the data or write data.
> When compiling job in the job manager, we don't require extra credentials
You are making a lot of assumptions here regarding the catalog
implementations that are not generally true.
Let me give you a concrete examp
Hi.
> My point regarding the credentials wasn't about lazy/non-lazy catalog
> initialization. The problem is that JM may run in an env where you don't
> necessarily want to have the same credentials at all.
Actually, a job requires same credentials to read the data or write data.
When compiling j
Hey!
My point regarding the credentials wasn't about lazy/non-lazy catalog
initialization. The problem is that JM may run in an env where you don't
necessarily want to have the same credentials at all.
> I agree we need 2 different runners or designs here. But I don't think we
> should only suppo
Hi, Gyula.
Thanks a lot for your response.
> In a production environment the gateway would hold the credentials to the
different catalogs and may even contain temporary tables etc.
In the FLIP-295[1], SQL Gateway has already supported using catalog stores
to initialize the catalog lazily. I thin
Hey!
I am a bit concerned about the design for gateway based submission model.
I think it's not a good model to access catalog information on the
JobManager side. In a production environment the gateway would hold the
credentials to the different catalogs and may even contain temporary tables
etc
Hi, eveyone.
The discussion doesn't receive any response for a while. I will close the
discussion and start the vote tomorrow if the discussion doesn't receive
any response today. Thanks all yours response!
Best,
Shengkai
Shengkai Fang 于2024年11月5日周二 14:00写道：
> Hi, Lincoln. Thanks for your resp
Hi, Lincoln. Thanks for your response.
> since both scriptPath and script(statements) can be null, we need to
clarify the behavior when both are empty, such as throwing an error
Yes, you are correct. I have updated the FLIP about this. When these fields
are both empty, the server throws an excep
Thanks Shengkai for driving this! Overall, looks good!\
I have two minor questions:
1. Regarding the interface parameters (including REST API
& Java interfaces), since both scriptPath and script(statements)
can be null, we need to clarify the behavior when both are
empty, such as throwing an error
Hi, Ferenc.
Thanks for your clarification. We can hard code these different options in
the sql-gateway module. I have updated the FLIP and PoC branch about this
part. But I think we should provide a unified API to ship artifacts to
different deployment.
Best,
Shengkai
Ferenc Csaky 于2024年11月4日
Hi, Xuyang. Thanks a lot for your response!
> Does that means we will support multi DMLs, multi DQLs, mixed DMLs & DQLs
in one sql script?
According to the doc[1], application mode only supports one job in ha
mode[2]. If users submit multiple jobs, dispatcher throws a
DuplicateJobSubmissionExcept
11 matches
Mail list logo
