Hi Gordon
Any chance we could get this reviewed and released to the central repo?
We're currently forced to use a Flink version that has a nasty bug causing
an operational nightmare
Many thanks
Fil
On Sat, 19 Aug 2023 at 01:38, Galen Warren via user
wrote:
> Gotcha, makes sense as to the original division.
>
> >> Can this be solved by simply passing in the path to the artifacts
>
> This definitely works if we're going to be copying the artifacts on the
> host side -- into the build context -- and then from the context into the
> image. It only gets tricky to have a potentially varying path to the
> artifacts if we're trying to *directly *include the artifacts in the
> Docker context -- then we have a situation where the Docker context must
> contain both the artifacts and playground files, with (potentially)
> different root locations.
>
> Maybe the simplest thing to do here is just to leave the playground as-is
> and then copy the artifacts into the Docker context manually, prior to
> building the playground images. I'm fine with that. It will mean that each
> Statefun release will require two PRs and two sets of build/publish steps
> instead of one, but if everyone else is fine with that I am, too. Unless
> anyone objects, I'll go ahead and queue up a PR for the playground that
> makes these changes.
>
> Also, I should mention -- in case it's not clear -- that I have already
> built and run the playground examples with the code from the PR and
> everything worked. So that PR is ready to move forward with review, etc.,
> at this point.
>
> Thanks.
>
>
>
>
>
>
>
> On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai
> wrote:
>
>> Hi Galen,
>>
>> The original intent of having a separate repo for the playground repo,
>> was that StateFun users can just go to that and start running simple
>> examples without any other distractions from the core code. I personally
>> don't have a strong preference here and can understand how it would make
>> the workflow more streamlined, but just FYI on the reasoning why are
>> separate in the first place.
>>
>> re: paths for locating StateFun artifacts.
>> Can this be solved by simply passing in the path to the artifacts? As
>> well as the image tag for the locally build base StateFun image. They could
>> probably be environment variables.
>>
>> Cheers,
>> Gordon
>>
>> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
>> user@flink.apache.org> wrote:
>>
>>> Yes, exactly! And in addition to the base Statefun jars and the jar for
>>> the Java SDK, it does an equivalent copy/register operation for each of the
>>> other SDK libraries (Go, Python, Javascript) so that those libraries are
>>> also available when building the playground examples.
>>>
>>> One more question: In order to copy the various build artifacts into the
>>> Docker containers, those artifacts need to be part of the Docker context.
>>> With the playground being a separate project, that's slightly tricky to do,
>>> as there is no guarantee (other than convention) about the relative paths
>>> of *flink-statefun* and* flink-statefun-playground *in someone's local
>>> filesystem. The way I've set this up locally is to copy the playground into
>>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
>>> such that I can set the Docker context to the root folder of
>>> *flink-statefun* and then have access to any local code and/or build
>>> artifacts.
>>>
>>> I'm wondering if there might be any appetite for making that move
>>> permanent, i.e. moving the playground to *flink-statefun*/playground
>>> and deprecating the standalone playground project. In addition to making
>>> the problem of building with unreleased artifacts a bit simpler to solve,
>>> it would also simplify the process of releasing a new Statefun version,
>>> since the entire process could be handled with a single PR and associated
>>> build/deploy tasks. In other words, a single PR could both update and
>>> deploy the Statefun package and the playground code and images.
>>>
>>> As it stands, at least two PRs would be required for each Statefun
>>> version update -- one for *flink-statefun* and one for
>>> *flink-statefun-playground*.
>>>
>>> Anyway, just an idea. Maybe there's an important reason for these
>>> projects to remain separate. If we do want to keep the playground project
>>> where it is, I could solve the copying problem by requiring the two
>>> projects to be siblings in the file system and by pre-copying the local
>>> build artifacts into a location accessible by the existing Docker contexts.
>>> This would still leave us with the need to have two PRs and releases
>>> instead of one, though.
>>>
>>> Thanks for your help!
>>>
>>>
>>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai
>>> wrote:
>>>
Hi Galen,
> locally built code is copied into the build containers
so that it can be accessed during the build.
That's exactly what we had been doing for release testing, yes. Sorry
