Issues with passing complex JVM opts into process_variables in PortablePipelineOptions

[Bharath Kumara Subramanian]

Hi,

We are leveraging beam portability in java for achieving isolation between
user and framework(runner) code.

Here is how we spin up the user java process as a separate process

portableOptions.setJobServerConfig(PortableUtils.getServerDriverArgs().toArray(new
String[0]));
> portableOptions.setRunner(TestPortableRunner.class);
> portableOptions.setDefaultEnvironmentType(Environments.ENVIRONMENT_PROCESS);
> portableOptions.setEnvironmentOptions(
>     ImmutableList.of(
>         "process_command=" + getWorkerProcessCommand(),
>         "process_variables=LOG4J2_FILE_NAME=log4j2.xml,JAVA_OPTS="));
>
>

The JAVA_OPTS is currently empty and we run into issues when we populate
JAVA_OPTS with remote debugging flags such as agentlib:jdwp

I investigated a bit into why it doesn't work and it seems like the parsing
logic in environment doesn't handle complex process variables that have *,.*
Here is the snippet of the parsing logic

private static Map<String, String>
getProcessVariables(PortablePipelineOptions options) {
>   ImmutableMap.Builder<String, String> variables = ImmutableMap.builder();
>   String assignments =
>       PortablePipelineOptions.getEnvironmentOption(options, 
> processVariablesOption);
>   for (String assignment : assignments.split(",", -1)) {
>     String[] tokens = assignment.split("=", -1);
>     if (tokens.length == 1) {
>       throw new IllegalArgumentException(
>           String.format("Process environment variable '%s' is not assigned a 
> value.", tokens[0]));
>     }
>     variables.put(tokens[0], tokens[1]);
>   }
>   return variables.build();
> }
>
>
Questions for the community

   1.  Is there any other way to pass in configurations the starting
   process? I tried DefaultEnvironmentConfiguration although it can't be used
   in conjunction with EnvironmentOptions and ran into different issues around
   external worker URL w/ DefaultEnvironmentConfiguration
   2. I noticed similar issues with Google Container Tools (
   https://github.com/GoogleContainerTools/jib/pull/3522/files) and it got
   fixed by enhancing the parsing logic. Is the path forward to fix the above
   parsing logic in OSS?


Thanks,
Bharath