Not knowing the library or the config needs I do not have a suggestion. If the 
config is accumulated from inputs and needs to see all inputs I would suggest 
setting parallelism to 1 as an experiment, but it would need a redesign to run 
in parallel. 

Michael

Sent from my iPad

> On Apr 26, 2018, at 12:50 AM, Soheil Pourbafrani <soheil.i...@gmail.com> 
> wrote:
> 
> Thanks, So what is your suggestion to solve the problem? Is it possible to 
> use Broadcast Variables for this senario?
> 
>> On Thu, Apr 26, 2018 at 10:57 AM, TechnoMage <mla...@technomage.com> wrote:
>> What parallelism are you using?  If it is > 1 you can not rely on the config 
>> value being passed to each of the parsing functions as they are running on 
>> separate threads or even separate machines.
>> 
>> Michael
>> 
>> 
>>> On Apr 26, 2018, at 12:24 AM, Soheil Pourbafrani <soheil.i...@gmail.com> 
>>> wrote:
>>> 
>>> As I said at first version of the code I didn't pass any argument to the 
>>> parse function and the HashMap was static among the Parser class, but it 
>>> didn't get the desired anwser and I test giving the HashMap as an argument 
>>> for parse method, but still not getting desired answers! The code is 
>>> something like the following:
>>> 
>>> public class Test {
>>> 
>>>     public static void main(String[] args) throws Exception {
>>> 
>>>         CassandraConnection.connect();
>>>         Parser.setInsert(true);
>>> 
>>>         stream.flatMap(new FlatMapFunction<byte[], Void>() {
>>>             @Override
>>>             public void flatMap(byte[] value, Collector<Void> out) throws 
>>> Exception {
>>>                 // Parser.parse(ByteBuffer.wrap(value));
>>>             }
>>>         });
>>>         env.execute();
>>>     }
>>> }
>>> 
>>> 
>>> 
>>> 
>>> I summary I want the HashMap to be shared among the taskmanagers.
>>> 
>>> 
>>> 
>> 
> 

Reply via email to