You can use csv reader with delimiter as '|' to split data and create a
dataframe on top of the file data.
Second step, filter the dataframe on column value like indicator type=A,D
etc and put it in tables. Saving to tables you can use dataframewriter(not
sure what is you destination db type here).
Hi Swetha,
It would be great if you ask same question in stackoverflow , we have very
active community and monitor stack for each spark questions.
If you ask same question via stack other ppl also get benefits for similar
problems.
Regards,
Vaquar khan
On Sun, Sep 29, 2019, 10:26 PM swetha kadi
dear friends,
I am new to spark. can you please help me to read the below text file using
spark and scala.
Sample data
bret|lee|A|12345|ae545|gfddfg|86786786
142343345||D|ae342
67567|6|U|aadfsd|34k4|84304|020|sdnfsdfn|3243|benej|32432|jsfsdf|3423
67564|67747|U|aad434|3435|843454|203|sdn4
