Hello experts, I have a JSON data like below:
[ { "123": { "Party1": { "FIRSTNAMEBEN": "ABC", "ALIASBEN": "", "RELATIONSHIPTYPE": "ABC, FGHIJK LMN", "DATEOFBIRTH": "7/Oct/1969" }, "Party2": { "FIRSTNAMEBEN": "ABCC", "ALIASBEN": "", "RELATIONSHIPTYPE": "ABC, FGHIJK LMN", "DATEOFBIRTH": "7/Oct/1969" } }, "GeneratedTime": "2022-01-30 03:09:26" }, { "456": { "Party1": { "FIRSTNAMEBEN": "ABCD", "ALIASBEN": "", "RELATIONSHIPTYPE": "ABC, FGHIJK LMN", "DATEOFBIRTH": "7/Oct/1969" }, "Party2": { "FIRSTNAMEBEN": "ABCDD", "ALIASBEN": "", "RELATIONSHIPTYPE": "ABC, FGHIJK LMN", "DATEOFBIRTH": "7/Oct/1969" }, "Party3": { "FIRSTNAMEBEN": "ABCDDE", "ALIASBEN": "", "RELATIONSHIPTYPE": "ABC, FGHIJK LMN", "DATEOFBIRTH": "7/Oct/1969" } }, "GeneratedTime": "2022-01-30 03:09:26" }, { "345": { }, "GeneratedTime": "2022-01-30 03:09:26" } ] However, when I try to display this JSON using below code, it doesn't show the blank records. In my case I don't get any records for 345 since it is null but I want to display it in the final flattened dataset. val df = spark.read.option("multiline", true).json("/home/siddhesh/Documents/nested_json.json") Spark version:3.1.1 Thanks, Sid