There is no record "345" here it seems, right? it's not that it exists and
has null fields; it's invalid w.r.t. the schema that the rest suggests.
On Wed, Feb 23, 2022 at 11:57 AM Sid <flinkbyhe...@gmail.com> wrote:
> Hello experts,
>
> I have a JSON data like below:
>
> [
> {
> "123": {
> "Party1": {
> "FIRSTNAMEBEN": "ABC",
> "ALIASBEN": "",
> "RELATIONSHIPTYPE": "ABC, FGHIJK LMN",
> "DATEOFBIRTH": "7/Oct/1969"
> },
> "Party2": {
> "FIRSTNAMEBEN": "ABCC",
> "ALIASBEN": "",
> "RELATIONSHIPTYPE": "ABC, FGHIJK LMN",
> "DATEOFBIRTH": "7/Oct/1969"
> }
> },
> "GeneratedTime": "2022-01-30 03:09:26"
> },
> {
> "456": {
> "Party1": {
> "FIRSTNAMEBEN": "ABCD",
> "ALIASBEN": "",
> "RELATIONSHIPTYPE": "ABC, FGHIJK LMN",
> "DATEOFBIRTH": "7/Oct/1969"
> },
> "Party2": {
> "FIRSTNAMEBEN": "ABCDD",
> "ALIASBEN": "",
> "RELATIONSHIPTYPE": "ABC, FGHIJK LMN",
> "DATEOFBIRTH": "7/Oct/1969"
> },
> "Party3": {
> "FIRSTNAMEBEN": "ABCDDE",
> "ALIASBEN": "",
> "RELATIONSHIPTYPE": "ABC, FGHIJK LMN",
> "DATEOFBIRTH": "7/Oct/1969"
> }
> },
> "GeneratedTime": "2022-01-30 03:09:26"
> },
> {
> "345": {
>
>
> },
> "GeneratedTime": "2022-01-30 03:09:26"
> }
> ]
>
> However, when I try to display this JSON using below code, it doesn't show
> the blank records. In my case I don't get any records for 345 since it is
> null but I want to display it in the final flattened dataset.
>
> val df = spark.read.option("multiline",
> true).json("/home/siddhesh/Documents/nested_json.json")
>
> Spark version:3.1.1
>
> Thanks,
> Sid
>
