I'm really brand new with Scala, but if I'm defining a case class then is
becouse I know how is the json's structure is previously?
If I'm able to define dinamicaly a case class from the JSON structure then
even with spark I will be able to extract the data
On Wed, Jan 27, 2016 at 4:01 PM, Sahil Sareen <sareen...@gmail.com> wrote:
> Isn't this just about defining a case class and using
> parse(json).extract[CaseClassName] using Jackson?
>
> -Sahil
>
> On Wed, Jan 27, 2016 at 11:08 PM, Andrés Ivaldi <iaiva...@gmail.com>
> wrote:
>
>> We dont have Domain Objects, its a service like a pipeline, data is read
>> from source and they are saved it in relational Database
>>
>> I can read the structure from DataFrames, and do some transformations, I
>> would prefer to do it with Spark to be consistent with the process
>>
>>
>>
>> On Wed, Jan 27, 2016 at 12:25 PM, Al Pivonka <alpivo...@gmail.com> wrote:
>>
>>> Are you using an Relational Database?
>>> If so why not use a nojs DB ? then pull from it to your relational?
>>>
>>> Or utilize a library that understands Json structure like Jackson to
>>> obtain the data from the Json structure the persist the Domain Objects ?
>>>
>>> On Wed, Jan 27, 2016 at 9:45 AM, Andrés Ivaldi <iaiva...@gmail.com>
>>> wrote:
>>>
>>>> Sure,
>>>> The Job is like an etl, but without interface, so I decide the rules of
>>>> how the JSON will be saved into a SQL Table.
>>>>
>>>> I need to Flatten the hierarchies where is possible in case of list
>>>> flatten also, nested objects Won't be processed by now
>>>>
>>>> {"a":1,"b":[2,3],"c"="Field", "d":[4,5,6,7,8] }
>>>> {"a":11,"b":[22,33],"c"="Field1", "d":[44,55,66,77,88] }
>>>> {"a":111,"b":[222,333],"c"="Field2", "d":[44,55,666,777,888] }
>>>>
>>>> I would like something like this on my SQL table
>>>>
>>>> a b c d
>>>> 1 2,3 Field 4,5,6,7,8
>>>> 11 22,33 Field1 44,55,66,77,88
>>>> 111 222,333 Field2 444,555,,666,777,888
>>>>
>>>> Right now this is what i need
>>>>
>>>> I will later add more intelligence, like detection of list or nested
>>>> objects and create relations in other tables.
>>>>
>>>>
>>>>
>>>> On Wed, Jan 27, 2016 at 11:25 AM, Al Pivonka <alpivo...@gmail.com>
>>>> wrote:
>>>>
>>>>> More detail is needed.
>>>>> Can you provide some context to the use-case ?
>>>>>
>>>>> On Wed, Jan 27, 2016 at 8:33 AM, Andrés Ivaldi <iaiva...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello, I'm trying to Save a JSON filo into SQL table.
>>>>>>
>>>>>> If i try to do this directly the IlligalArgumentException is raised,
>>>>>> I suppose this is beacouse JSON have a hierarchical structure, is that
>>>>>> correct?
>>>>>>
>>>>>> If that is the problem, how can I flatten the JSON structure? The
>>>>>> JSON structure to be processed would be unknow, so I need to do it
>>>>>> programatically
>>>>>>
>>>>>> regards
>>>>>>
>>>>>> --
>>>>>> Ing. Ivaldi Andres
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Those who say it can't be done, are usually interrupted by those doing
>>>>> it.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ing. Ivaldi Andres
>>>>
>>>
>>>
>>>
>>> --
>>> Those who say it can't be done, are usually interrupted by those doing
>>> it.
>>>
>>
>>
>>
>> --
>> Ing. Ivaldi Andres
>>
>
>
--
Ing. Ivaldi Andres
