Hi Prashant,
I have 2 concerns here
1. As I mentioned earlier, my file consists of more than 2 lac lines. So it is
not possible to load that many number of lines inside json.loads(). We need to
pass the file, which itself is a limitation as explained earlier.
2. It is not an ideal solution for me - creating a list of the values of
duplicate keys, because my values are not just single word strings. They are
again big dicts.
Thanks,
Lokesh
On Friday, 21 March 2014 3:35 PM, Prashant Gaur <91prashantg...@gmail.com>
wrote:
EDIT :
import simplejson as json
from collections import defaultdict
def duplicate_key_lookup(ordered_pairs):
"""Convert duplicate keys values to lists."""
# read all values into lists
d = defaultdict(list)
for k, v in ordered_pairs:
d[k].append(v)
# unpack lists that have only 1 item
for k, v in d.items():
if len(v) == 1:
d[k] = v[0]
return dict(d)
>>> json.loads('''{"test":"hi","test":"bye", "test1": "tata"}''',
>>> object_pairs_hook=duplicate_key_lookup)
{'test': ['hi', 'bye'], 'test1': 'tata'}
On Fri, Mar 21, 2014 at 3:34 PM, Prashant Gaur <91prashantg...@gmail.com> wrote:
Hello Lokesh ,
>
>
>as we know json.loads return data in form of dict and dict can never have same
>keys .
>
>
>so we can do one thing and that is to return a list of all values which are
>having same names.
>
>import simplejson as json
>from collections import defaultdict
>
>def duplicate_key_lookup(ordered_pairs):
> """Convert duplicate keys values to lists."""
> # read all values into lists
> d = defaultdict(list)
> for k, v in ordered_pairs:
> d[k].append(v)
> # unpack lists that have only 1 item
> for k, v in d.items():
> if len(v) == 1:
> d[k] = v[0]
> return dict(d)
>
>>>> json.loads('''{"test":"hi","test":"bye", "test1": "tata"}''',
>>>> object_pairs_hook=multidict)
>{'test': ['hi', 'bye'], 'test1': 'tata'}
>
>
>
>
>On Fri, Mar 21, 2014 at 3:22 PM, lokesh bobby <lokeshbob...@yahoo.com> wrote:
>
>Hi Jayanth/Prashant,
>>
>>
>>Either "the duplicates to be loaded" or "An ouput of all the duplicate key
>>names in a JSON file" should be fine for me :-)
>>
>>
>>NOTE: JSON file consits of more than 2 lakhs LOC
>>
>>
>>Lokesh
>>
>>
>>
>>On Friday, 21 March 2014 3:17 PM, Jayanth Koushik <jnkous...@gmail.com> wrote:
>>
>>Hi Prashant
>>
>>I think he wants the duplicates to be loaded, not cause an exception.
>>
>>Jayanth
>>
>>
>>
>>
>>On Fri, Mar 21, 2014 at 3:15 PM, Prashant Gaur <91prashantg...@gmail.com>
>>wrote:
>>
>>Hi Lokesh,
>>>
>>>
>>>we can pass lookup while parsing your json file which will make sure that
>>>name is repetitive or not .
>>>
>>>
>>>import json
>>>
>>>
>>>def duplicate_checking_hook(pairs):
>>>
>>> ''' lookup for duplicate names'''
>>>
>>> result = dict()
>>>
>>> for key, val in pairs:
>>>
>>> if key in result:
>>>
>>> raise KeyError('Duplicate Key specified: %s % key)
>>>
>>> result[key] = val
>>>
>>> return result
>>>
>>>>>> json.loads('''{"test":"hi","test":"bye"}''',
>>>>>> object_pairs_hook=duplicate_checking_hook)
>>>Traceback (most recent call last):
>>> File "<stdin>", line 1, in <module>
>>> File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
>>> return cls(encoding=encoding, **kw).decode(s)
>>> File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
>>> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>>> File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
>>> obj, end = self.scan_once(s, idx)
>>> File "<stdin>", line 5, in duplicate_checking_hook
>>>KeyError: u'Duplicate key specified: a'
>>>
>>>
>>>While
>>>
>>>>>> json.loads('''{"test":"hi","test1":"bye"}''',
>>>>>> object_pairs_hook=duplicate_checking_hook)
>>>{u'test': u'hi', u'test1': u'bye'}
>>>>>>
>>>
>>>
>>>
>>>
>>>
>>>defdupe_checking_hook(pairs):result =dict()forkey,val inpairs:ifkey
>>>inresult:raiseKeyError("Duplicate key specified: %s"%key)result[key]=val
>>>returnresult
>>>
>>>
>>>
>>>
>>>
>>>
>>>On Fri, Mar 21, 2014 at 2:59 PM, lokesh bobby <lokeshbob...@yahoo.com> wrote:
>>>
>>>Hi Jayanth,
>>>>
>>>>Ideally speaking a JSON shouldn't be created with repetitive key names. But
>>>>manually it is possible that a proper JSON file can be appended with a
>>>>duplicate key. We need to catch that duplicate key. If we are going to use
>>>>json.load(), the repetitive keys of the JSON file wont get loaded. Only the
>>>>last occurrence of the key will be there in that loaded data.
>>>>
>>>>Instead I want that entire data in JSON file should be loaded. Hoping that
>>>>you got the point what I want :-)
>>>>
>>>>Thanks,
>>>>Lokesh.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On Friday, 21 March 2014 2:45 PM, Jayanth Koushik <jnkous...@gmail.com>
>>>>wrote:
>>>>
>>>>Hi Lokesh
>>>>
>>>>
>>>>The 'problem' that you talk about isn't really a problem. Since the JSON
>>>>specification does not say what is to be done for repeated names, it is up
>>>>to the implementation to decide. What is your requirement for handling
>>>>repeated names?
>>>>
>>>>
>>>>Jayanth
>>>>
>>>>
>>>>
>>>>
>>>>On Fri, Mar 21, 2014 at 2:30 PM, lokesh bobby <lokeshbob...@yahoo.com>
>>>>wrote:
>>>>
>>>>Hi Noufal,
>>>>>
>>>>>Thanks for your reply. I am not looking for loading the JSON file. There
>>>>>is a limitation in it. Go thru the links
>>>>>
>>>>>http://docs.python.org/2/library/json.html#repeated-names-within-an-object
>>>>>http://docs.python.org/3.2/library/json.html#repeated-names-within-an-object
>>>>>
>>>>>In order to get rid of that problem, I am looking for some JSON stream
>>>>>parsers.
>>>>>
>>>>>Thanks,
>>>>>Lokesh.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>On Friday, 21 March 2014 2:09 PM, Noufal Ibrahim KV
>>>>><nou...@nibrahim.net.in> wrote:
>>>>>
>>>>>On Fri, Mar 21 2014, lokesh bobby wrote:
>>>>>
>>>>>
>>>>>> Hi ALL,
>>>>>>
>>>>>> Can you share your thoughts on how to parse a JSON file by using
>>>>>> python?
>>>>>
>>>>>import json
>>>>>
>>>>>with open("data.json") as f:
>>>>> json.load(f)
>>>>>
>>>>>
>>>>>[...]
>>>>>
>>>>>
>>>>>--
>>>>>Cordially,
>>>>>Noufal
>>>>>http://nibrahim.net.in
>>>>>_______________________________________________
>>>>>BangPypers mailing list
>>>>>BangPypers@python.org
>>>>>https://mail.python.org/mailman/listinfo/bangpypers
>>>>>
>>>>_______________________________________________
>>>>BangPypers mailing list
>>>>BangPypers@python.org
>>>>https://mail.python.org/mailman/listinfo/bangpypers
>>>>
>>>
>>>
>>>
>>>--
>>>
>>>Prashant Gaur
>>>
>>>Mobile : +91 9717353657
>>>http://gaurprashant.blogspot.in/
>>>http://stackoverflow.com/users/1850358/prashant-gaur
>>>http://www.about.me/prashantgaur/
>>>
>>>
>>
>>
>>
>
>
>--
>
>Prashant Gaur
>
>Mobile : +91 9717353657
>http://gaurprashant.blogspot.in/
>http://stackoverflow.com/users/1850358/prashant-gaur
>http://www.about.me/prashantgaur/
>
>
--
Prashant Gaur
Mobile : +91 9717353657
http://gaurprashant.blogspot.in/
http://stackoverflow.com/users/1850358/prashant-gaur
http://www.about.me/prashantgaur/
_______________________________________________
BangPypers mailing list
BangPypers@python.org
https://mail.python.org/mailman/listinfo/bangpypers
