No ‘from __future__ import print_function’ in latest jython
When I execute in the latest Jython: from __future__ import print_function I get: Traceback (innermost last): (no code object) at line 0 File "", line 1 SyntaxError: future feature print_function is not defined This is not implemented in the latest jython? I prefer to keep all my code as far as possible the same. The closer I get to DRY, the happier I am. ;-) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Jython and can't write cache file
On openSUSE there is a very old version of Jython installed (2.2.1), so I installed the latest version (2.7.0). But when starting this I get: *sys-package-mgr*: can't write cache file for '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar' *sys-package-mgr*: can't write cache file for '/home/cecil/java/jars/sqlite-jdbc.jar' Is that something to worry about? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Classic OOP in Python
Ah, turns out there was an entry. I updated it. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: How to construct matrix from vectors?
On 21/06/2015 04:47, Nasser M. Abbasi wrote: But it is not as intuitive as with Matlab For those of us who don't know would you be kind enough to do a cost comparison of Matlab vs Python licenses? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
How to check in script if Python or Jython is used
I installed Jython and will start playing with it. There probably will be differences between Python and Jython. Is there a way to determine if a script is run by Python or Jython? Then different execution paths could be taken. With sys.version(_info) you do not get this information. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Using Jython for Android development?
I have no experience yet with Jython or Android development. But I was wondering: would it be possible to write applications for Android with Jython? You normally use Java for it, but I think I would like Jython more. :-D -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: No ‘from __future__ import print_function’ in latest jython
On Sunday 21 Jun 2015 09:56 CEST, Cecil Westerhof wrote: > When I execute in the latest Jython: > from __future__ import print_function > I get: > Traceback (innermost last): > (no code object) at line 0 > File "", line 1 > SyntaxError: future feature print_function is not defined > > This is not implemented in the latest jython? I prefer to keep all > my code as far as possible the same. The closer I get to DRY, the > happier I am. ;-) My bad, I started the old version instead of the latest when I tested this. Sorry for the noise. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: How to construct matrix from vectors?
On 06/21/2015 07:21 AM, Nasser M. Abbasi wrote: v1=np.array([(1,2,3)]).T v2=np.array([(4,5,6)]).T v3=np.array([(7,8,9)]).T v4=np.array([(10,11,12)]).T mat =np.hstack(( np.vstack((v1,v3)), np.vstack((v2,v4))) ) Out[236]: array([[ 1, 4], [ 2, 5], [ 3, 6], [ 7, 10], [ 8, 11], [ 9, 12]]) There are way too many '(([[]]))' things in Python :) another solution with less "(([[]]))", and less ";". There are way too many ";" in Matlab ;) import numpy as np v1 = [1, 2, 3] v2 = [4, 5, 6] v3 = [7, 8, 9] v4 = [10, 11, 12] np.hstack([[v1, v2], [v3, v4]]).T Out[]: array([[ 1, 4], [ 2, 5], [ 3, 6], [ 7, 10], [ 8, 11], [ 9, 12]]) -- https://mail.python.org/mailman/listinfo/python-list
Re: Jython and can't write cache file
In a message of Sun, 21 Jun 2015 09:53:23 +0200, Cecil Westerhof writes: >On openSUSE there is a very old version of Jython installed (2.2.1), >so I installed the latest version (2.7.0). But when starting this I >get: >*sys-package-mgr*: can't write cache file for > '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar' >*sys-package-mgr*: can't write cache file for > '/home/cecil/java/jars/sqlite-jdbc.jar' > >Is that something to worry about? > >-- >Cecil Westerhof >Senior Software Engineer >LinkedIn: http://www.linkedin.com/in/cecilwesterhof >-- >https://mail.python.org/mailman/listinfo/python-list Yes. Read this: http://www.prasannatech.net/2009/02/jython-sys-package-mgr-processing-jar.html Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: No ‘from __future__ import print_function’ in latest jython
Do you have Jython 2.7 released a few weeks ago? Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Lawful != Mutable (was Can Python function return multiple data?)
In a message of Sat, 20 Jun 2015 19:50:21 -0700, Rustom Mody writes: >Here is Eric Snow: > >| Keep in mind that by "immutability" I'm talking about *really* >| immutable, perhaps going so far as treating the full memory space >| associated with an object as frozen. For instance, we'd have to >| ensure that "immutable" Python objects like strings, ints, and tuples >| do not change (i.e. via the C API). The contents of involved >| tuples/containers would have to be likewise immutable. Even changing >| refcounts could be too much, hence the idea of moving refcounts out to >| a separate table. >| >| This level of immutability would be something new to Python. We'll >| see if it's necessary. If it isn't too much work it might be a good >| idea regardless of the multi-core proposal. > >Does the second para look like CPython implementation or python-the-language? Regardless of how it looks, it's only about the CPython implementation. PyPy and Jython will not have this problem, as they don't have a C API to worry about. Of course the PyPy position is that STM will solve all of this in a much cleaner way, and we're getting closer to a perfectly working solution for as many cores as you have (as opposed to only 4) that works on all OS's (as opposed to just 64 bit linux) all the time. This is very much a 'how to hack CPython to support multiple cores' idea. It's a 'kill the GIL' proposal. But PyPy-STM and Jython don't have a GIL, so they don't need to kill it. >Also note the 'Even' in the first para. ie Eric is talking of low-level >(ie thread-safety, refcounting etc) immutability after the even and higher >level semantic immutability before The ref-counting should be a very strong hint that this is a CPython implementation proposal. Again, Jython and PyPy don't use a ref counting GC. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: How to check in script if Python or Jython is used
In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof writes: >I installed Jython and will start playing with it. There probably will >be differences between Python and Jython. Is there a way to determine >if a script is run by Python or Jython? Then different execution paths >could be taken. With sys.version(_info) you do not get this >information. > >-- >Cecil Westerhof >Senior Software Engineer >LinkedIn: http://www.linkedin.com/in/cecilwesterhof >-- >https://mail.python.org/mailman/listinfo/python-list import platform platform.python_implementation() If your jython is old (pre 2.6) you will not have this. Then try platform.system() which will give you 'Java' Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Using Jython for Android development?
In a message of Sun, 21 Jun 2015 10:14:15 +0200, Cecil Westerhof writes: >I have no experience yet with Jython or Android development. But I was >wondering: would it be possible to write applications for Android with >Jython? You normally use Java for it, but I think I would like Jython >more. :-D > >-- >Cecil Westerhof >Senior Software Engineer >LinkedIn: http://www.linkedin.com/in/cecilwesterhof You cannot use Jython for this at present. PyPy for android coming. (Works great on ARM, but oh, the environment changes to get the libtraries to work. ) In the meatime, use kivy. Actually, use kivy anyway, it is just too much fun. :) http://kivy.org/#home Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: HOPE: A Python just-in-time compiler for astrophysical computations
On 21/06/2015 01:29, Mark Lawrence wrote: Another beasty I've just stumbled across which you may find interesting http://www.sciencedirect.com/science/article/pii/S2213133714000687 Blimey, that's a lot of waffle in there, but I suppose that's to be expected from a published paper. I think the gist of it is, that you highlight specific Python functions that you need to be fast (add a decorator), then it tries to translate those into actual C++ by inferring types. All done transparently at runtime (although I imagine it would be hard to hide the huge machinery of a C++ compiler in action). The benchmarks seem to be individual functions which it presumably successfully translated fully into C++, so it is effectively comparing CPython to C++. It also puts in a good dig at PyPy by including one benchmark where it is 6 times as slow as CPython! It's not clear why it's particularly useful for astrophysics. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Jython and can't write cache file
On Sunday 21 Jun 2015 10:56 CEST, Laura Creighton wrote: > In a message of Sun, 21 Jun 2015 09:53:23 +0200, Cecil Westerhof > writes: >> On openSUSE there is a very old version of Jython installed >> (2.2.1), so I installed the latest version (2.7.0). But when >> starting this I get: *sys-package-mgr*: can't write cache file for >> '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar' *sys-package-mgr*: >> can't write cache file for '/home/cecil/java/jars/sqlite-jdbc.jar' >> >> Is that something to worry about? >> >> -- >> Cecil Westerhof >> Senior Software Engineer >> LinkedIn: http://www.linkedin.com/in/cecilwesterhof >> -- >> https://mail.python.org/mailman/listinfo/python-list > > Yes. > > Read this: > http://www.prasannatech.net/2009/02/jython-sys-package-mgr-processing-jar.html Thanks. Good that I asked it. :-D -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Using Jython for Android development?
On Sunday 21 Jun 2015 11:26 CEST, Laura Creighton wrote: > In a message of Sun, 21 Jun 2015 10:14:15 +0200, Cecil Westerhof > writes: >> I have no experience yet with Jython or Android development. But I >> was wondering: would it be possible to write applications for >> Android with Jython? You normally use Java for it, but I think I >> would like Jython more. :-D >> >> -- >> Cecil Westerhof >> Senior Software Engineer >> LinkedIn: http://www.linkedin.com/in/cecilwesterhof > > You cannot use Jython for this at present. PyPy for android coming. > (Works great on ARM, but oh, the environment changes to get the > libtraries to work. ) > > In the meatime, use kivy. Actually, use kivy anyway, it is just too > much fun. :) > > http://kivy.org/#home Something else to evaluate. I do not have to be afraid I will become bored. :-D -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: No ‘from __future__ import print_function’ in latest jython
On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote: > Do you have Jython 2.7 released a few weeks ago? Yes, but I was dumb enough to start the old version when I did this. :-( There is still one problem: == >>> from __future__ import print_function >>> print() () == While it should be: == >>> from __future__ import print_function >>> print() == But I think I can live with it for the moment. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: How to check in script if Python or Jython is used
On Sunday 21 Jun 2015 11:22 CEST, Laura Creighton wrote: > In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof > writes: >> I installed Jython and will start playing with it. There probably >> will be differences between Python and Jython. Is there a way to >> determine if a script is run by Python or Jython? Then different >> execution paths could be taken. With sys.version(_info) you do not >> get this information. >> >> -- >> Cecil Westerhof >> Senior Software Engineer >> LinkedIn: http://www.linkedin.com/in/cecilwesterhof >> -- >> https://mail.python.org/mailman/listinfo/python-list > > import platform > platform.python_implementation() > > If your jython is old (pre 2.6) you will not have this. > > Then try > platform.system() > which will give you 'Java' Works like a charm. CPython gives (on my system) 'Linux'. But as long I only work with CPython and Jython, I only have to check for 'Java'. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: No ‘from __future__ import print_function’ in latest jython
In a message of Sun, 21 Jun 2015 12:21:03 +0200, Cecil Westerhof writes: >On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote: > >> Do you have Jython 2.7 released a few weeks ago? > >Yes, but I was dumb enough to start the old version when I did this. >:-( > >There is still one problem: >== from __future__ import print_function print() >() >== > >While it should be: >== from __future__ import print_function print() > >== > >But I think I can live with it for the moment. > >-- >Cecil Westerhof >Senior Software Engineer >LinkedIn: http://www.linkedin.com/in/cecilwesterhof >-- >https://mail.python.org/mailman/listinfo/python-list That's a bug. Go add it to http://bugs.jython.org/issue2007 so when the bugs with print_function get fixed, they will be sure to fix that one too. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: How to construct matrix from vectors?
Fabien wrote: >another solution with less "(([[]]))", and less ";". There are way too >many ";" in Matlab ;) > >import numpy as np >v1 = [1, 2, 3] >v2 = [4, 5, 6] >v3 = [7, 8, 9] >v4 = [10, 11, 12] >np.hstack([[v1, v2], [v3, v4]]).T >Out[]: >array([[ 1, 4], >[ 2, 5], >[ 3, 6], >[ 7, 10], >[ 8, 11], >[ 9, 12]]) Neat. And if the OP wants "vectors" in np array form to start with, and to stack them together without transposing at that point, he could do it like this: >>> v1=np.vstack([1,2,3]) >>> v2=np.vstack([4,5,6]) >>> v3=np.vstack([7,8,9]) >>> v4=np.vstack([10,11,12]) >>> np.r_[np.c_[v1,v2],np.c_[v3,v4]] array([[ 1, 4], [ 2, 5], [ 3, 6], [ 7, 10], [ 8, 11], [ 9, 12]]) And since he seems to want a Matlab-like environment, then the somewhat depreciated pylab was intended to dump a Matlab-like set of functions into the namespace, which is OK for an interactive environment, an not too much of a problem for a short program in a single module. Probably best to do that with iPython, though. >>> from matplotlib.pylab import * >>> v1=vstack([1,2,3]) >>> v2=vstack([4,5,6]) >>> v3=vstack([7,8,9]) >>> v4=vstack([10,11,12]) >>> r_[c_[v1,v2],c_[v3,v4]] array([[ 1, 4], [ 2, 5], [ 3, 6], [ 7, 10], [ 8, 11], [ 9, 12]]) -- https://mail.python.org/mailman/listinfo/python-list
Re: How to check in script if Python or Jython is used
Sun, Jun 21, 2015 12:24 PM CEST Cecil Westerhof wrote: >On Sunday 21 Jun 2015 11:22 CEST, Laura Creighton wrote: > >> In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof >> writes: >> I installed Jython and will start playing with it. There probably >> will be differences between Python and Jython. Is there a way to >> determine if a script is run by Python or Jython? Then different >> execution paths could be taken. With sys.version(_info) you do not >> get this information. >> os.path.basename(sys.executable), if sys.implementation is not an option -- https://mail.python.org/mailman/listinfo/python-list
Re: Jython and can't write cache file
On 21.06.2015 11:40, Cecil Westerhof wrote: > Thanks. Good that I asked it. :-D Good for you that you found someone able to enter words into a Google query. That's a skill you might want to acquire some time in the future. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht öffentlich! Ah, der neueste und bis heute genialste Streich unsere großen Kosmologen: Die Geheim-Vorhersage. - Karl Kaos über Rüdiger Thomas in dsa -- https://mail.python.org/mailman/listinfo/python-list
Re: No ‘from __future__ import print_function’ in latest jython
On Sunday 21 Jun 2015 12:54 CEST, Laura Creighton wrote: > In a message of Sun, 21 Jun 2015 12:21:03 +0200, Cecil Westerhof > writes: >> On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote: >> >>> Do you have Jython 2.7 released a few weeks ago? >> >> Yes, but I was dumb enough to start the old version when I did >> this. :-( >> >> There is still one problem: >> == > from __future__ import print_function > print() >> () >> == >> >> While it should be: >> == > from __future__ import print_function > print() >> >> == >> >> But I think I can live with it for the moment. >> >> -- >> Cecil Westerhof >> Senior Software Engineer >> LinkedIn: http://www.linkedin.com/in/cecilwesterhof >> -- >> https://mail.python.org/mailman/listinfo/python-list > > That's a bug. Go add it to > http://bugs.jython.org/issue2007 so when the bugs with > print_function get fixed, they will be sure to fix that one too. Done. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list
Re: Lawful != Mutable (was Can Python function return multiple data?)
On 06/20/2015 10:50 PM, Rustom Mody wrote: Here is Eric Snow: | Keep in mind that by "immutability" I'm talking about*really* | immutable, perhaps going so far as treating the full memory space | associated with an object as frozen. For instance, we'd have to | ensure that "immutable" Python objects like strings, ints, and tuples | do not change (i.e. via the C API). The contents of involved | tuples/containers would have to be likewise immutable. Even changing | refcounts could be too much, hence the idea of moving refcounts out to | a separate table. | | This level of immutability would be something new to Python. We'll | see if it's necessary. If it isn't too much work it might be a good | idea regardless of the multi-core proposal. Does the second para look like CPython implementation or python-the-language? Also note the 'Even' in the first para. ie Eric is talking of low-level (ie thread-safety, refcounting etc) immutability after the even and higher level semantic immutability before It seems to me, this will take a lot more changes to python overall. Adding a way to raise an exception if an object is mutated would be a good initial step. Have it turned off by default. I think it will be needed to test how any of the above is working. It may also allow some multiprocessing just by avoiding raising any MutatedObject exceptions. Cheers, Ron -- https://mail.python.org/mailman/listinfo/python-list
Re: HOPE: A Python just-in-time compiler for astrophysical computations
In a message of Sun, 21 Jun 2015 10:29:32 +0100, BartC writes: >It also puts in a good dig at PyPy by including one benchmark where it >is 6 times as slow as CPython! > >It's not clear why it's particularly useful for astrophysics. > >-- >Bartc It's not that good a dig, as they say that it took less than 1 second to run most of their benchmarks, which mostly means that the PyPy JIT won't have started to run anyway. One can argue that HOPE isn't actually a method JIT at all -- or if it is one, then so is Cython. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
>
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
>
> > {
> > "CF": {
> ...
> > "CF": "Fee",
>
> Your json object seems to have the same key used for two elements at the
> same level, are you sure this is legal json?
>
> --
> Denis McMahon, denismfmcma...@gmail.com
I converted this from an XML file given to me from a third party. It is as is.
I am not sure what you mean by "valid"; that is a very subjective measure for
any form of quantitative or qualitative data.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On Sunday, 21 June 2015 02:54:48 UTC-4, Denis McMahon wrote: > On Thu, 18 Jun 2015 18:47:30 -0700, Sahlusar wrote: > > > I have a conundrum regarding JSON objects and converting them to CSV: > > I think your conundrum is that you've taken on a coding task beyond your > abilities to comprehend, and as a result not only can you not code it, > you can't even adequately describe it. > > At least, it seems that every time you do try and describe it either the > data format or the task description changes. > > -- > Denis McMahon, denismfmcma...@gmail.com It is difficult to explain this to someone asynchronously and without an in person meeting. Moreover, the strict guidelines for disclosing information make it difficult for me to explain the client's requirements and the problems that they face. I do agree with you Denis that this is an unconventional approach. -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Sunday, June 21, 2015 at 2:47:31 AM UTC-4, Denis McMahon wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
>
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
>
> > {
> > "CF": {
> ...
> > "CF": "Fee",
>
> Your json object seems to have the same key used for two elements at the
> same level, are you sure this is legal json?
>
> --
> Denis McMahon, denismfmcma...@gmail.com
I converted this from an XML file given to me from a third party. It is as is.
I am not sure what you mean by "valid"; that is a very subjective measure for
any form of quantitative or qualitative data.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
It is difficult to explain this to someone asynchronously and without an in
person meeting. Moreover, the strict guidelines for disclosing information make
it difficult for me to explain the client's requirements and the problems that
they face.
I do agree with you Denis that this is an unconventional approach. I was
wondering then that perhaps I should add additional functionality at the XML to
JSON step? So far, with JSON objects without nested lists (as values) I have
been successful with this (the following is rather lengthy):
import xml.etree.cElementTree as ElementTree
from xml.etree.ElementTree import XMLParser
import json
import csv
import tokenize
import token
try:
from collections import OrderedDict
import json
except ImportError:
from ordereddict import OrderedDict
import simplejson as json
import itertools
import six
import string
from csvkit import CSVKitWriter
class XmlListConfig(list):
def __init__(self, aList):
for element in aList:
if element:
# treat like dict
if len(element) == 1 or element[0].tag != element[1].tag:
self.append(XmlDictConfig(element))
# treat like list
elif element[0].tag == element[1].tag:
self.append(XmlListConfig(element))
elif element.text:
text = element.text.strip()
if text:
self.append(text)
class XmlDictConfig(dict):
'''
Example usage:
>>> tree = ElementTree.parse('your_file.xml')
>>> root = tree.getroot()
>>> xmldict = XmlDictConfig(root)
Or, if you want to use an XML string:
>>> root = ElementTree.XML(xml_string)
>>> xmldict = XmlDictConfig(root)
And then use xmldict for what it is..a dictionary.
'''
def __init__(self, parent_element):
if parent_element.items():
self.update(dict(parent_element.items()))
for element in parent_element:
if element:
# treat like dict - we assume that if the first two tags
# in a series are different, then they are all different.
if len(element) == 1 or element[0].tag != element[1].tag:
aDict = XmlDictConfig(element)
# treat like list - we assume that if the first two tags
# in a series are the same, then the rest are the same.
else:
# here, we put the list in dictionary; the key is the
# tag name the list elements all share in common, and
# the value is the list itself
aDict = {element[0].tag: XmlListConfig(element)}
# if the tag has attributes, add those to the dict
if element.items():
aDict.update(dict(element.items()))
self.update({element.tag: aDict})
# this assumes that if you've got an attribute in a tag,
# you won't be having any text. This may or may not be a
# good idea -- time will tell. It works for the way we are
# currently doing XML configuration files...
elif element.items():
self.update({element.tag: dict(element.items())})
# finally, if there are no child tags and no attributes, extract
# the text
else:
self.update({element.tag: element.text})
def main():
#Lines 88-89stantiate the class Elementree
#and applies the method to recursively traverse from the root node
#XmlDictConfig is instantiated in line 90
with open('C:\\Users\\somefile.xml', 'r', encoding='utf-8') as f:
xml_string = f.read()
xml_string= xml_string.replace('�', '') #optional to remove ampersands.
root = ElementTree.XML(xml_string)
xmldict = XmlDictConfig(root)
json_str = json.dumps(xmldict, sort_keys=True, indent=4, separators=(',',
': '))
newly_formatted_data = json.loads(json_str) #encode into JSON
with open('data2.json', 'w') as f: #writing JSON file
json.dump(newly_formatted_data, f)
Peter Otten was very helpful with subsequently converting aJ SON string to a
CSV file:
import csv
import json
import sys
def hook(obj):
return obj
def flatten(obj):
for k, v in obj:
if isinstance(v, list):
yield from flatten(v)
else:
yield k, v
if __name__ == "__main__":
with open("somefileneame.json") as f:
data = json.load(f, object_pairs_hook=hook)
pairs = list(flatten(data))
writer = csv.writer(sys.stdout)
header = writer.writerow([k for k, v in pairs])
row = writer.writerow([v for k, v in pairs]) #writer.writerows for any
other iterable object
However, for the nested keys as dictionaries with values as dictionaries, I
have be
Re: JSON Object to CSV file
On Sunday, June 21, 2015 at 2:47:31 AM UTC-4, Denis McMahon wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
>
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
>
> > {
> > "CF": {
> ...
> > "CF": "Fee",
>
> Your json object seems to have the same key used for two elements at the
> same level, are you sure this is legal json?
>
> --
> Denis McMahon, denismfmcma...@gmail.com
This is a duplicate to the post titled: JSON to CSV Troubleshooting:
It is difficult to explain this to someone asynchronously and without an in
person meeting. Moreover, the strict guidelines for disclosing information make
it difficult for me to explain the client's requirements and the problems that
they face.
I do agree with you Denis that this is an unconventional approach. I was
wondering then that perhaps I should add additional functionality at the XML to
JSON step? So far, with JSON objects without nested lists (as values) I have
been successful with this (the following is rather lengthy):
import xml.etree.cElementTree as ElementTree
from xml.etree.ElementTree import XMLParser
import json
import csv
import tokenize
import token
try:
from collections import OrderedDict
import json
except ImportError:
from ordereddict import OrderedDict
import simplejson as json
import itertools
import six
import string
from csvkit import CSVKitWriter
class XmlListConfig(list):
def __init__(self, aList):
for element in aList:
if element:
# treat like dict
if len(element) == 1 or element[0].tag != element[1].tag:
self.append(XmlDictConfig(element))
# treat like list
elif element[0].tag == element[1].tag:
self.append(XmlListConfig(element))
elif element.text:
text = element.text.strip()
if text:
self.append(text)
class XmlDictConfig(dict):
'''
Example usage:
>>> tree = ElementTree.parse('your_file.xml')
>>> root = tree.getroot()
>>> xmldict = XmlDictConfig(root)
Or, if you want to use an XML string:
>>> root = ElementTree.XML(xml_string)
>>> xmldict = XmlDictConfig(root)
And then use xmldict for what it is..a dictionary.
'''
def __init__(self, parent_element):
if parent_element.items():
self.update(dict(parent_element.items()))
for element in parent_element:
if element:
# treat like dict - we assume that if the first two tags
# in a series are different, then they are all different.
if len(element) == 1 or element[0].tag != element[1].tag:
aDict = XmlDictConfig(element)
# treat like list - we assume that if the first two tags
# in a series are the same, then the rest are the same.
else:
# here, we put the list in dictionary; the key is the
# tag name the list elements all share in common, and
# the value is the list itself
aDict = {element[0].tag: XmlListConfig(element)}
# if the tag has attributes, add those to the dict
if element.items():
aDict.update(dict(element.items()))
self.update({element.tag: aDict})
# this assumes that if you've got an attribute in a tag,
# you won't be having any text. This may or may not be a
# good idea -- time will tell. It works for the way we are
# currently doing XML configuration files...
elif element.items():
self.update({element.tag: dict(element.items())})
# finally, if there are no child tags and no attributes, extract
# the text
else:
self.update({element.tag: element.text})
def main():
#Lines 88-89stantiate the class Elementree
#and applies the method to recursively traverse from the root node
#XmlDictConfig is instantiated in line 90
with open('C:\\Users\\somefile.xml', 'r', encoding='utf-8') as f:
xml_string = f.read()
xml_string= xml_string.replace('�', '') #optional to remove ampersands.
root = ElementTree.XML(xml_string)
xmldict = XmlDictConfig(root)
json_str = json.dumps(xmldict, sort_keys=True, indent=4, separators=(',',
': '))
newly_formatted_data = json.loads(json_str) #encode into JSON
with open('data2.json', 'w') as f: #writing JSON file
json.dump(newly_formatted_data, f)
Peter Otten was very helpful with subsequently converting aJ SON string to a
CSV file:
import csv
import json
i
Re: JSON Object to CSV file
On Sunday, June 21, 2015 at 10:54:44 AM UTC-4, Sahlusar wrote: > This is a duplicate to the post titled: JSON to CSV Troubleshooting: Please don't do that. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Sunday, 21 June 2015 11:31:44 UTC-4, Ned Batchelder wrote: > On Sunday, June 21, 2015 at 10:54:44 AM UTC-4, Sahlusar wrote: > > > This is a duplicate to the post titled: JSON to CSV Troubleshooting: > > Please don't do that. > > --Ned. My apologies - is it possible to delete one of my posts? -- https://mail.python.org/mailman/listinfo/python-list
Re: How to construct matrix from vectors?
On Sunday, 21 June 2015 20:43:15 UTC+5:30, Dennis Lee Bieber wrote: > On Sun, 21 Jun 2015 09:32:55 +0100, Mark Lawrence declaimed the following: > > >On 21/06/2015 04:47, Nasser M. Abbasi wrote: > > > >> > >> But it is not as intuitive as with Matlab > >> > > > >For those of us who don't know would you be kind enough to do a cost > >comparison of Matlab vs Python licenses? > > Or even Matlab vs Octave Well if its a parentheses minimization contest, APL will beat the pants of everything. Heres a web-session at http://baruchel.hd.free.fr/apps/apl/ v1 ← 1 2 3 v2 ← 4 5 6 v3 ← 7 8 9 v4 ← 10 11 12 v1 1 2 3 # We need to reshape (⍴) v1 ← 3 1⍴ v1 v1 1 2 3 # Likewise v2 ← 3 1⍴ v2 v3 ← 3 1⍴ v3 v4 ← 3 1⍴ v4 v1,v2 1 4 2 5 3 6 # So we need to catenate in a different dimension v1⍪v2 1 2 3 4 5 6 #Likewise v3⍪v4 7 8 9 10 11 12 # Thats the only parentheses used (v1⍪v2),(v3⍪v4) 1 7 2 8 3 9 4 10 5 11 6 12 # And we dont even need all these (v1⍪v2),v3⍪v4 1 7 2 8 3 9 4 10 5 11 6 12 -- https://mail.python.org/mailman/listinfo/python-list
Re: How to check in script if Python or Jython is used
Cecil Westerhof wrote: > I installed Jython and will start playing with it. There probably will > be differences between Python and Jython. Is there a way to determine > if a script is run by Python or Jython? Then different execution paths > could be taken. With sys.version(_info) you do not get this > information. “print sys.__doc__” in (C)python(2) hinted at “sys.platform”. And so: $ python -V Python 2.7.10 $ python -c 'from sys import platform; print platform' linux2 $ python3 -V Python 3.4.3+ $ python3 -c 'from sys import platform; print(platform)' linux $ jython -V "my" variable $jythonHome masks earlier declaration in same scope at /usr/bin/jython line 15. Jython 2.5.3 $ jython -c 'from sys import platform; print platform' "my" variable $jythonHome masks earlier declaration in same scope at /usr/bin/jython line 15. java1.7.0_79 -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. -- https://mail.python.org/mailman/listinfo/python-list
Python functionality in Javascript
Check this out using a 8 digit base with a 100 digit number no problem. http://jt.node365.se/baseconversion3.html -- https://mail.python.org/mailman/listinfo/python-list
Do I need license to release the Python version of old BASIC games?
Greetings, I'm in the process of converting 101 old BASIC games into Python (see link below). http://www.atariarchives.org/basicgames/ The short term goal is to learn the finer aspects of the Python language and reliving my misbegotten past on the Commodore 64. The long term goal is to use the completed project as part of a programming portfolio, as many of my past I.T. jobs have required no programming and my current job requires some PowerShell scripting (meh). Many of these BASIC games floated around the university computer labs and DEC for years before being published in Creative Computing magazine and into book form in 1978. My Python scripts are faithful to the video output as listed in the book, but underlying code is vastly different than the BASIC code. Do I need to release my scripts under a license? If so, which one? Thanks, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes: >Do I need to release my scripts under a license? If so, which one? You should, because if you don't you could pop up some day and assert copyright and sue the hell out of people who use your code, which means that many people won't touch it until you license it. http://blog.codinghorror.com/pick-a-license-any-license/ is a pretty good introduction to the quagmire, and talks about some popular choices. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Catching exceptions with multi-processing
Fabien writes: > I am developing a tool which works on individual entities (glaciers) > and do a lot of operations on them. There are many tasks to do, one > after each other, and each task follows the same interface: ... If most of the resources will be spent on computation and the communications overhead is fairly low, the path of least resistance may be to: 1) write a script that computes just one glacier (no multiprocessing) 2) write a control script that runs the glacier script through something like os.popen(), so normally it will collect an answer, but it can also notice if the glacier script crashes, or kill it from a timeout if it takes too long 3) Track the glacier tasks in an external queue server: I've used Redis (redis.io) for this, since it's simple and powerful, but there are other tools like 0mq that might be more precisely fitted. 4) The control script can read the queue server for tasks and update the queue server when results are ready The advantages of this over multiprocessing are: 1) Redis is a TCP server which means you can spread your compute scripts over multiple computers easily, getting more parallelism. You can write values into it as JSON strings if they are compound values that are not too large. Otherwise you probably have to use files, but can pass the filenames through Redis. You can connect new clients whenever you want through the publish/subscribe interface, etc. 2) by using a simple control script you don't have to worry too much about the many ways that the computation script might fail, you can restart it, you can put the whole thing under your favorite supervision daemon (cron, upstart, systemd or whatever) so it can restart automatically even if your whole computer reboots, etc. Redis can even mirror itself to a failover server in real time if you think you need that, plus it can checkpoint its state to disk. -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 6/21/2015 1:00 PM, Laura Creighton wrote: In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes: Do I need to release my scripts under a license? If so, which one? You should, because if you don't you could pop up some day and assert copyright and sue the hell out of people who use your code, which means that many people won't touch it until you license it. I want to strike a right balance between respecting the 1987 copyright of the book, which much of the code was either in the public domain or submitted to Creative Computing magazine, and protecting my own code that uses the video output from the book. I'm leaning towards the MIT license as many of games were developed in university computer labs and freely shared among computer users. For a copyright blast from the past, consider Bill Gate's open letter to hobbyists stealing Microsoft Basic in 1976. https://en.wikipedia.org/wiki/Open_Letter_to_Hobbyists Thanks, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Sun, 21 Jun 2015 06:57:01 -0700, sahluwalia wrote:
> On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon wrote:
>> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
>>
>> > I would like to have this JSON object written out to a CSV file so
>> > that the keys are header fields (for each of the columns) and the
>> > values are values that are associated with each header field.
>>
>> > {
>> > "CF": {
>> ...
>> > "CF": "Fee",
>>
>> Your json object seems to have the same key used for two elements at
>> the same level, are you sure this is legal json?
> I converted this from an XML file given to me from a third party. It is
> as is. I am not sure what you mean by "valid"; that is a very subjective
> measure for any form of quantitative or qualitative data.
Put it this way, when I feed your json object into a jason lint, the
output is the following:
{
"CF": "Fee",
"ID": "2"
}
The second occurrence in the parent object of name "CF" with a value of
string literal "Fee" overwrites the earlier name "CF" whose value is (in
python terms) a dictionary or (in json terms) an object.
--
Denis McMahon, denismfmcma...@gmail.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
Laura Creighton : > In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes: > >>Do I need to release my scripts under a license? If so, which one? > > You should, because if you don't you could pop up some day and assert > copyright and sue the hell out of people who use your code, which > means that many people won't touch it until you license it. Converting BASIC games to Python results in derived works, which are under the original copyright of the BASIC games. From the given link: BASIC Computer Games is copyright © 1978 by David H. Ahl, and is posted on www.atariarchives.org with permission. Do not redistribute, mirror, or copy this online book. So a license from David H. Ahl is required before publishing Python translations. As for licensing one's own code, you can grant a license (or several), or you could place your code in the public domain. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On Sun, 21 Jun 2015 07:38:13 -0700, Sahlusar wrote: > It is difficult to explain this to someone asynchronously and without an > in person meeting. Moreover, the strict guidelines for disclosing > information make it difficult for me to explain the client's > requirements and the problems that they face. > > I do agree with you Denis that this is an unconventional approach. I was > wondering then that perhaps I should add additional functionality at the > XML to JSON step? So far, with JSON objects without nested lists (as > values) I have been successful with this (the following is rather > lengthy): No, step back and force yourself to answer these questions: Why use JSON as an intermediate step? What benefit does using JSON as an intermediate step bring me? I see no evidence in any of your posts that the use of JSON as an intermediate format for the data brings any benefit whatsoever, however I have seen evidence that it may be introducing errors and potential data loss, and it is certainly adding coding complexity. None of these are good reasons to do it, and all of them are good reasons not to do it. If your data is in XML and your requirement is for CSV, then you should be converting from XML to CSV. Also stop posting reams of code. No-one is reading it. If you have a specific error you need to fix, then post a shortest possible example of code that generates the error. This should never be more than about 10 lines. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 06/21/2015 02:58 PM, Marko Rauhamaa wrote: > Laura Creighton : > >> In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes: >> >>> Do I need to release my scripts under a license? If so, which one? >> >> You should, because if you don't you could pop up some day and assert >> copyright and sue the hell out of people who use your code, which >> means that many people won't touch it until you license it. > > Converting BASIC games to Python results in derived works, which are > under the original copyright of the BASIC games. > > From the given link: > >BASIC Computer Games is copyright © 1978 by David H. Ahl, and is >posted on www.atariarchives.org with permission. Do not redistribute, >mirror, or copy this online book. > > So a license from David H. Ahl is required before publishing Python > translations. > > As for licensing one's own code, you can grant a license (or several), > or you could place your code in the public domain. I disagree. Especially where the resulting python program is not a transliteration (which it certainly won't be) of the original BASIC programs. As well, these programs implement well-known algorithms and games. There are only so many ways to implement certain algorithms, and each implementation is going to be closely similar. The games and algorithms in his BASIC programs are by no means original or not well-known and well-discussed. Also the quoted copyright notice is for the entire work, which is to say the book. Chris is in no way reproducing the guy's text in whole or in part--I note that the notice says nothing about fair use. That's not to say the original book author couldn't pursue a copyright claim, even if it's without merit. As to the question of assigning a copyright license to code, in this case I suggest just releasing the code marked as public domain. The programs are small and trivial. If you do not release it explicitly as public domain than you must release it under *some* license, because to not do so prohibits anyone from doing *anything* with your code by default, something a lot of githubbers seem to forget. Also as long as the code is entirely your work (and it is, despite Marko's assertions about derivative works), you can change the license at any time. -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 6/21/2015 1:58 PM, Marko Rauhamaa wrote:
Converting BASIC games to Python results in derived works, which are
under the original copyright of the BASIC games.
From the given link:
BASIC Computer Games is copyright © 1978 by David H. Ahl, and is
posted on www.atariarchives.org with permission. Do not redistribute,
mirror, or copy this online book.
So a license from David H. Ahl is required before publishing Python
translations.
The copyright applies to the book ("Do not redistribute, mirror, or copy
this *online book*.") and any derivative work is based on the book.
Using the video output from the BASIC games in the book could fall
underneath the fair use provision, which allows me to use a small
portion of the book without infringing on the copyright. I'm not
publishing a book. I just want to put my code on a website as an
educational example of what I did to convert a spaghetti language into a
modern scripting language.
Thanks,
Chris R.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On 21 June 2015 at 17:38, Sahlusar wrote:
>
> [snip]
> I do agree with you Denis that this is an unconventional approach. I was
> wondering then that perhaps I should add additional functionality at the XML
> to JSON step? So far, with JSON objects without nested lists (as values) I
> have been successful with this (the following is rather lengthy):
> [snip]
> ##JSON sample:
>
> data2 = {
> "OTF": "0",
> "F": "False",
> "F": {
> "Int32": ["0",
> "0",
> "0",
> "0"]
> },
> [snip]
> "PBDS": {
> "DateTime": ["1/1/0001 12:00:00 AM",
> "1/1/0001 12:00:00 AM",
> "1/1/0001 12:00:00 AM",
> "1/1/0001 12:00:00 AM"]
> },
> "PBDS": {
> "Double": ["0",
> "0",
> "0"]
> },
> "SCS": {
> "String": ["1",
> "2"]
> }
> }
>
> The result:
and compare those closely now
>
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> 'F_Int32': ['0',
> '0',
> '0',
> '0'],
> 'OTF': '0',
> 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
> 'SCS_String': ['1', '2']}
>
Notice in the original text you have 2 entries under the name F and
later 2 entiries under the name PBDS. in the result you are missing
the first entry of each.
you say you have succeeded in generating json, unless you meant to
throw away huge swafts of data i would say... nope..
[snip]
>
> I know that this is alot of sequential steps. I am wondering if I could
> insert or conditionally pass these functions when originally parsing the XML,
> so that the JSON is formatted for more recursive reading of the JSON
> dictionary and then writing to CSV? I welcome constructive feedback for
> refactoring
theres things you could do to fix up the generated json .. tho really,
stop generating json when you need to generate csv.
you are winning nothing. you are losing.. well pretty much .. a little
of everything .. by doing this
there are fundemental properties of xml and json you fail to grasp,
you are touting code claiming that it works when the output it
produces is horribly deformed :(
In xml for instance this is valid:
1
.. and so is this:
1
2
a naive translatio n of the first might yield
{"a":
{"b":1}
}
but this will not work with the second example, it would emit
{"a":
{"b":1,"b":2}
}
which really means
{"a":
{"b":2}
}
if you insist on emitting json as an intermediate step you need to
take care of these inconsistencies somehow.
you need to decide which behaviour you want and be explicit about it.
is it desireable that the last entry overrites the previous one? (you
have this now, i doubt this is what you want)
would you like some mergine behaviour? (some config file might work
well with this, or not)
would you like to have every entry be a list? (this is simple, but you
will end up with a lot of junk like {a:[{b:[1]}]}
do you wrap some things in list but not others?
and the conversion from json to CSV has similar issues ofc.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
"C.D. Reimer" : > Using the video output from the BASIC games in the book could fall > underneath the fair use provision, which allows me to use a small > portion of the book without infringing on the copyright. I'm not > publishing a book. I just want to put my code on a website as an > educational example of what I did to convert a spaghetti language into > a modern scripting language. As they say, tell that to the judge. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 06/21/2015 03:52 PM, C.D. Reimer wrote:
> The copyright applies to the book ("Do not redistribute, mirror, or copy
> this *online book*.") and any derivative work is based on the book.
> Using the video output from the BASIC games in the book could fall
> underneath the fair use provision, which allows me to use a small
> portion of the book without infringing on the copyright. I'm not
> publishing a book. I just want to put my code on a website as an
> educational example of what I did to convert a spaghetti language into a
> modern scripting language.
Completely agree. Either post code with your own copyright and pick any
license (you can change it later), such as BSD, or just release it as
public domain. I think your work could really be a good thing for
teaching purposes.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
Michael Torrie : > On 06/21/2015 02:58 PM, Marko Rauhamaa wrote: >> Converting BASIC games to Python results in derived works, which are >> under the original copyright of the BASIC games. >> >> [...] > > I disagree. Especially where the resulting python program is not a > transliteration (which it certainly won't be) of the original BASIC > programs. As well, these programs implement well-known algorithms and > games. There are only so many ways to implement certain algorithms, > and each implementation is going to be closely similar. The games and > algorithms in his BASIC programs are by no means original or not > well-known and well-discussed. The court will have to decide if the Python version is a reimplementation or a more or less direct translation of the original. > Also the quoted copyright notice is for the entire work, which is to say > the book. Chris is in no way reproducing the guy's text in whole or in > part--I note that the notice says nothing about fair use. Fair use is not granted. In United States copyright law, fair use is a doctrine that permits limited use of copyrighted material without acquiring permission from the rights holders. https://en.wikipedia.org/wiki/Fair_use> Again, whether a fair use defense applies in this case is for the courts to decide. My guess is it wouldn't apply here. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 21/06/2015 22:52, C.D. Reimer wrote:
On 6/21/2015 1:58 PM, Marko Rauhamaa wrote:
Converting BASIC games to Python results in derived works, which are
under the original copyright of the BASIC games.
From the given link:
BASIC Computer Games is copyright © 1978 by David H. Ahl, and is
posted onwww.atariarchives.org with permission. Do not redistribute,
mirror, or copy this online book.
So a license from David H. Ahl is required before publishing Python
translations.
The copyright applies to the book ("Do not redistribute, mirror, or copy
this *online book*.") and any derivative work is based on the book.
Using the video output from the BASIC games in the book could fall
underneath the fair use provision, which allows me to use a small
portion of the book without infringing on the copyright. I'm not
publishing a book. I just want to put my code on a website as an
educational example of what I did to convert a spaghetti language into a
modern scripting language.
Thanks,
Chris R.
Regarding fair use I suppose it depends on which country you're (plural)
in. From
http://www.copyrightservice.co.uk/copyright/p22_derivative_works.en.htm
Can I claim that my copy is fair use/fair dealing, or de minimis?
Unless your activities are explicitly allowed under law, there is no
solid legal footing for such a claim.
Having said that I agree with Michael Torrie's earlier comments so I
suggest you get on with it, noting at the same time that my legal skills
are nowhere near as good as my computing skills, which is really saying
something :)
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote: > In xml for instance this is valid: > > 1 > > .. and so is this: > > 1 2 > What the OP needs to do is sit down with the XML and work out how it needs to be represented in CSV terms, and then code that transformation, but it appears that he's not listening. ie, does your xml: 1 2 translate to CSV: "a","b" // headers "b",1 // data row 1 "b",2 // data row 2 or to CSV: "a", "b", "b" // headers "", 1, 2 // data row or even CSV: "b" // headers 1 // data row 1 2 // data row 2 If he can't codify that in a consistent manner across all the XML he wishes to process, then he really does need to find someone competent to do the job instead of wallowing around in json until the client gives up in despair at the lack of progress and finds someone else to do the job. This should really have been defined by whoever set the task to do the conversion. If the job is to convert from some XML DTD to a CSV format, then there should be a clear description of what extracts from the XML are expected to be in which positions in the CSV. This is the sort of data conversion code I generally turn out in a day or so, it's hardly rocket science as long as you have a clear description of what is required. If you don't have a clear description of what is required, you have to keep asking questions until you get one. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Sunday, June 21, 2015 at 4:54:27 PM UTC-4, Denis McMahon wrote:
> On Sun, 21 Jun 2015 06:57:01 -0700, sahluwalia wrote:
>
> > On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon wrote:
> >> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
> >>
> >> > I would like to have this JSON object written out to a CSV file so
> >> > that the keys are header fields (for each of the columns) and the
> >> > values are values that are associated with each header field.
> >>
> >> > {
> >> > "CF": {
> >> ...
> >> > "CF": "Fee",
> >>
> >> Your json object seems to have the same key used for two elements at
> >> the same level, are you sure this is legal json?
>
> > I converted this from an XML file given to me from a third party. It is
> > as is. I am not sure what you mean by "valid"; that is a very subjective
> > measure for any form of quantitative or qualitative data.
>
> Put it this way, when I feed your json object into a jason lint, the
> output is the following:
>
> {
> "CF": "Fee",
> "ID": "2"
> }
>
> The second occurrence in the parent object of name "CF" with a value of
> string literal "Fee" overwrites the earlier name "CF" whose value is (in
> python terms) a dictionary or (in json terms) an object.
>
> --
> Denis McMahon, denismfmcma...@gmail.com
I am stepping back and rethinking the notion of JSON/dictionaries. Maybe this
was a very overcomplicated approach for this type of data.
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On Sunday, June 21, 2015 at 5:56:00 PM UTC-4, Waffle wrote:
> On 21 June 2015 at 17:38, Sahlusar wrote:
> >
> > [snip]
> > I do agree with you Denis that this is an unconventional approach. I was
> > wondering then that perhaps I should add additional functionality at the
> > XML to JSON step? So far, with JSON objects without nested lists (as
> > values) I have been successful with this (the following is rather lengthy):
> > [snip]
>
>
> > ##JSON sample:
> >
> > data2 = {
> > "OTF": "0",
> > "F": "False",
> > "F": {
> > "Int32": ["0",
> > "0",
> > "0",
> > "0"]
> > },
> > [snip]
> > "PBDS": {
> > "DateTime": ["1/1/0001 12:00:00 AM",
> > "1/1/0001 12:00:00 AM",
> > "1/1/0001 12:00:00 AM",
> > "1/1/0001 12:00:00 AM"]
> > },
> > "PBDS": {
> > "Double": ["0",
> > "0",
> > "0"]
> > },
> > "SCS": {
> > "String": ["1",
> > "2"]
> > }
> > }
> >
> > The result:
>
> and compare those closely now
>
> >
> > {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> > 'F_Int32': ['0',
> > '0',
> > '0',
> > '0'],
> > 'OTF': '0',
> > 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
> > 'SCS_String': ['1', '2']}
> >
> Notice in the original text you have 2 entries under the name F and
> later 2 entiries under the name PBDS. in the result you are missing
> the first entry of each.
> you say you have succeeded in generating json, unless you meant to
> throw away huge swafts of data i would say... nope..
>
>
>
> [snip]
> >
> > I know that this is alot of sequential steps. I am wondering if I could
> > insert or conditionally pass these functions when originally parsing the
> > XML, so that the JSON is formatted for more recursive reading of the JSON
> > dictionary and then writing to CSV? I welcome constructive feedback for
> > refactoring
>
> theres things you could do to fix up the generated json .. tho really,
>
> stop generating json when you need to generate csv.
> you are winning nothing. you are losing.. well pretty much .. a little
> of everything .. by doing this
>
> there are fundemental properties of xml and json you fail to grasp,
> you are touting code claiming that it works when the output it
> produces is horribly deformed :(
>
> In xml for instance this is valid:
>
>
> 1
>
> .. and so is this:
>
> 1
> 2
>
>
> a naive translatio n of the first might yield
> {"a":
> {"b":1}
> }
> but this will not work with the second example, it would emit
> {"a":
> {"b":1,"b":2}
> }
> which really means
> {"a":
> {"b":2}
> }
>
> if you insist on emitting json as an intermediate step you need to
> take care of these inconsistencies somehow.
> you need to decide which behaviour you want and be explicit about it.
> is it desireable that the last entry overrites the previous one? (you
> have this now, i doubt this is what you want)
> would you like some mergine behaviour? (some config file might work
> well with this, or not)
> would you like to have every entry be a list? (this is simple, but you
> will end up with a lot of junk like {a:[{b:[1]}]}
> do you wrap some things in list but not others?
>
> and the conversion from json to CSV has similar issues ofc.
I am rethinking my strategy and going back to the drawing board. The data is
malformed. I am quite familiar with JSON and XML. I am trying to work within
the parameters given to me. I will rethink this. Thank you for giving me a
reality check with a splash of water. Sometimes it is hard to get one's head
out of the groundespecially when one is new to enterprise and commercial
development.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 6/21/2015 3:02 PM, Marko Rauhamaa wrote: As they say, tell that to the judge. More than likely, the original copyright owner can issue an DMCA take down notice and that will be end of that. Thanks, Chris R. -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV File Troubleshooting
On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote: > On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote: > > > In xml for instance this is valid: > > > > > 1 > > > > .. and so is this: > > > > 1 2 > > > > What the OP needs to do is sit down with the XML and work out how it > needs to be represented in CSV terms, and then code that transformation, > but it appears that he's not listening. > > ie, does your xml: > > > 1 2 > > > translate to CSV: > > "a","b" // headers > "b",1 // data row 1 > "b",2 // data row 2 > > or to CSV: > > "a", "b", "b" // headers > "", 1, 2 // data row > > or even CSV: > > "b" // headers > 1 // data row 1 > 2 // data row 2 > > If he can't codify that in a consistent manner across all the XML he > wishes to process, then he really does need to find someone competent to > do the job instead of wallowing around in json until the client gives up > in despair at the lack of progress and finds someone else to do the job. > > This should really have been defined by whoever set the task to do the > conversion. If the job is to convert from some XML DTD to a CSV format, > then there should be a clear description of what extracts from the XML > are expected to be in which positions in the CSV. > > This is the sort of data conversion code I generally turn out in a day or > so, it's hardly rocket science as long as you have a clear description of > what is required. If you don't have a clear description of what is > required, you have to keep asking questions until you get one. > > -- > Denis McMahon, denismfmcma...@gmail.com On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote: > On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote: > > > In xml for instance this is valid: > > > > > 1 > > > > .. and so is this: > > > > 1 2 > > > > What the OP needs to do is sit down with the XML and work out how it > needs to be represented in CSV terms, and then code that transformation, > but it appears that he's not listening. > > ie, does your xml: > > > 1 2 > > > translate to CSV: > > "a","b" // headers > "b",1 // data row 1 > "b",2 // data row 2 > > or to CSV: > > "a", "b", "b" // headers > "", 1, 2 // data row > > or even CSV: > > "b" // headers > 1 // data row 1 > 2 // data row 2 > > If he can't codify that in a consistent manner across all the XML he > wishes to process, then he really does need to find someone competent to > do the job instead of wallowing around in json until the client gives up > in despair at the lack of progress and finds someone else to do the job. > > This should really have been defined by whoever set the task to do the > conversion. If the job is to convert from some XML DTD to a CSV format, > then there should be a clear description of what extracts from the XML > are expected to be in which positions in the CSV. > > This is the sort of data conversion code I generally turn out in a day or > so, it's hardly rocket science as long as you have a clear description of > what is required. If you don't have a clear description of what is > required, you have to keep asking questions until you get one. > > -- > Denis McMahon, denismfmcma...@gmail.com On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote: > On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote: > > > In xml for instance this is valid: > > > > > 1 > > > > .. and so is this: > > > > 1 2 > > > > What the OP needs to do is sit down with the XML and work out how it > needs to be represented in CSV terms, and then code that transformation, > but it appears that he's not listening. > > ie, does your xml: > > > 1 2 > > > translate to CSV: > > "a","b" // headers > "b",1 // data row 1 > "b",2 // data row 2 > > or to CSV: > > "a", "b", "b" // headers > "", 1, 2 // data row > > or even CSV: > > "b" // headers > 1 // data row 1 > 2 // data row 2 > > If he can't codify that in a consistent manner across all the XML he > wishes to process, then he really does need to find someone competent to > do the job instead of wallowing around in json until the client gives up > in despair at the lack of progress and finds someone else to do the job. > > This should really have been defined by whoever set the task to do the > conversion. If the job is to convert from some XML DTD to a CSV format, > then there should be a clear description of what extracts from the XML > are expected to be in which positions in the CSV. > > This is the sort of data conversion code I generally turn out in a day or > so, it's hardly rocket science as long as you have a clear description of > what is required. If you don't have a clear description of what is > required, you have to keep asking questions until you get one. > > -- > Denis McMahon, denismfmcma...@gmail.com I have gone back to the drawing boa
Re: Do I need license to release the Python version of old BASIC games?
On Mon, Jun 22, 2015 at 9:54 AM, C.D. Reimer wrote: > On 6/21/2015 3:02 PM, Marko Rauhamaa wrote: >> >> As they say, tell that to the judge. > > > More than likely, the original copyright owner can issue an DMCA take down > notice and that will be end of that. Or, alternatively, the original copyright owner will be tickled to see his work given a new lease on life, and will be thanking you for doing it. That's also possible. As long as you acknowledge the prior work, most people are happy to see their stuff used. Slap a simple free license on your actual code (you suggested MIT earlier, and I support that notion strongly), and include some sort of pointer to the BASIC code and book, and most likely the matter will never go to court. For extra safety, though, do your best to get in contact with the original author and ask how best to acknowledge the book. That would cover you. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On Mon, 22 Jun 2015 09:54 am, C.D. Reimer wrote: > On 6/21/2015 3:02 PM, Marko Rauhamaa wrote: >> As they say, tell that to the judge. > > More than likely, the original copyright owner can issue an DMCA take > down notice and that will be end of that. The way the DMCA is implemented in practice, anybody can issue a DMCA take down notice and that will be the end of that. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On Mon, 22 Jun 2015 07:28 am, Michael Torrie wrote: > As to the question of assigning a copyright license to code, in this > case I suggest just releasing the code marked as public domain. Public domain is not a licence, and many places (including the US) do not allow individuals to put works into the public domain. (US government works are a special case.) Some places will not recognise a public domain dedication, and will treat the work as being legally copyrighted by you with All Rights Reserved. Some legal jurisdictions prohibit owners from surrendering some or all rights. You might not choose to pursue a copyright claim, but your heirs might, which means that a "public domain" release is legally unsafe unless the work actually is in the public domain due to age. For that reason, you are better off publishing the work with a Creative Commons CC0 licence: https://creativecommons.org/about/cc0 https://wiki.creativecommons.org/wiki/Public_domain As far as whether your work counts as a derivative work, you *may* be on solid ground if you just re-use the algorithms from the BASIC code, and not either the author's descriptive text or the BASIC code itself. But I am not a lawyer (neither is anyone else here). My advice is to make an attempt to contact the original author, or at least the atarigames website, explain that you have converted the code to Python, and ask permission to publish under the cc0 licence or similar. It's just safer that way. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On 06/21/2015 08:27 PM, Steven D'Aprano wrote: > Public domain is not a licence, and many places (including the US) do not > allow individuals to put works into the public domain. (US government works > are a special case.) Some places will not recognise a public domain > dedication, and will treat the work as being legally copyrighted by you > with All Rights Reserved. Some legal jurisdictions prohibit owners from > surrendering some or all rights. > > You might not choose to pursue a copyright claim, but your heirs might, > which means that a "public domain" release is legally unsafe unless the > work actually is in the public domain due to age. Yes of course public domain is not a source code license. That's not what I meant anyway. But sure, pick some license like MIT or BSD and call it good. As for it being not possible to place something in the public domain, that's news to me and certainly news to a number of projects including Libravox. All of their recordings state that they are in the public domain. Of course the texts being read are in the public domain by virtue of their copyright expiring. But the recordings themselves could be copyrighted, but are not. >From some brief research, it appears there is some question about the ability to declare something to be in the public domain, but it is by no means a sure thing and lots of people feel it's just fine to declare something to be in the public domain. What his heirs try to assert is probably not his concern nor would I worry about it. This is wandering in the weeds in typical python-list style! -- https://mail.python.org/mailman/listinfo/python-list
Re: Do I need license to release the Python version of old BASIC games?
On Mon, Jun 22, 2015 at 2:23 PM, Michael Torrie wrote: > From some brief research, it appears there is some question about the > ability to declare something to be in the public domain, but it is by no > means a sure thing and lots of people feel it's just fine to declare > something to be in the public domain. What his heirs try to assert is > probably not his concern nor would I worry about it. This is wandering > in the weeds in typical python-list style! It's a consideration for anyone who wants to make use of it. Imagine, instead of stating that the recordings are in the public domain, it said "Copyright, blah blah, but we pledge not to sue anyone for copyright violation for an unspecified and unspecifiable period". Would you use it? Because that's potentially what a "public domain" declaration is, with the period being defined as "until the current owner dies". That's not something to depend on, so I'd much rather give things away under something more explicit like MIT. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
