"Yes, if you find that appropriate or if they share some implementation bits (I'm not sure of the latter, but it may help to organise stuff). What you do is, in the grammar, you add a %hierarchy directive, like that:
%hierarchy Atom (List Dictionary); And, at AST generation, SmaCC will inherit as much as possible from Atom definition in List and Dictionary (at least, I suppose it does: some of the SmaCC code generation tools are rather impressive, and, if you look carefully, it is prepared for more than just generation of Smalltalk code)" Nice and I was meaning to ask that how to do this with the grammar. Great answer. "For me, the way to deal with that is to have a model of that data and visitors on it, to tackle the "future" complexity. If you go that way (recursive structure), two good (and not that easy) things to have are: equality and copy." this I dont understand. I want python types to convert to popular pharo data classes . I don't want to create my own custom classes because the way I see it python and pharo are very close together as dynamic languages. So my thinking is that if equality works for an ordered collection and a array , then there is no need for me to add anything new since I will convert to those classes. Unless you mean something else that I am missing here. My goal is to allow people to use python code and libraries without having to worry for how the python side maps to the pharo side. At least not when it comes to types. So they will use those python libraries as if they are pharo libraries. A challenge will be converting types that reference python objects. The good news is that SmaCC already handles this situation but I will have to find a way to sync references between pharo and python. I think I know a way for that one. "Good point. It would be nice to have a diff on the grammars of Python2 and Python3; at the moment, there is space in the naming for a Python 3 parser (named PythonParser3, of course), however we would have a collision on the AST nodes (two PyAtomNodes, two ...). Have to consider that and maybe have Py2AtomNode, etc... to leave space for the Python 3 AST if it differs too much." >From what I have seen , because I have to confess I am more of python 3 than python 2 coder, there are no big diffirences in python types at least not on the basic ones, but syntax wise there is as it is explained very well in this reference http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/key_differences_between_python_2_and_3.ipynb So yes there is a clear diffirence between python 3 and python 2 but its not massive. And yes those 2 grammars should be kept separately. For the time being I have no reason to encourage you to make a python 3 grammar port, because frankly I dont need it. I am happy with SmaCC as it is. There is no need for my project to fully parse python syntax and I dont see in the next years at least any need to go beyond python types. If the need arises and no py3 parser exists I will of course make it myself and send you a pull request. I am so grateful that already SmaCC has saved me from so much work , thanks to all people contributing to SmaCC , one very happy customer :D On Wed, Feb 11, 2015 at 7:36 PM, kilon alios <kilon.al...@gmail.com> wrote: > Well I tried in the past to use openqwaq , but what I learned from the > experience is that sometimes understanding others code can be even more > time consuming than remaking it yourself. I spent like an afternoon trying > to understand openqwaq before starting Ephestos from complete scratch. > Suffice to say that in another afternoon I had the prototype of Ephestos > working with the ability to send simple python strings to the cptyhon > interpreter to be execute by the time I was not even able to understand the > basic architecture of openqwaq. > > The module you recommended alone contains like 11.000 lines of code. Thats > a lot of work to just read that code and I cant find any class named > CModel. In any case I like it that I started from scratch, I want to keep > my project small, light, well documented and super easy to hack. Even if > that means less features :) > > Also the code is written for squeak not pharo. In any case I will need > someone like Thierry to guide me through the code and help me understand. > So its definitely not as simple as "hey I am going to use that code". > > Executing python code with Ephestos is already possible and easy , my > problems now are a) converting from python to pharo types which is why I am > using SmaCC b) use of callbacks , which should be easy to do , so that > python code can call pharo code. In the future if socket are proven too > slow I may move my architecture into a shared DLL instead to make things > faster. But as far as using python code from pharo I am very close to > achieving my goal by the end of this year. > > On Wed, Feb 11, 2015 at 4:40 PM, Sebastian Heidbrink <shei...@yahoo.de> > wrote: > >> Hi! >> >> Maybe you also want to have a look at >> http://www.squeaksource.com/openqwaq/ >> >> There is a part PyBridge included. MAybe you can take some of the >> Smalltalk Python Cmodel classes from there. >> >> Sebastian >> >> >> Am 10.02.2015 um 23:24 schrieb kilon alios: >> >> Ok so after rereading the tutorial and testing again and again , I think >> I have finally managed to understand how SmaCC really works and I was >> succesful into converting simple python litsts to pharo arrays and ordered >> collections. >> >> The tricky part now is to apply this knowledge to complex python types >> like multi dimensional lists, tuples and dictionaries. I understand that >> the visitor allows me to visit a specific object instances each time they >> are found in the AST . But because I want to walk the AST in order to build >> multi dimensional ordered collections I need something more, or maybe my >> understanding of the visitor pattern is flawed. >> >> The problem I am having here is that each time I parse a python type >> that python type is not necessarily represented by different kind of node. >> For example whether its a list or a tuple or a dictionary the same class is >> used PyAtomNode. In order to differentiate between those diffirent python >> types PyAtomNode has instance variables for right and left bracket, >> parantheses, and curly. So my initial thinking is to check those instance >> variables to see if they are nil and from that I can conclude which python >> type I am parsing. >> >> So I can preform simple ifs that check that the instance variable is >> Nil or not but the question is if my strategy is a good one or a bad one. >> >> I could define my own syntax to simplify the AST tree including >> different nodes for different python types , because from the looks of it , >> it seems it is a bit too verbose for my needs but On the other hand I am >> not so sure because in the future my needs may become more verbose too. >> >> So I am very close and ready to create my full python types converter >> for pharo but I wanted some good advice before wasting time on something >> that is not efficient. >> >> >> By the way Thierry I have to agree with you Smacc is a very capable >> parser, also I like the use of regex syntax, makes it uglier compared >> Pettit Parser but I prefer the compact regex syntax to having to define and >> browse tons of classes and send tons of messages. Also the Python support >> is very good and I am impressed how easily SmaCC can parse whole python >> applications since some of the test are very complex. Well done great work! >> >> >> >> On Sat, Jan 31, 2015 at 12:04 AM, kilon alios <kilon.al...@gmail.com> >> wrote: >> >>> thank for your congratulations, because at times I fear I ask too >>> obvious questions. I have to say I find this parsing very complex but very >>> fascinating too :) Time to experiment with the visitor. >>> >>> On Fri, Jan 30, 2015 at 11:49 PM, Thierry Goubier < >>> thierry.goub...@gmail.com> wrote: >>> >>>> >>>> >>>> 2015-01-30 14:04 GMT+01:00 kilon alios <kilon.al...@gmail.com>: >>>> >>>>> Ok thanks for the info, I am still however curious about these "tests" >>>>> are just tests (which may or may not happen) that determine the AST, for >>>>> example which node to use ? >>>>> >>>> >>>> 'tests' is just there because, in the grammar, there is this at a >>>> certain point: >>>> >>>> testlist: >>>> test 'test' "," testlist >>>> | test 'test' comma_opt >>>> ; >>>> >>>> I have named this use of test 'test', so SmaCC has deduced that >>>> testlist will be a list of test(s) nodes (or maybe other stuff such as >>>> atoms, depending on the productions for test).. so, SmaCC, in each rule >>>> where testlist is found, it will add a 'tests' instance variable. >>>> >>>> So, basically, the grammar rules explain how each node can be >>>> decomposed in sub-nodes, and the additional annotations (the 'test' naming >>>> and the {{}} or {{}}) drive how the classes for the nodes you want to keep >>>> are generated. In that testlist case, no node will be generated, but >>>> everywhere testlist appear on the right of the rule, then it will add a >>>> 'tests' instance variable. >>>> >>>> >>>>> >>>>> Or are they tests related to unit testing class PythonParserTests ? >>>>> >>>> >>>> Not at all :) >>>> >>>> >>>>> >>>>> Also you said I need to use the visitor created by PythonParser I >>>>> assume you mean PyRootNodeVisitor ? Just as it is explained in the AST >>>>> chapter of the documentation. >>>>> >>>>> In my case this simple python list will need me to subclass it and >>>>> override method visitListmaker , the aListmaker passed as argument to the >>>>> method should I assume it is PyListmakerNode ? >>>>> >>>> >>>> In my experience, what you need to do is you have a look at the ast >>>> generated and see if you can recognize the elements. From what I see in >>>> your simple example, the key to your list is that PyAtomNode instance with >>>> something in list. Once you have that, you know that you need to visit >>>> PyAtomNode (and check that it has the [ ] tokens). >>>> >>>> Looking into what is listmaker in atom in the grammar >>>> (congratulations by the way, you have seen it :) ), you'll see that it >>>> creates a listmaker node only in the first case: test followed by a >>>> list_for, otherwise it falls back to testlist... >>>> >>>> Thierry >>>> >>>> >>>> >>>>> >>>>> On Fri, Jan 30, 2015 at 10:50 AM, Thierry Goubier < >>>>> thierry.goub...@gmail.com> wrote: >>>>> >>>>>> Hi kilon, >>>>>> >>>>>> The tests instance variable is linked to the python grammar: top >>>>>> level items in an expression are probably tests, and, through the >>>>>> grammar, >>>>>> tests can be just atoms. >>>>>> >>>>>> So the tests instance variable doesn't means it is testing anything :) >>>>>> >>>>>> Thierry >>>>>> Le 30 janv. 2015 09:23, "kilon alios" <kilon.al...@gmail.com> a >>>>>> écrit : >>>>>> >>>>>> Ok so I tried to parse a very simple list like >>>>>>> >>>>>>> [ 67,12342,5 ] >>>>>>> >>>>>>> using Parse and explore , I can find these number by following >>>>>>> this AST (for example 67) >>>>>>> >>>>>>> PyFileInputNode>>statements: -> 1: PySimpleStmNode>>stmts: -> 1: >>>>>>> PyExprStmtNode>>tests: ->1: PyPowerNode>>atom: -> PyAtomNode>>list: -> >>>>>>> 1: >>>>>>> PyPowerNode>>atom: -> PyPowerNode>>numberToken -> numberToken>>value -> >>>>>>> 67 >>>>>>> >>>>>>> quite a structure, but the one thing I dont get is "tests" , why >>>>>>> "tests" ? >>>>>>> >>>>>>> Does it tests something and if yes what ? >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >
