[OT] Simulation Results Managment
Hi, This is a general question, loosely related to python since it will be the implementation language. I would like some suggestions as to manage simulation results data from my ASIC design. For my design, - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within each of these test cases we have: - a number of properties (P_AA_BB_CC) - For each property, the following information is given - Property name (P_NAME) - Number of times it was checked (within the testcase) N_CHECKED - Number of times if failed (within the testcase) N_FAILED - A simulation runs a testcase with a set of parameters. - Simple example, SLOW_CLOCK, FAST_CLOCK, etc - For the design, I will run regression every night (at least), so I will have results from multiple timestamps We have < 1000 TESTCASES, and < 1000 PROPERTIES. At the moment, I have a script that extracts property information from simulation logfile, and provides single PASS/FAIL and all logfiles stored in a directory structure with timestamps/testnames and other parameters embedded in paths I would like to be easily look at (visualize) the data and answer the questions - When did this property last fail, and how many times was it checked - Is this property checked in this test case. Initial question: How to organize the data within python? For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, N_FAILED I then have to store multiple instances of testcase based on date (and simulation parameters. Any comments, suggestions? Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Simulation Results Managment
On Sunday, July 15, 2012 2:42:39 AM UTC+2, Neal Becker wrote: > me wrote: > > > Hi, > > This is a general question, loosely related to python since it will be > the > > implementation language. I would like some suggestions as to manage > simulation > > results data from my ASIC design. > > > > For my design, > > - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within > each of > > these test cases we have: > > - a number of properties (P_AA_BB_CC) > > - For each property, the following information is given > > - Property name (P_NAME) > > - Number of times it was checked (within the testcase) N_CHECKED > > - Number of times if failed (within the testcase) N_FAILED > > - A simulation runs a testcase with a set of parameters. > > - Simple example, SLOW_CLOCK, FAST_CLOCK, etc > > - For the design, I will run regression every night (at least), so I > will have > > results from multiple timestamps We have < 1000 TESTCASES, and < > 1000 > > PROPERTIES. > > > > At the moment, I have a script that extracts property information from > > simulation logfile, and provides single PASS/FAIL and all logfiles > stored in a > > directory structure with timestamps/testnames and other parameters > embedded in > > paths > > > > I would like to be easily look at (visualize) the data and answer the > > questions - When did this property last fail, and how many times was it > > checked - Is this property checked in this test case. > > > > Initial question: How to organize the data within python? > > For a single testcase, I could use a dict. Key P_NAME, data in N_CHECKED, > > N_FAILED I then have to store multiple instances of testcase based on > date > > (and simulation parameters. > > > > Any comments, suggestions? > > Thanks, > > Steven > > One small suggestion, > I used to store test conditions and results in log files, and then write > parsers > to read the results. The formats kept changing (add more > conditions/results!) > and maintenance was a pain. > > Now, in addition to a text log file, I write a file in pickle format > containing > a dict of all test conditions and results. Much more convenient. Hi Neal, We already store the original log files. Does pickle have any advantages over json/yaml? Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Simulation Results Managment
On Sunday, July 15, 2012 5:25:14 AM UTC+2, rusi wrote: > On Jul 14, 10:50 am, moo...@yahoo.co.uk wrote: > > Hi, > > This is a general question, loosely related to python since it will be > the implementation language. > > I would like some suggestions as to manage simulation results data from > my ASIC design. > > > > For my design, > > - I have a number of simulations testcases (TEST_XX_YY_ZZ), and within > each of these test cases we have: > > - a number of properties (P_AA_BB_CC) > > - For each property, the following information is given > > - Property name (P_NAME) > > - Number of times it was checked (within the testcase) N_CHECKED > > - Number of times if failed (within the testcase) N_FAILED > > - A simulation runs a testcase with a set of parameters. > > - Simple example, SLOW_CLOCK, FAST_CLOCK, etc > > - For the design, I will run regression every night (at least), so I > will have results from multiple timestamps > > We have < 1000 TESTCASES, and < 1000 PROPERTIES. > > > > At the moment, I have a script that extracts property information from > simulation logfile, and provides single PASS/FAIL and all logfiles stored in > a directory structure with timestamps/testnames and other parameters embedded > in paths > > > > I would like to be easily look at (visualize) the data and answer the > questions > > - When did this property last fail, and how many times was it checked > > - Is this property checked in this test case. > > > > Initial question: How to organize the data within python? > > For a single testcase, I could use a dict. Key P_NAME, data in > N_CHECKED, N_FAILED > > I then have to store multiple instances of testcase based on date (and > simulation parameters. > > > > Any comments, suggestions? > > Thanks, > > Steven > > Not sure if you are asking about: > 1. Python data structure organization > or > 2. Organization of data outside python for conveniently getting in and > out of python > > For 2. if the data is modestly sized and is naturally managed with > builtin python types -- lists and dictionaries -- yaml gives a nice > fit. I used pyyaml some years ago, today I guess json which is > similar, is the way to go. > > For 1, you need to say what are your questions/issues. Hi Rusi, For (1), I guess that the only question I had was how to handle regression results. But I think that the most logical way for string this data is as a dict with key = datestamp, and entries being list of testcases/results. For (2), I will look at both these. Thanks for the help. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Simulation Results Managment
On Sunday, July 15, 2012 6:20:34 PM UTC+2, rusi wrote: > On Jul 15, 11:35 am, Dieter Maurerwrote: > > moo...@yahoo.co.uk writes: > > > ... > > > Does pickle have any advantages over json/yaml? > > > > It can store and retrieve almost any Python object with almost no effort. > > > > Up to you whether you see it as an advantage to be able to store > > objects rather than (almost) pure data with a rather limited type set. > > > > Of course, "pickle" is a proprietary Python format. Not so > easy to > > decode it with something else than Python. In addition, when > > you store objects, the retrieving application must know the classes > > of those objects -- and its knowledge should not be too different > > from how those classes looked when the objects have been stored. > > > > I like very much to work with objects (rather than with pure data). > > Therefore, I use "pickle" when I know that the storing and > retrieving > > applications all use Python. I use pure (and restricted) data formats > > when non Python applications come into play. > > Pickle -> JSON -> Yaml > are roughly in increasing order of human-friendliness and decreasing > order of machine friendliness (where machine means python 'machine') > > This means that > - Pickle is most efficient, Yaml least > - Pickle comes with python from as far back as I know >Json started coming somewhere round 2.5 (I think) >(py)yaml needs to be installed separately > - reading pickled data will spoil your eyes whereas yaml is pleasant > to read (just like python) Hi Everyone, Thanks for the feedback. For now, I store the data using Pickle. Steven -- http://mail.python.org/mailman/listinfo/python-list
Basic JSON question: Do I really need the quotes
Hi, I need to define some configuration in a file that will be manually created. Internally, the data will be stored as a dict, which contains various properties related to a design e.g. Design Name, dependencies, lists of files (and associated libraries). json seemed a quick an easy way of achieving this Anyway, in simple terms my question - if I know everything is a string, how can I omit the quotation marks? i.e. I can do >>> json.loads('{"mykey":["data0", "data1"]}') {u'mykey': [u'data0', u'data1']} But I would like to do >>> json.loads('{mykey:[data0, data1]}') Traceback (most recent call last): The problem is that I don't want to make users have to type redundant characters. Is it possible? Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Basic JSON question: Do I really need the quotes
On Friday, 12 October 2012 16:09:14 UTC+2, (unknown) wrote: > Hi, > > I need to define some configuration in a file that will be manually created. > > Internally, the data will be stored as a dict, which contains various > properties related to a design > > e.g. Design Name, dependencies, lists of files (and associated libraries). > > json seemed a quick an easy way of achieving this > > Anyway, in simple terms my question - if I know everything is a string, how > can I omit the quotation marks? > > > > i.e. I can do > > > > >>> json.loads('{"mykey":["data0", "data1"]}') > > {u'mykey': [u'data0', u'data1']} > > > > But I would like to do > > >>> json.loads('{mykey:[data0, data1]}') > > Traceback (most recent call last): > > > > The problem is that I don't want to make users have to type redundant > characters. > > Is it possible? > > Thanks, > > Steven Hi, Thanks to everyone for the responses. I'll look at YAML and ConfigParser. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Delay Format (SDF) Parsing
On Monday, 26 May 2014 08:15:53 UTC+2, garg.p...@gmail.com wrote: > Hi Steven, > > > > did you get the module to parse the sdf file? > > > > regards, > > pankaj Unfortunately not. I actually can't remember why I wanted/needed this. I guess that it wasn't that important since I didn't actually do any parsing. Steven -- https://mail.python.org/mailman/listinfo/python-list
OT : Bug/Issue tracking systems
Hi, (Off-topic) I am looking to put an open-source bug/issue tracking system in place for our current project (eventually expanded for all projects), and would appreciate any experiences/comments/suggestions. Note the project is encompasses embedded hardware (ASIC plus firmware) plus application software. The easiest method is using a spreadsheet, but this is not very expandable. The requirements I have come up with - Free ;-) - Easy to setup and maintain (I want this to be an engieering tool, not an IT project) - A non linux expert should be able to download and install (rpms OK, deep understanding of makefiles and linux kernel not OK) - Ideally via text files, a bit of perl/tcl/python OK. I'd rather avoid SQL - Prove the system and then hand-off to IT for maintenance - Easy use - Browser UI - Mail ? - Linux - Flexible reporting/search - User/admin accounts - Initially internal network access only, eventually external (customer, partner) access - Cover HDL, H/W, F/W, Documentation, Change requests. Both ASIC and FPGA. - Eventually production issues (Yeild, Test programs?) - Maybe allow project deadlines included. - We use CVS, so any loose coupling useful - We have per project repositories, plus and repository containing common IP's (i.e a project will always use 2) - Medium size projects (upto 15-20 people) - Possible migration to other system in future (which I guess means a well supported database) Googling provided with lots of names - Bugzilla (seems to be in widest use for S/W projects) - GNATS (I recall using this in a previous job) - IssueTrackerSystem (ZOPE, Python) - Trac (Python) - Plus lots of others Any suggestions, comments, recommendations or pointers to papers/tutorals greatly appreciated. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: OT : Bug/Issue tracking systems
Marc 'BlackJack' Rintsch wrote: > In <[EMAIL PROTECTED]>, moogyd wrote: > > > The requirements I have come up with > > > > [...] > > > > - Ideally via text files, a bit of perl/tcl/python OK. I'd rather > > avoid SQL > > You should drop that requirement. The tracker will be used concurrently > and this is handled very efficiently and reliable by a database backend. > > Ciao, > Marc 'BlackJack' Rintsch Hi Marc, I am aware that I will probably need a database (probably supporting SQL), but I would like to avoid having to write SQL queries myself (i.e. It should all be transparent to me). Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Newbie Question : "grep"
Hello, I am attempting to write my first Python script to extract some information from a file, and place it into another file. (I am trying to find the physical postions of 4 cells within an FPGA) I have a working solution, and would appreciate any comments. for line in lines: if "placed" in line: if "i_a/i_b/ROM/" in line: pos = (line.split()[4]).split("_")[1] if "B18" in line: print "i_a/i_b/ROM/B18 [7:6] LOC =", pos elif "B14" in line: print "i_a/i_b/ROM/B14 [5:4] LOC =", pos elif "B10" in line: print "i_a/i_b/ROM/B10 [3:2] LOC =", pos elif "B6" in line: print "i_a/i_b/ROM/B6 [1:0] LOC =", pos else: print "Error" Specific questions - Use for "Phrase" in line or line.find("Phrase") >= 0 ? - If I increase number of elements I am searching for, then I have more elif...elif. Is there a cleaner solution? Thanks for any feedback. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Question : "grep"
On 12 Mar, 18:55, [EMAIL PROTECTED] wrote: > On Mar 12, 10:01 am, "Erik Johnson" <[EMAIL PROTECTED]> wrote: > > > > > Sorry, I forgot to paste the modified version of my code in the post:. I > > think this is the same behaviour: > > > for line in lines: > > if "placed" in line: > > if "i_a/i_b/ROM/" in line: > > pos = (line.split()[4]).split("_")[1] > > found = False > > > for (tag, (start, end)) in tags: > > if tag in line: > > found = True > > print "i_a/i_b/ROM/%s [%i:%i] LOC =" %\ > > (pos, tag, start, end) > > break > > > if not found: > > print "Error" > > Instead of using a sentinal value (i.e., found), > one can use the 'else' clause of a 'for' loop... > > pos = (line.split()[4]).split("_")[1] > for (tag, (start,end)) in tags: > if tag in line: > print "i_a/i_b/ROM/%s [%i:%i] LOC=%s" %\ > (pos,tag,start,end) > break > else > print "Error" > > -- > Hope this helps, > Steven Thanks for the responses. I had an inkling what the solution may be, but I wondering whether I could avoid the nested loops (i.e. loop every line and then loop every pattern), maybe using maps or list comprehensions (this is just intellectual curiosity as the current solution obviously does the job. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Implementing an 8 bit fixed point register
On 1 Jul, 08:57, nickola <[EMAIL PROTECTED]> wrote: > Hello to all > I'm about to write a simulator for a microcontroller in python > (why python? because I love it!!!) > > but I have a problem. > > The registry of this processor are all 8 bit long (and 10 bit for some > other strange register) > and I need to simulate the fixed point behaviour of the register, > and to access the single bit. > > f.x. (this is a pseudo python session, only for understanding) > > >>> reg1 = fixed_int(8) > >>> reg2 = fixed_int(10) > >>> reg1[0].set() > or > >>> reg1[0] = 1 # or True? how to rapresent a binary bit > >>> reg1[0] > 1 > >>> reg1[1] > 0 > >>> reg1[9] > > >>> reg2 = 0x7FE # in binary > 110 , or 11 bit long > >>> reg2 > > 0x7FE > #or 10, the memorization truncate the upper bits ( or perhaps > generate an exception?)>>> reg2 += 0x02 #the result is 100, again not > contained in 10 bit > >>> reg2 > > 0x00 > # truncated again>>> myprocessor.flags['z'] > > 1 > # or True? Z flag indicated an overflow in arithmetic operations > > Is possibile to do so in python? > > thanks I am not sure if it is exactly what you are looking for (it may be a bit low level), but myHDL may be interesting. It provides low level H/ W simulation capabilities. http://myhdl.jandecaluwe.com/doku.php Steven -- http://mail.python.org/mailman/listinfo/python-list
Beginner Question : Iterators and zip
Hi group, I have a basic question on the zip built in function. I am writing a simple text file comparison script, that compares line by line and character by character. The output is the original file, with an X in place of any characters that are different. I have managed a solution for a fixed (3) number of files, but I want a solution of any number of input files. The outline of my solution: for vec in zip(vec_list[0],vec_list[1],vec_list[2]): res = '' for entry in zip(vec[0],vec[1],vec[2]): if len(set(entry)) > 1: res = res+'X' else: res = res+entry[0] outfile.write(res) So vec is a tuple containing a line from each file, and then entry is a tuple containg a character from each line. 2 questions 1) What is the general solution. Using zip in this way looks wrong. Is there another function that does what I want 2) I am using set to remove any repeated characters. Is there a "better" way ? Any other comments/suggestions appreciated. Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Question : Iterators and zip
On 12 Jul, 21:50, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > On 12 juil, 20:55, [EMAIL PROTECTED] wrote: > > > > zip is (mostly) ok. What you're missing is how to use it for any > arbitrary number of sequences. Try this instead: > > >>> lists = [range(5), range(5,11), range(11, 16)] > >>> lists > > [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9, 10], [11, 12, 13, 14, 15]]>>> for item in > zip(*lists): > > ... print item > ... > (0, 5, 11) > (1, 6, 12) > (2, 7, 13) > (3, 8, 14) > (4, 9, 15) What is this *lis operation called? I am having trouble finding any reference to it in the python docs or the book learning python. > > Any other comments/suggestions appreciated. > > There's a difflib package in the standard lib. Did you give it a try ? I'll check it out, but I am a newbie, so I am writing this as a (useful) learning excercise. Thanks for the help Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Question : Iterators and zip
On 13 Jul, 19:49, Terry Reedy <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: > > What is this *lis operation called? I am having trouble finding any > > reference to it in the python docs or the book learning python. > > One might call this argument unpacking, but > Language Manual / Expressions / Primaries / Calls > simply calls it *expression syntax. > "If the syntax *expression appears in the function call, expression must > evaluate to a sequence. Elements from this sequence are treated as if > they were additional positional arguments; if there are positional > arguments x1,...,*xN* , and expression evaluates to a sequence > y1,...,*yM*, this is equivalent to a call with M+N positional arguments > x1,...,*xN*,*y1*,...,*yM*." > > See Compound Statements / Function definitions for the mirror syntax in > definitions. > > tjr Thanks, It's starting to make sense :-) Steven -- http://mail.python.org/mailman/listinfo/python-list
noob: subprocess clarification
Hi, I generally use csh scripts for generally scripting (controlling simulations). Basically the script processing options, generates the command line, executes it and then processes the results. I would usually use the -f option on the shebang line to ensure that the environment from the current shell is used. i.e. #!/bin/csh -f How do I acheive this in python? I have been looking at the import subprocess as sub p = sub.Popen(['echo $PATH'],shell=True).wait() This *seems* to do what I want (in that the path looks correct), but I don't really understand the documentation. Can somebody please clarify what the shell=True does, and whether I am using it correctly. Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: noob: subprocess clarification
On 14 Sep, 22:06, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote: > On Sun, 14 Sep 2008 02:29:52 -0700 (PDT), [EMAIL PROTECTED] declaimed > the following in comp.lang.python: > > > Can somebody please clarify what the shell=True does, and whether I am > > using it correctly. > > What part of: > > """ > On Unix, with shell=False (default): In this case, the Popen class uses > os.execvp() to execute the child program. args should normally be a > sequence. A string will be treated as a sequence with the string as the > only item (the program to execute). > > On Unix, with shell=True: If args is a string, it specifies the command > string to execute through the shell. If args is a sequence, the first > item specifies the command string, and any additional items will be > treated as additional shell arguments. > """ > > is giving problems? I assume that this is sarchasm :-) > > For the most part, as I recall, "shell=True" allows you to invoke > commands that are built-in/native to the default shell, or even a shell > script. False requires the specified command to be a stand-alone > executable program. > -- -- http://mail.python.org/mailman/listinfo/python-list
Standard Delay Format (SDF) Parsing
Hi, A Standard Delay Format (SDF) file is a text file used to store delay information, and is used in ASIC design. I need to parse some of these files and before I re-invent the wheel, is anyone aware of a module to allow parsing of these files? Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Popen Question
Hi, I usually use csh for my simulation control scripts, but these scripts are becoming more complex, so I plan to use python for the next project. To this end, I am looking at subprocess.Popen() to actually call the simulations, and have a very basic question which is demonstrated below. [sde:st...@lbux03 ~]$ python Python 2.6 (r26:66714, Feb 21 2009, 02:16:04) [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os, subprocess >>> os.environ['MYVAR'] = "myval" >>> p = subprocess.Popen(['echo', '$MYVAR'],shell=True) >>> >>> p = subprocess.Popen(['echo', '$MYVAR']) >>> $MYVAR >>> p = subprocess.Popen('echo $MYVAR',shell=True) >>> myval >>> p = subprocess.Popen('echo $MYVAR') Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1106, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory I am not really sure I understand these results. 1) No idea what is going on 2) As (1). What isn't myval printed out (rather than $MYVAR) 3) Works as I wanted it to 4) Why do I need shell=True ? The documentation isn't very clear to me (it seems you need to understand the underlying system calls). Can anyone explain (or provide link) for this behaviour in simple English? Thanks, Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
Hi, Thanks everyone for the replies - it is now clearer. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Brainstorming on recursive class definitions
On Tuesday, September 12, 2017 at 5:37:31 PM UTC+2, Johannes Bauer wrote: > Hi group, > > so I'm having a problem that I'd like to solve *nicely*. I know plenty > of ways to solve it, but am curious if there's a solution that allows me > to write the solution in a way that is most comfortable for the user. > > I'm trying to map registers of a processor. So assume you have a n bit > address space, the processor might have duplicate units of identical > functionality mapped at different places in memory. For example, assume > there's a GPIO unit that has registers FOO, BAR and KOO and two GPIO > ports GPIOA and GPIOB. I'd like to write code along the lines of this: > > class GpioMap(BaseRegisterMap): > FOO = 0x0 > BAR = 0x4 > KOO = 0x8 > > class CPURegisterMap(BaseRegisterMap): > GPIOA = GpioMap(0x1) > GPIOB = GpioMap(0x2) > > cpu = CPURegisterMap(0x8000) > > assert(cpu.addr == 0x8000) > assert(cpu.GPIOA.addr == 0x8000 + 0x1) > assert(cpu.GPIOB.addr == 0x8000 + 0x2) > assert(cpu.GPIOA.FOO.addr == 0x8000 + 0x1 + 0x0) > assert(cpu.GPIOA.KOO.addr == 0x8000 + 0x1 + 0x8) > assert(cpu.GPIOB.BAR.addr == 0x8000 + 0x2 + 0x4) > > So, obviously, FOO, BAR and KOO are of type "int" without any "addr" > property, so there would need to be some magic there. Additionally, > through some way the instanciation of GpioMap() would need the knowledge > of its parent base, which I'm not sure is even possible. Maybe (that's > what I'm currently trying to get right) the __getattribute__ would > propagate the information about the accumulated parent's base address to > the child during lookup. > > Anyways, I'm looking for your ideas on how to solve such a thing > "nicely". Note that "BaseRegisterMap" is allowed to do dirty things as > long as the definition code has a clean look & feel. > > Cheers, > Joe > > > -- > >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > > Zumindest nicht öffentlich! > Ah, der neueste und bis heute genialste Streich unsere großen > Kosmologen: Die Geheim-Vorhersage. > - Karl Kaos über Rüdiger Thomas in dsa A child (e.g. instance of GpioMap) should not have any knowledge of it's own base address. This is a function of the GPIORegisyerMap only. i.e. The GPIORegisyerMap would do the selection of particular instance. It may be worth looking at some of the IP-XACT documentation, which defines some common terms (IIRC register, address map, address space etc). Having a consistent definition here will definitely help moving forward Although it may seem OTT at the moment, it is something you should consider at the outset. e.g. A single GPIO register may occur at two different addresses, depending on the bus master (CPU Core, Debugger, External SPI access). Steven -- https://mail.python.org/mailman/listinfo/python-list