EuroPython 2017: Early-bird Tickets now on sale!
Interested in attending EuroPython? Entry tickets are now on sale and available on our website. *** Buy your EuroPython ticket *** https://ep2017.europython.eu/en/registration/buy-tickets/ Tickets for EuroPython will be sold in three phases: First, we’ll have a short early-bird ticket phase, where we’ll sell tickets at a very low rate. Only 200 tickets will be available for this rate, so be quick. It usually takes just a few days for them to sell out. We’ll then switch to the regular rates, and closer to the conference, to the on-desk rates. You can get tickets for the whole week, if you don’t want to miss anything, or buy day passes at the on-desk rates in July to attend only one or two days. We further offer three different rates: ‘students’ (including PhD and postdoc researchers), ‘personal’ and ‘business’ passes (for companies). Speakers and trainers will benefit of special discounts. See Call for Proposals for further details: https://ep2017.europython.eu/en/call-for-proposals/ Take this opportunity and get your ticket now! Prices will increase the closer we get to the event. Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/849177756873224193 Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: Improve Python + Influxdb import performance
Hi Inada
Thank you for your response
Weblogic works on Jython 2.2.1
and I think the Python requests module requires >= python 2.6
Please correct me if I'm wrong
or if there is another way to get this to work
Thanks
P
On Monday, April 3, 2017 at 9:52:38 PM UTC+5:30, INADA Naoki wrote:
> You can reuse connection, instead of creating for each request. (HTTP
> keep-alive).
>
> On Tue, Apr 4, 2017 at 1:11 AM, Prathamesh
> wrote:
> > Hello World
> >
> > The following script is an extract from
> >
> > https://github.com/RittmanMead/obi-metrics-agent/blob/master/obi-metrics-agent.py
> >
> > <>
> >
> > import calendar, time
> > import sys
> > import getopt
> >
> > print '---'
> >
> > # Check the arguments to this script are as expected.
> > # argv[0] is script name.
> > argLen = len(sys.argv)
> > if argLen -1 < 2:
> > print "ERROR: got ", argLen -1, " args, must be at least two."
> > print '$FMW_HOME/oracle_common/common/bin/wlst.sh obi-metrics-agent.py
> > [] []
> > [] [] [targetDB influx db>'
> > exit()
> >
> > outputFormat='CSV'
> > url='t3://localhost:7001'
> > targetHost='localhost'
> > targetDB='obi'
> > targetPort='8086'
> >
> > try:
> > wls_user = sys.argv[1]
> > wls_pw = sys.argv[2]
> > url = sys.argv[3]
> > outputFormat=sys.argv[4]
> > targetHost=sys.argv[5]
> > targetPort=sys.argv[6]
> > targetDB=sys.argv[7]
> > except:
> > print ''
> >
> > print wls_user, wls_pw,url, outputFormat,targetHost,targetPort,targetDB
> >
> > now_epoch = calendar.timegm(time.gmtime())*1000
> >
> > if outputFormat=='InfluxDB':
> > import httplib
> > influx_msgs=''
> >
> > connect(wls_user,wls_pw,url)
> > results = displayMetricTables('Oracle_BI*','dms_cProcessInfo')
> > for table in results:
> > tableName = table.get('Table')
> > rows = table.get('Rows')
> > rowCollection = rows.values()
> > iter = rowCollection.iterator()
> > while iter.hasNext():
> > row = iter.next()
> > rowType = row.getCompositeType()
> > keys = rowType.keySet()
> > keyIter = keys.iterator()
> > inst_name= row.get('Name').replace(' ','-')
> > try:
> > server= row.get('Servername').replace(' ','-').replace('/','_')
> > except:
> > try:
> > server= row.get('ServerName').replace('
> > ','-').replace('/','_')
> > except:
> > server='unknown'
> > try:
> > host= row.get('Host').replace(' ','-')
> > except:
> > host=''
> > while keyIter.hasNext():
> > columnName = keyIter.next()
> > value = row.get(columnName )
> > if columnName.find('.value')>0:
> > metric_name=columnName.replace('.value','')
> > if value is not None:
> > if value != 0:
> > if outputFormat=='InfluxDB':
> > influx_msg=
> > ('%s,server=%s,host=%s,metric_group=%s,metric_instance=%s value=%s %s') %
> > (metric_name,server,host,tableName,inst_name, value,now_epoch*100)
> > influx_msgs+='\n%s' % influx_msg
> > conn = httplib.HTTPConnection('%s:%s' %
> > (targetHost,targetPort))
> > ## TODO pretty sure should be urlencoding this
> > ...
> > a=conn.request("POST", ("/write?db=%s" %
> > targetDB), influx_msg)
> > r=conn.getresponse()
> >
> > <>
> >
> > It currently takes about 3 minutes to execute completely and I was thinking
> > of a way to make it run faster
> >
> > Data alignment (Influx line protocol) & data loading - done together takes
> > up most of the time
> >
> > Influxdb is currently loading data at around 3 points/second
> >
> > Any way to align the data separately, store it and load it as a batch?
> >
> > I feel that would help improve performance
> >
> > Please let me know if you have any pointers
> > I can send the data sheet if required
> >
> > Thanks P
> > --
> > https://mail.python.org/mailman/listinfo/python-list
On Monday, April 3, 2017 at 9:52:38 PM UTC+5:30, INADA Naoki wrote:
> You can reuse connection, instead of creating for each request. (HTTP
> keep-alive).
>
> On Tue, Apr 4, 2017 at 1:11 AM, Prathamesh
> wrote:
> > Hello World
> >
> > The following script is an extract from
> >
> > https://github.com/RittmanMead/obi-metrics-agent/blob/master/obi-metrics-agent.py
> >
> > <>
> >
> > import calendar, time
> > import sys
> > import getopt
> >
> > print '---'
> >
> > # Check the arguments to this script are as expected.
> > # argv[0] is script name.
> > argLen = len(sys.argv)
> > if argLen -1 < 2:
> > print "ERROR: got ", argLen -1, " args, must be at least two."
> > print '$FMW_HOME/oracle_common/common/bin/wlst.sh obi-metrics-agent.py
> > [] []
> > []
Re: Text-mode apps (Was :Who are the "spacists"?)
On Saturday, 1 April 2017 17:01:03 UTC+1, Steve D'Aprano wrote: > On Sat, 1 Apr 2017 12:17 pm, Rick Johnson wrote: > > > Most people just quietly change the filename and move on > > > There are over a billion people in China, almost a billion more in India, > about 140 million people in Russia, nearly 130 million people in Japan, 250 > million in Indonesia, about 290 million native Arabic speakers, and even 9 > million speakers of Hebrew. > > If you think that all these people, and hundreds of millions more, will > just "quietly change the filename" to ASCII because you're too lazy, > self-centred and arrogant to move on from 1963, then you truly are an > example of the Ugly American. > Try to get your cliches straight. The "Ugly American" was the good guy, in contrast to the good-looking, smooth-talking State Department types who were the baddies. But hey, experience tells us that anyone ugly must automatically be wicked, not so? Will -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/03/2017 11:31 PM, dieter wrote: > Dave writes: > >> I created a python program that gets data from a user, stores the data >> as a dictionary in a list of dictionaries. When the program quits, it >> saves the data file. My desire is to append the new data to the >> existing data file as is done with purely text files. > > Usually, you cannot do that: > "JSON" stands for "JavaScript Object Notation": That's assuming he's using JSON; he never specified what he's using to represent data as plain text. Though I suspect you're correct, for all we know he could just be writing data using his own text representation or writing to an ini file. And in his case it sounds like JSON is not an ideal method for saving his data since he's wanting to only append data, not read it in his program. In the future, Dave, please provide all the information pertaining to the problem so we can give accurate advice. Don't make us guess or assume we can all infer this information. -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
Michael Torrie writes: > On 04/03/2017 11:31 PM, dieter wrote: >> Dave writes: >> >>> I created a python program that gets data from a user, stores the data >>> as a dictionary in a list of dictionaries. When the program quits, it >>> saves the data file. My desire is to append the new data to the >>> existing data file as is done with purely text files. >> >> Usually, you cannot do that: >> "JSON" stands for "JavaScript Object Notation": > > That's assuming he's using JSON; he never specified what he's using to > represent data as plain text. Though I suspect you're correct, for all > we know he could just be writing data using his own text representation > or writing to an ini file. And in his case it sounds like JSON is not > an ideal method for saving his data since he's wanting to only append > data, not read it in his program. > > In the future, Dave, please provide all the information pertaining to > the problem so we can give accurate advice. Don't make us guess or > assume we can all infer this information. The clue is on the subject line. -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/04/2017 08:19 AM, Jussi Piitulainen wrote: > The clue is on the subject line. Ahh, so it is. -- https://mail.python.org/mailman/listinfo/python-list
Data transmission from Python script to bash script
Hello All, I am writing a python code for processing a data obtained from a sensor. The data from sensor is obtained by executing a python script. The data obtained should be further given to another python module where the received data is used for adjusting the location of an object. For achieving this, there is a central bash script, which runs both the python modules parallel. Something like: python a.py & python b.py & I am trying to return the sensor data to the bash .sh file, therefore it can be provided to the other script. This, based on the online tutorials: sensor_data=$(python execute_sensor_process.py) & and the sensor_data is assigned by printing the required data in the corresponding python script. For example, the data is printed in a.py as follows: print >>sys.stderr,sens_data By printing the data onto sys.stderr and assigning a return variable in the bash, I am expecting the data to be assigned. But this is not happening. The sensor data is a dictionary and I like to have this data for further analysis. Can someone help me to understand why the code is not working? I tried other approaches of function call such as sensor_data=$`python execute_sensor_process.py` & #python a.py tempfile.txt & kinexon_data=`cat tempfile.txt` & But none of the approaches are working. Thank you, Venkatachalam Srinivasan -- https://mail.python.org/mailman/listinfo/python-list
Data exchange between python script and bash script
Hello All, I am writing a python code for processing a data obtained from a sensor. The data from sensor is obtained by executing a python script. The data obtained should be further given to another python module where the received data is used for adjusting the location of an object. For achieving this, there is a central bash script, which runs both the python modules parallel. Something like: python a.py & python b.py & I am trying to return the sensor data to the bash .sh file, therefore it can be provided to the other script. This, based on the online tutorials looks like: sensor_data=$(python execute_sensor_process.py) & and the sensor_data is assigned by printing the required data in the corresponding python script. For example, the data is printed in execute_sensor_process.py as follows: print >>sys.stderr,sens_data By printing the data onto sys.stderr and assigning a return variable in the bash, I am expecting the data to be assigned. But this is not happening. The sensor data is a dictionary and I like to have this data for further analysis. I am not getting the data returned from the python script on to the bash variable. Can someone help me to understand why the code is not working? I tried other approaches of function call such as sensor_data=$`python execute_sensor_process.py` & python execute_sensor_process.py tempfile.txt & kinexon_data=`cat tempfile.txt` & But none of the approaches are working. Thank you, Venkatachalam Srinivasan -- https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On Tue, 04 Apr 2017 08:01:42 -0700, venkatachalam.19 wrote: > Hello All, > > I am writing a python code for processing a data obtained from a sensor. The > data from sensor is obtained by executing a python script. The data obtained > should be further given to another python module where the received data is > used for adjusting the location of an object. > > For achieving this, there is a central bash script, which runs both the > python modules parallel. Something like: > > python a.py & > python b.py & What is going on that two python scripts are needed? Which one generates the data needed by the bash script? > I am trying to return the sensor data to the bash .sh file, therefore it can > be provided to the other script. This, based on the online tutorials looks > like: > > sensor_data=$(python execute_sensor_process.py) & Presumably is simply getting the exit status code from the python interpreter, not the data, right? What are you seeing? > and the sensor_data is assigned by printing the required data in the > corresponding python script. For example, the data is printed in > execute_sensor_process.py as follows: > > print >>sys.stderr,sens_data > > By printing the data onto sys.stderr and assigning a return variable in the > bash, I am expecting the data to be assigned. Assigned to what? Some return variable in bash? What?? Why not use stdout? Either pipe the data from python directly into a (possibly modified) bash script, or into a file which gets read by the bash script. > But this is not happening. The sensor data is a dictionary and I like to have > this data for further analysis. I am not getting the data returned from the > python script on to the bash variable. Bash doesn't have dictionaries like python. Why is bash needed? > Can someone help me to understand why the code is not working? I tried other > approaches of function call such as You haven't given us enough of the code to really answer. > sensor_data=$`python execute_sensor_process.py` & > > python execute_sensor_process.py tempfile.txt & > kinexon_data=`cat tempfile.txt` & > > But none of the approaches are working. > > Thank you, > Venkatachalam Srinivasan I wonder if you could completely eliminate the bash script - do it all in python. I've written quite a few bash scripts, but not so many since I started using python. Only exception is for low level functions on systems without a functioning python. -- https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On Tue, Apr 4, 2017 at 8:01 AM, wrote: > Hello All, > > I am writing a python code for processing a data obtained from a sensor. > The data from sensor is obtained by executing a python script. The data > obtained should be further given to another python module where the > received data is used for adjusting the location of an object. > > For achieving this, there is a central bash script, which runs both the > python modules parallel. Something like: > > python a.py & > python b.py & > > I am trying to return the sensor data to the bash .sh file, therefore it > can be provided to the other script. This, based on the online tutorials > looks like: > > sensor_data=$(python execute_sensor_process.py) & > > and the sensor_data is assigned by printing the required data in the > corresponding python script. For example, the data is printed in > execute_sensor_process.py as follows: > > print >>sys.stderr,sens_data > > By printing the data onto sys.stderr and assigning a return variable in > the bash, I am expecting the data to be assigned. > > But this is not happening. The sensor data is a dictionary and I like to > have this data for further analysis. I am not getting the data returned > from the python script on to the bash variable. > > Can someone help me to understand why the code is not working? I tried > other approaches of function call such as > > > sensor_data=$`python execute_sensor_process.py` & > > > python execute_sensor_process.py tempfile.txt & > kinexon_data=`cat tempfile.txt` & > > But none of the approaches are working. > > Thank you, > Venkatachalam Srinivasan > -- > https://mail.python.org/mailman/listinfo/python-list > I'm not sure why you need the data in a bash script. What can you do with a bash script that you can't do with Python? That said, it seems to me that you need a data persistence layer, i.e. a way to store the dictionary after it is created. My best advice would be to use the json module to write out to json files. I believe this is the best approach because json has a near 1-to-1 mapping with Python dictionaries. Json is also a good choice because almost every language has a json parser as part of it's standard library, making you data extremely portable. -- https://mail.python.org/mailman/listinfo/python-list
Re: Basic Nested Dictionary in a Loop
Thanks Peter , Terry and others ! On Tue, Apr 4, 2017 at 12:11 PM, Peter Otten <__pete...@web.de> wrote: > Ganesh Pal wrote: > > >> > >> > >> Whenever you feel the urge to write range(len(whatever)) -- resist that > >> temptation, and you'll end up with better Python code ;) > >> > >> > > Thanks for this suggestion but for my better understanding can explain > > this further even Steve did point the same mistake. > > It's not directly a mistake, it's just that programming against a smaller > interface makes your code more flexible and more readable. > > Readability: > > items = [...] > for i in range(len(items)): > ... > > Are list entries replaced? We cannot know without inspecting the for suite. > > items = [...] > for item in items: > ... > > Are list entries replaced? No. > > Flexibility: > > def give_raise(employees, amount): > for i in range(len(items)): > employee = employees[i] > employee.payment += amount > > employees = [...] > give_raise(employees, 10) > > Now let's adapt the code to raise the payment of some employees: > > male_employees = [ > employees[i] for i in range(len(employees)) > if e.sex == "male" > ] > give_raise(male_emplyees, 10) > > We need a helper list, basically because there is a len() call inside the > give_raise() function. If we rewrite the function in an idiomatic way > > def give_raise(employees, amount): > for employee in employees: > employee.payment += amount > > it will become not just more concise, the calling code no longer needs to > build the intermediate list. Instead we save the extra memory and pass a > generator expression: > > give_raise((e for e in employees if e.sex == "female"), 10) > > In a similar way you can write code that iterates over data that is lazily > read into memory, like lines in a file > > # wrong, reads the whole file into memory > lines = open(...).readlines() > for i in range(len(lines)): > process(lines[i]) > > # idiomatic, read the lines as needed > for line in open(...): > process(line) > > or records from a database. > > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
Am 04.04.2017 um 04:59 schrieb Dave: I created a python program that gets data from a user, stores the data as a dictionary in a list of dictionaries. When the program quits, it saves the data file. My desire is to append the new data to the existing data file as is done with purely text files. However, I can't find a way to do that. The advice I have seen on the web is to load the data when the program starts, append the new user input to the list, then re-write the data file. Is that the best way, or is there a better way? As explained by others JSON does not allow text based data appending. But maybe you take a look at YAML (if it's appropriate for your project). -- Robin Koch -- https://mail.python.org/mailman/listinfo/python-list
Optimize this code further
Dear friends.
I am trying optimize the below sample code any suggestion on this please
let me know
yy-1# cat file1.py
#! /usr/bin/python
from __future__ import absolute_import, division, print_function
import os
import sys
import logging
logging.basicConfig(stream=sys.stderr, level=logging.INFO)
logger = logging.getLogger(__name__)
LOG_DIR = '/var/log/frobnitz'
def createTestCases(LOG_DIR):
'''create a test case data dictionary with parameters'''
d = dict()
d['test01'] = dict(object="inode", offset="18", size="4", optype="set")
lin_02 = "something"
d['test02'] = dict(object="lin", lin=lin_02, offset="18", size="5",
optype="set")
return addLogFilename(d, LOG_DIR)
def run_tool(logfile, **kw):
logger.info('%s would execute with %r', logfile, kw)
def addLogFilename(d, logdir):
'''put the logfile name into the test case data dictionary'''
for casename, args in d.items():
args['logfile'] = os.path.join(logdir, casename + '.log')
return d
def main():
testcases = createTestCases(LOG_DIR)
get_baddr = dict()
for casename, kw in testcases.items():
# -- yank the logfile name out of the dictionary, before calling
func
logfile = kw.pop('logfile')
get_baddr[casename] = run_tool(logfile, **kw)
if __name__ == '__main__':
main()
# -- end of file
yy-1# python file1.py
INFO:__main__:/var/log/frobnitz/test02.log would execute with {'size': '5',
'lin': 'something', 'optype': 'set', 'object': 'lin', 'offset': '18'}
INFO:__main__:/var/log/frobnitz/test01.log would execute with {'object':
'inode', 'optype': 'set', 'offset': '18', 'size': '4'}
1. I now need to have sub tests for every test i.e , so in the above
program I will now need to store logs in the below format say for
/var/log/frobnitz/Test_inode_01/ test01.log and
/var/log/frobnitz/Test_inode_01/ test02.log .. etc
2. Everything else remains same and I am supposed to retain the same
directory structure and logging
Here is what I plan to do :
1. Use dictionary and Change testcases variable as dictionary and let
its key store the subtest cases , so we have a nested dictionary
testcase = {}
testcase[‘Test_inode_1’] = createTestCases(LOG_DIR)
2.Since I need to retain the same logging directory structure , maybe
I should try to get the key first and then append it to the log file
Question :
1. Any other simple tips and tricks to solve the problem , I am a Linux
user using python 2.7
Regards,
Ganesh
--
https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On Tuesday, April 4, 2017 at 5:27:42 PM UTC+2, cassiope wrote: > On Tue, 04 Apr 2017 08:01:42 -0700, venkatachalam.19 wrote: > > > Hello All, > > > > I am writing a python code for processing a data obtained from a sensor. > > The data from sensor is obtained by executing a python script. The data > > obtained should be further given to another python module where the > > received data is used for adjusting the location of an object. > > > > For achieving this, there is a central bash script, which runs both the > > python modules parallel. Something like: > > > > python a.py & > > python b.py & > > What is going on that two python scripts are needed? Which one generates the > data needed by the bash script? One python script communicates with an external sensor and receives data. Another script controls a robot. That is based on ROS modules. >From the given code, a.py will be the script generating the data. Both the >modules are independent of each other and so I have written both of them in >different python scripts. > > > I am trying to return the sensor data to the bash .sh file, therefore it > > can be provided to the other script. This, based on the online tutorials > > looks like: > > > > sensor_data=$(python execute_sensor_process.py) & > > Presumably is simply getting the exit status code from the > python interpreter, not the data, right? > What are you seeing? > I am not getting the exit status (I am not setting or printing return status > in the python script). I need both the scripts running in parallel. Meaning > the sensor will be continuously monitoring the location of the robot and this > data will be used in another python script which controls the robot motion. For now, I am printing the output in the script and as far I understood, this can be assigned to a variable in the bash file from where this file is executed. I am not getting any output in the assigned variable in bash. > > and the sensor_data is assigned by printing the required data in the > > corresponding python script. For example, the data is printed in > > execute_sensor_process.py as follows: > > > > print >>sys.stderr,sens_data > > > > By printing the data onto sys.stderr and assigning a return variable in the > > bash, I am expecting the data to be assigned. > > Assigned to what? Some return variable in bash? What?? > Why not use stdout? Either pipe the data from python directly into a > (possibly modified) bash script, > or into a file which gets read by the bash script. > Yes. As shown in "sensor_data=$(python execute_sensor_process.py) &", the variable sensor_data is assigned to hold the return value from the python script. I was using the stdout. But in some forums for similar problems, it is suggested to print the data to stderr (I really dont know what is the significance). I am unaware of piping the data from python. I will look into this. > > But this is not happening. The sensor data is a dictionary and I like to > > have this data for further analysis. I am not getting the data returned > > from the python script on to the bash variable. > > Bash doesn't have dictionaries like python. Why is bash needed? > Bash is needed to run the different scripts. I tried using multiprocess and thread from python, but really they are not performing parallel processing, when there are completely two different process. I understood it by executing the multiprocess/thread. Sequential operation was happening, which lead to the situation that only one python process is always executed. > > Can someone help me to understand why the code is not working? I tried > > other approaches of function call such as > > You haven't given us enough of the code to really answer. > > > sensor_data=$`python execute_sensor_process.py` & > > > > python execute_sensor_process.py tempfile.txt & > > kinexon_data=`cat tempfile.txt` & > > > > But none of the approaches are working. > > > > Thank you, > > Venkatachalam Srinivasan > > I wonder if you could completely eliminate the bash script - do it all in > python. > I've written quite a few bash scripts, but not so many since I started using > python. Only exception is for low level functions on systems without a > functioning > python. -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/04/2017 10:17 AM, Michael Torrie wrote: On 04/03/2017 11:31 PM, dieter wrote: Dave writes: I created a python program that gets data from a user, stores the data as a dictionary in a list of dictionaries. When the program quits, it saves the data file. My desire is to append the new data to the existing data file as is done with purely text files. Usually, you cannot do that: "JSON" stands for "JavaScript Object Notation": That's assuming he's using JSON; he never specified what he's using to represent data as plain text. Though I suspect you're correct, for all we know he could just be writing data using his own text representation or writing to an ini file. And in his case it sounds like JSON is not an ideal method for saving his data since he's wanting to only append data, not read it in his program. In the future, Dave, please provide all the information pertaining to the problem so we can give accurate advice. Don't make us guess or assume we can all infer this information. I did. Please see the subject - it is json, not some bastardized version of it. While I didn't explicitly say that the data is read back in, it is - when the program starts. The user can enter more data, then save the data to the list (append). Then, like any other program, they can save the data to file. I don't care for the idea of replacing the data file for every save. My preference would to append to the existing data file - makes more sense. However, that is not how json works. So, I'm considering other alternatives for a data file structure. Paired data (key and value) is not really required, but my feeling is that it makes the data file more robust. A database seems a little over the top for this situation since there is no querying of data - just load and save. Dave, -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
Dave writes: > I don't care for the idea of replacing the data file for every save. This is the simplest implementation. It works. Can you say why you don't care for it? > My preference would to append to the existing data file - makes more > sense. Can you expand on that? You haven't defined how that would satisfy the requirement, so it isn't clear to me why a more complex method “makes more sense” than the simple way which works now. -- \ “Holy hole in a donut, Batman!” —Robin | `\ | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On Tuesday, April 4, 2017 at 5:42:23 PM UTC+2, justin walters wrote: > On Tue, Apr 4, 2017 at 8:01 AM, wrote: > > > Hello All, > > > > I am writing a python code for processing a data obtained from a sensor. > > The data from sensor is obtained by executing a python script. The data > > obtained should be further given to another python module where the > > received data is used for adjusting the location of an object. > > > > For achieving this, there is a central bash script, which runs both the > > python modules parallel. Something like: > > > > python a.py & > > python b.py & > > > > I am trying to return the sensor data to the bash .sh file, therefore it > > can be provided to the other script. This, based on the online tutorials > > looks like: > > > > sensor_data=$(python execute_sensor_process.py) & > > > > and the sensor_data is assigned by printing the required data in the > > corresponding python script. For example, the data is printed in > > execute_sensor_process.py as follows: > > > > print >>sys.stderr,sens_data > > > > By printing the data onto sys.stderr and assigning a return variable in > > the bash, I am expecting the data to be assigned. > > > > But this is not happening. The sensor data is a dictionary and I like to > > have this data for further analysis. I am not getting the data returned > > from the python script on to the bash variable. > > > > Can someone help me to understand why the code is not working? I tried > > other approaches of function call such as > > > > > > sensor_data=$`python execute_sensor_process.py` & > > > > > > python execute_sensor_process.py tempfile.txt & > > kinexon_data=`cat tempfile.txt` & > > > > But none of the approaches are working. > > > > Thank you, > > Venkatachalam Srinivasan > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > I'm not sure why you need the data in a bash script. What can you do with a > bash script that you > can't do with Python? > > That said, it seems to me that you need a data persistence layer, i.e. a > way to store the dictionary > after it is created. My best advice would be to use the json module to > write out to json files. I > believe this is the best approach because json has a near 1-to-1 mapping > with Python > dictionaries. > > Json is also a good choice because almost every language has a json parser > as part of it's standard > library, making you data extremely portable. Hi, Thanks for the answer. I need bash for connecting data exchange between two python scripts. To be more specific, data from one script has to be passed to the another script. You are right, I don't need the data in the bash other than to pass the obtained data to the another script which uses the data for further analysis. Regarding using the json file, I am new to this. A naive question is that if the data is too large then is json file is easy to handle? Is json file for large data is not computationally expensive? I am not using textfile for the same reason of being computationally expensive. Thanks, Venkatachalam Srinivasan -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/04/2017 10:23 AM, Dave wrote: I don't care for the idea of replacing the data file for every save. My preference would to append to the existing data file - makes more sense. However, that is not how json works. So, I'm considering other alternatives for a data file structure. Paired data (key and value) is not really required, but my feeling is that it makes the data file more robust. A database seems a little over the top for this situation since there is no querying of data - just load and save. Dave, Valid. That said, the database implementation (especially if you just use Sqlite) is written, done, and works. There are reasons that a lot of programs use it just to store things like user settings. Having a backend that's already ACID takes a lot of stress out of things like what I'm assuming you don't like about full data file replacement: what happens if you get a powerfail midway through. I'm not saying it's the right answer, but it's not nearly so much flyswatting with a sledgehammer as you may think. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On 04/04/17 18:07, Venkatachalam Srinivasan wrote: On Tuesday, April 4, 2017 at 5:27:42 PM UTC+2, cassiope wrote: On Tue, 04 Apr 2017 08:01:42 -0700, venkatachalam.19 wrote: Hello All, I am writing a python code for processing a data obtained from a sensor. The data from sensor is obtained by executing a python script. The data obtained should be further given to another python module where the received data is used for adjusting the location of an object. For achieving this, there is a central bash script, which runs both the python modules parallel. Something like: python a.py & python b.py & What is going on that two python scripts are needed? Which one generates the data needed by the bash script? One python script communicates with an external sensor and receives data. Another script controls a robot. That is based on ROS modules. From the given code, a.py will be the script generating the data. Both the modules are independent [snip] Except that they clearly aren't, because the operation of one depends on the results of the other. You may need to think harder about that. It would appear that your main problem is that you don't understand bash. This isn't a Python issue at all, it's about how the shell runs programs and gets results back. We aren't a Bash programming newsgroup (or mailing list), and honestly even if we were it seems like you need to absorb a lot of basic concepts. You will be much better off finding and reading a decent shell scripting tutorial. Pay particular attention to piping, which appears to be what you are trying to reinvent. If all else fails, Read The Fine Manual; it's not the easiest of reads, but it does contain all the information you need. -- Rhodri James *-* Kynesim Ltd -- https://mail.python.org/mailman/listinfo/python-list
Re: Data exchange between python script and bash script
On Tue, Apr 4, 2017 at 10:39 AM, Venkatachalam Srinivasan < venkatachalam...@gmail.com> wrote: > Hi, > > Thanks for the answer. I need bash for connecting data exchange between > two python scripts. To be more specific, data from one script has to be > passed to the another script. You are right, I don't need the data in the > bash other than to pass the obtained data to the another script which uses > the data for further analysis. > > Regarding using the json file, I am new to this. A naive question is that > if the data is too large then is json file is easy to handle? Is json file > for large data is not computationally expensive? I am not using textfile > for the same reason of being computationally expensive. > > Thanks, > Venkatachalam Srinivasan > -- > https://mail.python.org/mailman/listinfo/python-list > It could be expensive to create a dictionary from the json file depending on the amount of data. Alternatively, you could use a Unix socket to transmit the data between processes if you're on a linux or OSx machine. This is probably the best option as you can send the dictionary object itself. This would eliminate the need for the bash script entirely. The first script builds the dictionary object and sends it to the socket. The second script is listening on the socket and receives the dictionary object. Python's standard library has built in support for unix sockets. This means you can actually run two parallel Python instances and communicate between them. Assuming that the first script can build the dictionaries before the second script is done processing them, you will probably need some kind of task queue. Celery: http://www.celeryproject.org/ tends to be a good solution for this problem. In fact, Celery is probably the simplest option now that I think about it. Another alternative is using a SQLite database which Python has built in support for. Should be a bit faster and less memory-intensive. -- https://mail.python.org/mailman/listinfo/python-list
Re: Two variable dictionary comprehension
On 4/3/2017 2:35 AM, Gregory Ewing wrote: Deborah Swanson wrote: Oh, come on. That's a fairly obscure citation in the docs, one that would take a good deal of experience and time reading through them to know was there, Having said that, the index of the Python docs could be improved a bit in this area -- currently it only mentions "list" under "comprehension" (although the page it leads to discusses the other types as well). I am sure that this is because list comprehensions were once thr only comprehensions and that the index was not updated when the syntax and code was. I opened issue 29981 "Update Index for set, dict, and generator 'comprehensions'". -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/04/2017 01:50 PM, Rob Gaddi wrote: On 04/04/2017 10:23 AM, Dave wrote: I don't care for the idea of replacing the data file for every save. My preference would to append to the existing data file - makes more sense. However, that is not how json works. So, I'm considering other alternatives for a data file structure. Paired data (key and value) is not really required, but my feeling is that it makes the data file more robust. A database seems a little over the top for this situation since there is no querying of data - just load and save. Dave, Valid. That said, the database implementation (especially if you just use Sqlite) is written, done, and works. There are reasons that a lot of programs use it just to store things like user settings. Having a backend that's already ACID takes a lot of stress out of things like what I'm assuming you don't like about full data file replacement: what happens if you get a powerfail midway through. I'm not saying it's the right answer, but it's not nearly so much flyswatting with a sledgehammer as you may think. I'll take a look at it. Thanks! Dave, -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 04/04/2017 11:23 AM, Dave wrote: > I did. Please see the subject - it is json, not some bastardized > version of it. My fault for not seeing that! Sorry about the noise. -- https://mail.python.org/mailman/listinfo/python-list
ANN: Wing Python IDE version 6.0.4 released
Hi, We've just released Wing 6.0.4 which fixes remote development to systems with Python 3, addresses problems seen when switching between remote projects and when remote host configurations are missing or invalid, fixes text drag-and-drop, solves problems with analysis of some type annotations, and makes about 30 other minor improvements. For details, see http://wingware.com/pub/wingide/6.0.4/CHANGELOG.txt Wing 6 is the latest major release in Wingware's family of Python IDEs, including Wing Pro, Wing Personal, and Wing 101. Wing 6 adds many new features, introduces a new annual license option for Wing Pro, and makes Wing Personal free. New Features * Improved Multiple Selections: Quickly add selections and edit them all at once * Easy Remote Development: Work seamlessly on remote Linux, OS X, and Raspberry Pi systems * Debugging in the Python Shell: Reach breakpoints and exceptions in (and from) the Python Shell * Recursive Debugging: Debug code invoked in the context of stack frames that are already being debugged * PEP 484 and PEP 526 Type Hinting: Inform Wing's static analysis engine of types it cannot infer * Support for Python 3.6 and Stackless 3.4: Use async and other new language features * Optimized debugger: Run faster, particularly in multi-process and multi-threaded code * Support for OS X full screen mode: Zoom to a virtual screen, with auto-hiding menu bar * Added a new One Dark color palette: Enjoy the best dark display style yet * Updated French and German localizations: Thanks to Jean Sanchez, Laurent Fasnacht, and Christoph Heitkamp For a more detailed overview of new features see the release notice at http://wingware.com/news/2017-04-03 Annual Use License Option Wing 6 adds the option of purchasing a lower-cost expiring annual license for Wing Pro. An annual license includes access to all available Wing Pro versions while it is valid, and then ceases to function if it is now renewed. Pricing for annual licenses is US$ 179/user for Commercial Use and US$ 69/user for Non-Commercial Use. Perpetual licenses for Wing Pro will continue to be available at the same pricing. The cost of extending Support+Upgrades subscriptions on Non-Commercial Use perpetual licenses for Wing Pro has also been dropped from US$ 89 to US$ 39 per user. For details, see https://wingware.com/store/ Wing Personal is Free Wing Personal is now free and no longer requires a license to run. It now also includes the Source Browser, PyLint, and OS Commands tools, and supports the scripting API and Perspectives. However, Wing Personal does not include Wing Pro's advanced editing, debugging, testing and code management features, such as remote development, refactoring, find uses, version control, unit testing, interactive debug probe, multi-process and child process debugging, move program counter, conditional breakpoints, debug watch, framework-specific support (for Jupyter, Django, and others), find symbol in project, and other features. Links Release notice: http://wingware.com/news/2017-04-03 Downloads and Free Trial: http://wingware.com/downloads Buy: http://wingware.com/store/purchase Upgrade: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at supp...@wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 2017-04-04, Ben Finney wrote: > Dave writes: > >> I don't care for the idea of replacing the data file for every save. > > This is the simplest implementation. It works. Can you say why you don't > care for it? After the data is read, it requires reloading the ASR33 with a fresh spool of unpunched paper tape. >> My preference would to append to the existing data file - makes >> more sense. > > Can you expand on that? You haven't defined how that would satisfy the > requirement, so it isn't clear to me why a more complex method “makes > more sense” than the simple way which works now. A big advantage to reading the old file and generating a complete new file is that you can read the existing file, create a new temporary file, and then 'rename(2)' the temporary file to replace the old one. At least on Unix, rename is an filesystem-atomic operation and provides a far smaller window of vulerability for r/w race conditions, crash-induced problems, etc. I assume there's a similar way to do that under Windows... -- Grant Edwards grant.b.edwardsYow! Mary Tyler Moore's at SEVENTH HUSBAND is wearing gmail.commy DACRON TANK TOP in a cheap hotel in HONOLULU! -- https://mail.python.org/mailman/listinfo/python-list
Re: Two variable dictionary comprehension
On 4/3/2017 6:09 PM, Deborah Swanson wrote: Nathan Ernst wrote, on April 03, 2017 1:59 PM I was a bit surprised when I looked at the language reference for 3.6.x. I expected there'd be a direct link to comprehensions, but there's not. ... FWIW, If one was completely new to Python, even knowing the syntax is known as a "comprehension" might be unknown. I certainly didn't know what a comprehension was when I was learning Python. A coworker showed me, some 13 years ago. Thanks Nate, for your comprehension of the plight of many, if not most, newish Python coders. And it certainly doesn't help to ask the list to fill in some of the holes and be met with criticism for asking, but I digress. It is what it is. I have more than once looked at the Reference TOC: Expressions https://docs.python.org/3/reference/index.html and wondered 'where are comprehensions?' (and a few other things). I just opened Issue 29983: "Reference TOC: expand 'Atoms' and 'Primaries'", https://bugs.python.org/issue29983, suggesting that 6.2. Atoms 6.3. Primaries be expanded to 6.2. Atoms, including identifiers, literals, and comprehensions 6.3. Primaries: attributes, subscripts, slices, and calls The 'primaries' list is complete. The full 'atoms' list is too long so I picked what I thought might be the three most important and added 'including' to indicate that this is a selection. There might be opposition (or just +-0 indifference) from developers who are CS language theory wonks who have forgotten how obscure the terms 'atom' and 'primary' are in this context. So anyone supporting this should say so on the issue. https://bugs.python.org/issue29981, adding index entries, should be routine, but at least one +1 would be helpful. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
[ANNOUNCE] Mail Drake version 0.1.7
Howdy all, I am pleased to present Mail Drake (version 0.1.7), now available at PyPI https://pypi.python.org/pypi/maildrake>. Mail Drake is a development tool for testing email traffic. Run Mail Drake’s SMTP server on a local port, send email to it, and inspect the message queue in a local web browser. Mail Drake is free software, via the Affero GPL v3 or later. Documentation in this version is very sparse. The commands installed have help (via the ‘--help’ option) which is the best I can point you to currently. I'd be happy to have feedback, and even happier to have contributors make merge requests. Documentation improvements would be especially worthwhile! -- \ “Of all classes the rich are the most noticed and the least | `\ studied.” —John Kenneth Galbraith, _The Age of Uncertainty_, | _o__) 1977 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Appending data to a json file
On 2017-04-04 21:00, Dennis Lee Bieber wrote:
On Tue, 4 Apr 2017 13:23:52 -0400, Dave declaimed
the following:
I don't care for the idea of replacing the data file for every save. My
preference would to append to the existing data file - makes more sense.
However, that is not how json works. So, I'm considering other
alternatives for a data file structure. Paired data (key and value) is
not really required, but my feeling is that it makes the data file more
robust. A database seems a little over the top for this situation since
there is no querying of data - just load and save.
Somebody must eventually have to retrieve stuff from this file for some
use; otherwise it is nothing but an ever-expanding waste of disk space.
A database, at the very least, means you never have to read the old
contents before adding new data. However, lacking any indication of /what/
the data contains ("dictionary of dictionaries" only indicates a nested
structure at the top level -- but does not indicate if it is possible for
an "append" pass to overwrite a pre-existing subdictionary should the outer
dictionary key be a duplicate).
Possibly a CSV format would suffice...
Would the 'shelve' module module suffice? (OK, it's not plain text, but
neither is a DB.)
--
https://mail.python.org/mailman/listinfo/python-list
RE: Two variable dictionary comprehension
Terry Reedy wrote, on Tuesday, April 04, 2017 11:04 AM > > On 4/3/2017 2:35 AM, Gregory Ewing wrote: > > Deborah Swanson wrote: > > > >> Oh, come on. That's a fairly obscure citation in the docs, > one that > >> would take a good deal of experience and time reading > through them to > >> know was there, > > > Having said that, the index of the Python docs could be > improved a bit > > in this area -- currently it only mentions "list" under > > "comprehension" (although the page it leads to discusses the other > > types as well). > > I am sure that this is because list comprehensions were once thr only > comprehensions and that the index was not updated when the syntax and > code was. I opened issue 29981 "Update Index for set, dict, and > generator 'comprehensions'". > > -- > Terry Jan Reedy Thanks Terry. I does seem strange that there are so many types of comprehensions, yet under 'comprehensions' in the Index, you only see 'list'. And there are no entries for the other types, by their names. Deborah -- https://mail.python.org/mailman/listinfo/python-list
Re: Two variable dictionary comprehension
On 04/04/2017 03:34 PM, Deborah Swanson wrote: Terry Reedy wrote, on Tuesday, April 04, 2017 11:04 AM On 4/3/2017 2:35 AM, Gregory Ewing wrote: Deborah Swanson wrote: Oh, come on. That's a fairly obscure citation in the docs, one that would take a good deal of experience and time reading through them to know was there, Having said that, the index of the Python docs could be improved a bit in this area -- currently it only mentions "list" under "comprehension" (although the page it leads to discusses the other types as well). I am sure that this is because list comprehensions were once thr only comprehensions and that the index was not updated when the syntax and code was. I opened issue 29981 "Update Index for set, dict, and generator 'comprehensions'". -- Terry Jan Reedy Thanks Terry. I does seem strange that there are so many types of comprehensions, yet under 'comprehensions' in the Index, you only see 'list'. And there are no entries for the other types, by their names. Deborah Does it seem... incomprehensible? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
RE: Two variable dictionary comprehension
Sorry I bailed on you last night, but you know it's bad when you can't
read anymore.
I read through this today and saw several things I really need to work
with more, especially dicts, sets, generators and the zip function. I've
used all of them in fairly typical forms, but it's the atypical ones
that throw me. And strangely enough, I've never seen dicts described as
having key:value syntax, although it's certainly true and clarifies a
lot in thinking about them.
The one thing you mentioned that was completely new to me was the use of
an if clause in a list comprehension, and I can see there being times
when that's exactly what you want to do. I'll have to try the other
types of comprehensions and see if this same grammar works in all of
them. It should. Interesting how computer languages seem to be a hybrid
of math and linguistics.
I added all of these concepts and code snippets to my study list. Thanks
again Steve. ;)
Deborah
Steve D'Aprano wrote, on Monday, April 03, 2017 6:05 PM
>
> On Tue, 4 Apr 2017 03:27 am, Deborah Swanson wrote:
>
> > I'll admit that both dictionaries and comprehensions are still a
> > little bit fuzzy to me, especially when I get away from the common
> > usages. This could be a good exercise to clarify some of the fuzzy
> > areas.
>
>
> As far as comprehensions go, how is your mathematics?
>
> If you remember your set builder notation from maths, list
> comprehensions are based on that.
>
> In maths, we say something like:
>
> {2n + 1 : 0 ≤ n ≤ 9}
>
> which might be read aloud as "the set of 2 times n, plus 1,
> such that n is between 0 and 9 inclusive".
>
> http://www.mathwords.com/s/set_builder_notation.htm
>
> A similar notation might be:
>
> {2n + 1 : n ∈ {1, 2, 3}}
>
> said as "the set of 2 times n, plus 1, such that n is an
> element of the set {1, 2, 3}".
>
>
> If you're a maths geek like me, this is second nature :-)
>
>
> Now, Python's list comprehension syntax is based on a similar
> notation, except we spell it out in words rather than
> symbols, using the familiar "for n in ..." syntax from
> for-loops instead of : "such that".
>
> {2*n + 1 for n in range(10)}
>
> {2*n + 1 for n in (1, 2, 3)}
>
>
> A few other differences:
>
> - list comprehensions use [ ] for the outermost brackets;
> - set and dict comprehensions use { };
> - generator expressions use ( ) (unless the parentheses can
> be implied).
>
> We're not restricted to mathematical expressions, and can use
> any Python expression we like:
>
> [value.method(arg)[1] for (value, arg) in zip(values, arguments)]
>
>
> translates roughly into this for-loop:
>
> result = []
> for (value, arg) in zip(values, arguments):
> result.append(value.method(arg)[1])
>
>
>
> Another difference is that comprehensions allow an "if" clause:
>
> [v.method(a)[1] for (v, a) in zip(values, arguments) if v is not None]
>
> translates something like:
>
>
> result = []
> for (v, a) in zip(values, arguments):
> if v is not None:
> result.append(v.method(a)[1])
>
>
> There's more, but that covers probably 98% of comprehension usage.
>
> And of course, remember that dict comprehensions use the same
> key:value syntax as ordinary dict "literals" (the official
> name is "dict display").
>
> result = {}
> for key, value in zip(keys, values):
> result[key] = value
>
>
> becomes
>
> result = {key:value for (key, value) in zip(keys, values)}
>
> although frankly that's better written as:
>
> dict(zip(keys, values))
>
>
>
> Hope this helps!
>
>
>
>
> --
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered
> up, and sure enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Text-mode apps (Was :Who are the "spacists"?)
On 3 April 2017 at 19:55, Steve D'Aprano wrote: > I didn't see you calling out Rick for his prejudice against those who aren't > American, his absurd belief that "most" people are satisfied with ASCII, Hmm... that is interesting. Everyone has some beliefs... If I make a bit more on-topic question: What is "satisfied" with ASCII, or what is "we need Unicode", where and what for? What is your "satisfaction" with Unicode? For me it is a font, where images given numbers, ok, then we need obviosly to look at it, e.g. if you make a codec. So you are just stating the fact that font files store them in this order and a txt file in Notepad will show them in correct order? hmm... I just thought you have some idea, e.g. what glyphs you adore esthetically or find especially useful in your life. Which of some 40 images that are needed for _you_ to represent speech and numbers, you find IDK, appealing or annoying, or what? In typesetting, I find annoying caps inculsion, too little space between sentences and not so good separator character design, some flaws in letter design. For development purposes I would need some glyphs for non-printable characters, ok few glyphs more. Tak zachem _nuzhen_ Unikod ? Rasskazhi chto-nibud interesnoe. -- https://mail.python.org/mailman/listinfo/python-list
RE: Two variable dictionary comprehension
Rob Gaddi wrote, on Tuesday, April 04, 2017 3:56 PM > > On 04/04/2017 03:34 PM, Deborah Swanson wrote: > > Terry Reedy wrote, on Tuesday, April 04, 2017 11:04 AM > >> > >> I am sure that this is because list comprehensions were once thr only > >> comprehensions and that the index was not updated when the syntax and > >> code was. I opened issue 29981 "Update Index for set, dict, and > >> generator 'comprehensions'". > >> > >> -- > >> Terry Jan Reedy > > > > Thanks Terry. I does seem strange that there are so many types of > > comprehensions, yet under 'comprehensions' in the Index, you only see > > 'list'. And there are no entries for the other types, by their names. > > > > Deborah > > > > Does it seem... incomprehensible? > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. Ha ha! Indeed it does. Funny how 'comprehension' word forms so easily fall into puns. Deborah -- https://mail.python.org/mailman/listinfo/python-list
Does automatic golden master unittest generation exist/or is it feasible?
I have a really large and mature codebase in py2, but with no test or documentation. To resolve this I just had a simple idea to automatically generate tests and this is how: 1. Have a decorator that logs all arguments and return values 2. Put them in a test case and have it run in production. Here's a quick prototype I just wrote, just to illustrate what I'm trying to achieve: http://codepad.org/bkHRbZ4R People told me that there's a similar concept called "Golden Master Testing" where it keeps a "golden record" and runs test case on it. So it looks like I'm trying to automate golden master testing. So my question is this: 1. Is this a concept already explored? 2. Is there already a framework for doing this? -- https://mail.python.org/mailman/listinfo/python-list
Re: Does automatic golden master unittest generation exist/or is it feasible?
On Tue, 04 Apr 2017 19:40:03 -0700, fleshw wrote: > People told me that there's a similar concept called "Golden Master > Testing" where it keeps a "golden record" and runs test case on it. So > it looks like I'm trying to automate golden master testing. > > So my question is this: > > 1. Is this a concept already explored? https://duckduckgo.com/?q=Golden+Master+Testing https://www.google.com.au/search?q=Golden+Master+Testing Gold master testing is not a substitute for unit tests. The purpose of gold master testing is different: not the ensure that the code is correct, but to ensure that the behaviour *doesn't change*. Or at least not without a human review to ensure the change is desired. You would use gold master testing when you have a legacy code base that needs refactoring. You don't care so much about fixing bugs (either there aren't any known bugs, or you have to keep them for backwards compatibility, or simply there's no budget for fixing them yet) but you care about keeping the behaviour unchanged as you refactor the code. If that describes your situation, then gold master testing is worth pursuing. If not, then forget about it. Just write some unit tests. I like small functions that take some arguments, do one thing, and return a result without any side-effects. They are very easy to test using doctests. Doctests make good documentation too, so you can kill two birds with one stone. Write doctests. For anything too complicated for a doctest, or for extensive and detailed functional tests that exercise all the corners of your function, write unit tests. The `unittest` module can run your doctests too, automatically turning your doctests into a test case. If your code base is well-designed, with lots of encapsulation, then writing unit tests are easy. If its not... well, you have my sympathy. > 2. Is there already a framework for doing this? This came up in googling. I make no comment about its maturity, functionality, bugginess or overall quality, as I have never used it. https://github.com/approvals/ApprovalTests.Python -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: using HTTPHandler cause Django server side shows http 400 and Invalid HTTP_HOST header message
On Tue, 04 Apr 2017 21:06:23 -0700, iMath wrote: > For those want to help, I also posted The question here > http://stackoverflow.com/questions/43185804/using-httphandler-cause- django-server-side-shows-http-400-and-invalid-http-host > > > I am using HTTPHandler to send logging messages to a Django Web server > with The following code, Are you sure you posted the code you are actually using? Because the error message suggests a difference. You have: > _TARGET = '192.168.8.100:8000' but the error message says: > Invalid HTTP_HOST header: '192.168.8.100:8000,192.168.8.100'. '192.168.8.100:8000,192.168.8.100' is not the same address you say you are using. (And it is not a valid address, just as the error message states.) I might be entirely wrong here, I'm not a Django expert, but it sure looks to me like a simple mistake in your code. It would be good if you could confirm that's not the case before people spend time trying to debug something in code they can't see. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
