Re: how to fast processing one million strings to remove quotes

2017-08-04 Thread Tim Daneliuk
On 08/04/2017 01:52 AM, Peter Otten wrote: > It looks like Python is fairly competetive: > > $ wc -l hugequote.txt > 100 hugequote.txt612250 > $ cat unquote.py > import csv > > with open("hugequote.txt") as instream: > for field, in csv.reader(instream): > print(field) > > $

Re: how to fast processing one million strings to remove quotes

2017-08-03 Thread Peter Otten
Tim Daneliuk wrote: > On 08/02/2017 10:05 AM, Daiyue Weng wrote: >> Hi, I am trying to removing extra quotes from a large set of strings (a >> list of strings), so for each original string, it looks like, >> >> """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" >> >> >> I like to r

Re: how to fast processing one million strings to remove quotes

2017-08-03 Thread Nick Mellor
Sorry Daiyue, Try this correction: I'm writing code without being able to execute it. > split_on_dbl_dbl_quote = original_list.join('|').split('""') > remove_dbl_dbl_quotes_and_outer_quotes = > split_on_dbl_dbl_quote[::2].join('').split('|') split_on_dbl_dbl_quote = original_list.join('|').sp

Re: how to fast processing one million strings to remove quotes

2017-08-03 Thread Tim Daneliuk
On 08/02/2017 10:05 AM, Daiyue Weng wrote: > Hi, I am trying to removing extra quotes from a large set of strings (a > list of strings), so for each original string, it looks like, > > """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" > > > I like to remove the start and end quotes

Re: how to fast processing one million strings to remove quotes

2017-08-03 Thread Thomas Jollans
On 2017-08-02 19:05, MRAB wrote: > On 2017-08-02 16:05, Daiyue Weng wrote: >> Hi, I am trying to removing extra quotes from a large set of strings (a >> list of strings), so for each original string, it looks like, >> >> """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" >> >> >> I lik

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Nick Mellor
On Thursday, 3 August 2017 01:05:57 UTC+10, Daiyue Weng wrote: > Hi, I am trying to removing extra quotes from a large set of strings (a > list of strings), so for each original string, it looks like, > > """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" > > > I like to remove the

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Peter Otten
Daiyue Weng wrote: > On 2 August 2017 at 19:13, Peter Otten <__pete...@web.de> wrote: > >> Daiyue Weng wrote: >> >> > Hi, I am trying to removing extra quotes from a large set of strings (a >> > list of strings), so for each original string, it looks like, >> > >> > """str_value1"",""str_value2""

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Daiyue Weng
it is getting from an encrypted and snappy file On 2 August 2017 at 19:13, Peter Otten <__pete...@web.de> wrote: > Daiyue Weng wrote: > > > Hi, I am trying to removing extra quotes from a large set of strings (a > > list of strings), so for each original string, it looks like, > > > > """str_valu

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Peter Otten
Daiyue Weng wrote: > Hi, I am trying to removing extra quotes from a large set of strings (a > list of strings), so for each original string, it looks like, > > """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" Where did you get that strange list from in the first place? If it is

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Terry Reedy
On 8/2/2017 1:05 PM, MRAB wrote: On 2017-08-02 16:05, Daiyue Weng wrote: Hi, I am trying to removing extra quotes from a large set of strings (a list of strings), so for each original string, it looks like, """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" I like to remove the s

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread Daiyue Weng
that works superbly! any idea about how to multi process the task and concatenate results from each process back into a list? On 2 August 2017 at 18:05, MRAB wrote: > On 2017-08-02 16:05, Daiyue Weng wrote: > >> Hi, I am trying to removing extra quotes from a large set of strings (a >> list of s

Re: how to fast processing one million strings to remove quotes

2017-08-02 Thread MRAB
On 2017-08-02 16:05, Daiyue Weng wrote: Hi, I am trying to removing extra quotes from a large set of strings (a list of strings), so for each original string, it looks like, """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" I like to remove the start and end quotes and extra pair

how to fast processing one million strings to remove quotes

2017-08-02 Thread Daiyue Weng
Hi, I am trying to removing extra quotes from a large set of strings (a list of strings), so for each original string, it looks like, """str_value1"",""str_value2"",""str_value3"",1,""str_value4""" I like to remove the start and end quotes and extra pairs of quotes on each string value, so the r