Hi Naresh Jonnala.
Yes, it's work to detect delimiter on csv file, But still I don't know how
to detect what is the current encoding of csv file 🤔
I need to know how to implement a good uploading csv file view on django
--
You received this message because you are subscribed to the Google Gro
Hi,
I am not sure this will help or not, Still i want add a peace of code.
sniffer = csv.Sniffer()
dialect = sniffer.sniff()
dialect.__dict__
mappingproxy({'__module__': 'csv', '_name': 'sniffed', 'lineterminator': '\r\n',
'quoting': 0, '__doc__': None, 'doublequote': False, 'delimiter': ',',
'q
Yes. You are right. Pandas' default behavior is as following:
encoding = sys.getsystemencoding() or "utf-8"
I tried to open a simple csv encoded into "utf16-LE" (popular on windows),
and got the following error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0:
invalid star
Hi Pandas require knows the encoding and delimiter previously when you use
pd.read_csv(filepath, encoding=" ", delimiter=" ") I think that is the same
🤔
El vie., 24 de julio de 2020 3:42 p. m., Jani Tiainen
escribió:
> Hi,
>
> I highly can recommend to use pandas to read csv. It does pretty good
Hi,
I highly can recommend to use pandas to read csv. It does pretty good job
to guess a lot of things without extra config.
Of course it's one more extra dependency.
pe 24. heinäk. 2020 klo 17.09 Ronaldo Mata
kirjoitti:
> Yes, I will try it. Anythin I will let you know
>
> El mié., 22 de jul
Yes, I will try it. Anythin I will let you know
El mié., 22 de julio de 2020 12:24 p. m., Liu Zheng
escribió:
> Hi,
>
> Are you sure that the file used for detection is the same as the file
> opened and decoded and gave you incorrect information?
>
> By the way, ascii is a proper subset of utf-8
Hi,
Are you sure that the file used for detection is the same as the file
opened and decoded and gave you incorrect information?
By the way, ascii is a proper subset of utf-8. If chardet said it ascii,
decoding it using utf-8 should always work.
If your file contains non-ascii UTF-8 bytes, maybe
Hi Kovy, this is not solved. Liu Zheng but using
chardet(request.FILES['file'].read()) return encoding "ascii" is not
correct, I've uploaded a file using utf-7 as encoding for example and the
result is wrog. and then I tried
request.FILES['file'].read().decode('ascii') and not work return bad data.
What i meant was that you can only feed binary data or binary handlers to
chardet. You can decode the binary data according to the detection results
afterward.
On Wed, 22 Jul 2020 at 11:11 PM, Liu Zheng wrote:
> Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and
> the chard
Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and
the chardet file handler are binary handlers. Binary handler presents the
raw data. chardet takes a sequence or raw data and then detect the encoding
format. With its prediction, if you want to open that puece of data in text
I’m confused. I don’t know if I can help.
> On Jul 22, 2020, at 11:11 AM, Liu Zheng wrote:
>
> Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and the
> chardet file handler are binary handlers. Binary handler presents the raw
> data. chardet takes a sequence or raw data a
Cool! I’m so happy I was able to help you!! Good luck!
> On Jul 22, 2020, at 11:11 AM, Liu Zheng wrote:
>
> Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and the
> chardet file handler are binary handlers. Binary handler presents the raw
> data. chardet takes a sequence
That’s probably not the proper answer, but that’s the best I can do. Sorry :-(
> On Jul 22, 2020, at 10:46 AM, Ronaldo Mata wrote:
>
> Yes, the problem here is that the files will be loaded by the user, so I
> don't know what delimiter I will receive. This is not a base command that I
> am usi
Maybe first use the standard file.open to save the file to a variable, search
that variable for the different delimiters using standard string manipulation
vichulu, and then open it using the corresponding delimiter.
> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata wrote:
>
> Hi Kovy, I'm using cs
Yes, the problem here is that the files will be loaded by the user, so I
don't know what delimiter I will receive. This is not a base command that I
am using, it is the logic that I want to incorporate in a view
El mié., 22 jul. 2020 a las 10:43, Kovy Jacob ()
escribió:
> Ah, so is the problem th
Ah, so is the problem that you don’t always know what the delimiter is when you
read it? If yes, what is the use case for this? You might not need a universal
solution, maybe just put all the info into a csv yourself, manually.
> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata wrote:
>
> Hi Kovy, I
Hi Kovy, I'm using csv module, but I need to handle the delimiters of the
files, sometimes you come separated by "," others by ";" and rarely by "|"
El mié., 22 jul. 2020 a las 10:28, Kovy Jacob ()
escribió:
> Could you just use the standard python csv module?
>
> On Jul 22, 2020, at 10:25 AM, Ro
Could you just use the standard python csv module?
> On Jul 22, 2020, at 10:25 AM, Ronaldo Mata wrote:
>
> Hi Liu thank for your answer.
>
> This has been a headache, I am trying to read the file using csv.DictReader
> initially i had an error trying to get the dict keys when iterating by rows
Hi Liu thank for your answer.
This has been a headache, I am trying to read the file using
csv.DictReader initially i had an error trying to get the dict keys when
iterating by rows, and i thought it could be encoding (for this reason i
wanted to prepare the view to use the correct encoding). for
Hi. First of all, I think it's impossible to perfectly detect encoding
without further information. See the answer in this SO post:
https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text
There
are many packages and tools to help detect encoding format, but keep in
min
How to deal with encoding when you try to read a csv file on view.
I have a view to upload csv file, in this view I read file and save each
row as new record.
My bug is when I try to upload a csv file with a differente encoding (not
UTF-8)
how to handle this on django (using request.FILES) I was
On Wednesday, 3 November 2010 09:34:38 UTC+5:30, Jorge wrote:
>
> Jirka & Everybody
>
> Back to basics is always a good advice. With your help and this guy:
> http://www.beardygeek.com/2010/03/adding-views-to-the-django-admin/
> I can create a form to upload csv files and input their records int
On Mon, Oct 3, 2011 at 10:30 AM, Victor Hooi wrote:
> heya,
> @Shawn: Hmm, that looks like quite an interesting approach - so the
> ModelForm would handle the validation logic? =) Me likes! Haha.
> I can parse the CSV and get the field names to match up (or just do a simple
> transform to get them
heya,
@Shawn: Hmm, that looks like quite an interesting approach - so the
ModelForm would handle the validation logic? =) Me likes! Haha.
I can parse the CSV and get the field names to match up (or just do a simple
transform to get them to match).
However, how do I then pass this information i
A clean way would be to read in the CSV with csv.DictReader, and
ensure that the key names match your field names.
Then use a ModelForm for your model, and pass in each dict as the 'data' field.
This lets you re-use all the hard work already done in the ModelForm,
because the is_valid() method wi
On Monday, 3 October 2011 14:42:36 UTC+1, Victor Hooi wrote:
>
> heya,
>
> I'm coding up a Django form which will let the user upload a CSV file, then
> create and save multiple Model instances for each row in the CSV file.
>
> At the moment, I'm trying to decide where to put the code that parses
heya,
I'm coding up a Django form which will let the user upload a CSV file, then
create and save multiple Model instances for each row in the CSV file.
At the moment, I'm trying to decide where to put the code that parses the
CSV file and creates/saves the models.
I don't think it'd be an ins
Jirka & Everybody
Back to basics is always a good advice. With your help and this guy:
http://www.beardygeek.com/2010/03/adding-views-to-the-django-admin/
I can create a form to upload csv files and input their records into
the database.
Short history:
Forms.py:
class DataInput(forms.Form):
> I try to follow the ideas, but i feel like taking the dirty way.
> Here's my work:
> Because i can't replace the modelform created by the admin for the
> "Data" model with a standard form as the Django docs says, i created a
> view to replace the "add" view generated by the admin and put a
> stan
'^add/$',
self.admin_site.admin_view(self.add_view)),
)
return my_urls + urls
def add_view(self, request, extra_content=None):
FormInput = DataInput()
context = {'form': FormInput}
return super(DataAdmin, self).add_view(requ
The ordinary user won't have to deal with the command line, you just
need to get the CSV file. A ModelForm to which an FileField is added
doesn't really have anything to do with that.
Instead, you should create a standard form with a single FileField to
upload the CSV file, save it temporarily or
Jirka
I need an easy method inside the admin to upload the csv file with an
web interface. I guess with your method the admin will need access to
the server and the command line, and something like this is not what i
try to do, because the admin (not me) is not a django developer, not
even a user
I must still be missing something here. If all you want to do is to
read CSV file and save the data in the database, I still don't
understand why you use the forms machinery.
You only *need* to use the data model, e.g.
from myapp.models import Data
csvfile = csv.reader('something.csv')
for line
>
> Hang on, what you're doing here is repeatedly setting the data values
> for each line to the *same* form_input model. Presumably what you
> actually want to do is to create new Data instances for each line in
> the CSV, with the place set to the value of the form.
>
> In this case, I'd recommen
On Oct 28, 5:08Â am, Jorge wrote:
> Shame on me!
> I forgot to make some changes inside the for loop.
> So now it is:
>
> Â def save(self, commit=True, *args, **kwargs):
> Â Â Â Â Â form_input = super(DataInput, self).save(commit=False, *args,
> Â **kwargs)
> Â Â Â Â Â form_input.place = self.cleaned_d
On Oct 27, 7:43Â pm, Jorge wrote:
> > On 27/10/2010, Daniel Roseman wrote:
>
> > > You're setting the values on `self` in the form's save method. In
> > > other words, you're setting them on the *form*, not the instance. You
> > > should be setting them on `form_instance`.
>
> > > Also, don't co
On Oct 27, 1:58Â pm, Jirka Vejrazka wrote:
> If you don't mind me asking, why do use ModelForm and not the ORM
> directly? I'm just curious as using just a model (possibly wirh model
> validation) seems like easies approach.
>
> Â Cheers
>
> Â Â Jirka
Hi Jirka!
The idea behind the modelform is
Field(blank=True)
> Â Â Â Â data_1 = models.DecimalField(max_digits=3, decimal_places=1,
> blank=True)
> Â Â Â Â data_2 = models.DecimalField(max_digits=3, decimal_places=1,
> blank=True)
> Â Â Â Â data_3 = models.DecimalField(max_digits=4, decimal_places=1,
> blank=True)
>
>
l_places=1,
blank=True)
data_2 = models.DecimalField(max_digits=3, decimal_places=1,
blank=True)
data_3 = models.DecimalField(max_digits=4, decimal_places=1,
blank=True)
Forms.py:
import csv
from datetime import datetime
class DataInput(ModelForm):
file = forms.FileField()
I got it to work by just using the (already) imported
django.contrib.admin
BatchModelAdmin now becomes admin.ModelAdmin
http://code.google.com/p/django-batchimport/issues/detail?id=4
On Mar 19, 1:22Â pm, Mark wrote:
> Hello,
> I'm having some trouble putting together a form for batch import of
>
Hello,
I'm having some trouble putting together a form for batch import of
csv files.
I found django-batchimport at http://code.google.com/p/django-batchimport/
and that seemed to be the answer. It looks like it was integrated
into Django a year ago (http://code.djangoproject.com/changeset/
10121
41 matches
Mail list logo
