request.application ??
On Tue, Oct 22, 2013 at 10:01 AM, Thomas S <thomas.schmel...@gmail.com>wrote:
> Yes, I use HDF5 for more than a year now.
> It's great. The only drawback is the lack of elegant Java tools.
> I upload those files and store their obfuscated names in a database, but
> the actual file in uploads.
> Having said that it's not very elegant to construct the URL to locate the
> file:
>
> def show():
> csv = db.csv(request.args(0, cast=int))
>
> import pandas
> import os
>
> fff = os.path.join("applications", "cda", "uploads", csv.csvfile)
>
> session.dataframe = pandas.read_csv(fff, parse_dates=True, index_col=0)
>
> return dict(title=csv.title, body=csv.body, author=csv.author,
> keys=session.dataframe.keys())
>
> Note that sesssion,dataframe is now a global variable. And note the ugly
> os.path.join with the name of the application hardcoded.... UGLY!
>
> Here's a link:
> https://tschmelz.**pythonanywher**e.com/cda<https://tschmelz.pythonanywhere.com/cda>
>
> and
>
> https://github.com/tschm/cda
>
> Thomas
>
>
>
>
>
> On Tuesday, 22 October 2013 15:16:25 UTC+2, Richard wrote:
>
>> I heard a lot of good about HDF5 file format to hande important volume of
>> data hierachical (mean you can query what ever data you need without load
>> the full data set into a json for instance) : http://en.wikipedia.org/**
>> wiki/Hierarchical_Data_Format<http://en.wikipedia.org/wiki/Hierarchical_Data_Format>
>>
>> It very much faster then postgres (sure postgres is not the faster
>> backend but it scale gracefully)...
>>
>> The intend of this file format is to be used in conjonction with a DB.
>>
>> If I remember Pandas can write HDF5, not sure which lib it uses, there is
>> two major lib in python which have different set of feature, one is more
>> fancy but not support all the HDF5 feature and the other is supporting
>> "all" the feature but is less sexy...
>>
>> Richard
>>
>>
>> On Tue, Oct 22, 2013 at 6:54 AM, Cliff Kachinske <cjk...@gmail.com>wrote:
>>
>>> use the rows field in auth_permission as described here.
>>>
>>> http://web2py.com/books/**default/chapter/29/09/access-**
>>> control#Authorization<http://web2py.com/books/default/chapter/29/09/access-control#Authorization>
>>>
>>>
>>> On Tuesday, October 22, 2013 5:40:49 AM UTC-4, Thomas S wrote:
>>>>
>>>> Ok, I am making progress. I guess all those JavaScript tools are not
>>>> great when it comes to plotting millions of points but I am happy to
>>>> downsample on the server side and send less points
>>>>
>>>> - I am using flot instead of highcharts
>>>>
>>>> - Currently, the user is uploading a csv file. I don't do any parsing
>>>> at this stage. However, I rather keep the file (under uploads) and parse it
>>>> on request. Having said that this will become a lot more slick soon. This
>>>> is my first application.
>>>>
>>>> One thing that puzzles me for now...
>>>>
>>>> A user has to login to upload a file (that's good), but he can then
>>>> also modify or delete entries in the SQL database created by others. How
>>>> can I make sure he/she only deletes rows he/she has created in the first
>>>> place. All users should be able to see all files though.
>>>>
>>>> Here's a link:
>>>> https://tschmelz.**pythonanywher**e.com/csv<https://tschmelz.pythonanywhere.com/csv>
>>>>
>>>> I will soon post it to my Github (username tschm)
>>>> thomas
>>>>
>>>>
>>>> On Sunday, 20 October 2013 15:38:07 UTC+2, Niphlod wrote:
>>>>>
>>>>> first things first: are you sure that highcharts can handle 10*100k
>>>>> points to draw a graph ?
>>>>> As for the storage, you can do anything you like: if the data doesn't
>>>>> change that much, storing into the database will be a long process only on
>>>>> the first time.
>>>>> On the other end, if you need to fetch 100k records and transform them
>>>>> to json, it's going to take some time.
>>>>> Not sure on how much you'll gain from parsing i.e. a csv file instead
>>>>> of a querying a db....
>>>>> if the returning json object is , let's say, 10 mb, it's always gonna
>>>>> feel heavy.
>>>>>
>>>>> On Sunday, October 20, 2013 9:11:07 AM UTC+2, Thomas S wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have created a standard application relying on Pandas and PyQt4 to
>>>>>> browse through a Pandas Dataframe.
>>>>>> A dataframe is essentially a dictionary of time series data.
>>>>>>
>>>>>> I am new to web2py but I have experience with Pandas and matplotlib.
>>>>>> I am also tempted to embed www.highcharts.com into my application.
>>>>>>
>>>>>> Before I dig into web2py I would like to know which route is probably
>>>>>> most promising.
>>>>>>
>>>>>> Should I parse the dataframe on the webserver and write it into a SQL
>>>>>> database? I guess that could be slow?
>>>>>> Such a dataframe may consist of a dictionary with 100 elements each
>>>>>> several 100k points.
>>>>>>
>>>>>> Should I parse a time series onto request into a json format and
>>>>>> export to javaScript?
>>>>>> In this case how could I provide a way to generate a menu from the
>>>>>> keys in the dictionary.
>>>>>> E.g. user clicks on a key, python does all the computations for some
>>>>>> stats.
>>>>>>
>>>>>> The plan is to upload the data using csv files.
>>>>>>
>>>>>> So, I am a bit lost by the wide range of possibilities in web2py. I
>>>>>> would be delighted if you would like to get involved in this open source
>>>>>> project.
>>>>>> The main goal for now is to learn web2py :-)
>>>>>>
>>>>>> Please find the Github of the original application here
>>>>>> https://github.com/tschm/**Panda**sMonitor<https://github.com/tschm/PandasMonitor>
>>>>>>
>>>>>> Sorry for being so unprecise in my questions but it just reflects
>>>>>> that I don't have a very precise plan at this stage.
>>>>>>
>>>>>> Kind regards
>>>>>> Thomas
>>>>>>
>>>>> --
>>> Resources:
>>> - http://web2py.com
>>> - http://web2py.com/book (Documentation)
>>> - http://github.com/web2py/**web2py
>>> <http://github.com/web2py/web2py>(Source code)
>>> -
>>> https://code.google.com/p/**web2py/issues/list<https://code.google.com/p/web2py/issues/list>(Report
>>> Issues)
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "web2py-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to web2py+un...@**googlegroups.com.
>>>
>>> For more options, visit
>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>>> .
>>>
>>
>> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
