Thank you all for the great input. Russell, your design I think will make the best framework - after all, defining and registering a model is hardly more difficult than defining each model in the settings. Plus, I'm realizing now that each *type* of data may come with its own particular functions, so it makes sense to group related models together into an app. It is indeed looking more and more like a clone of the admin app in terms of how it pulls together and abstracts data from other apps.
Jonathan, I'm certainly going to take inspiration from your time-series app, but I don't think I can use it for this particular application - my data points are much simpler (each of my data points is marked with only a single datetime, and that suffices). I'll look into postgresql as well, although again my models are fairly simple and not relying too heavily on relational fields. Regards, RL On Wednesday, March 12, 2014 9:44:48 AM UTC-4, Jonathan Morgan wrote: > > On the model side, to facilitate a generic visualization layer, you could > also consider an abstract parent class where you standardize time series > information that isn't the data. > > For my time series data, I have this: > > https://github.com/jonathanmorgan/django_time_series > > I built a few django time-series models, then started to abstract out the > parts that were the same across the models. This includes: > > - a Time_Period model to hold a set of defined time periods, if your time > periods are more complex than simply time increments (for example, I did > time-series of reddit activity within subreddits, broken out by hour, and > also categorized into being within 14 days before and after a certain date > - so before-1, before-2, before-3, ..., after-1, after-2, after-3, ...). > > - an AbstractTimeSeriesDataModel that includes basic information on a time > period (start and end datetime, time period index, a separate aggregate > index in case you have something like my before and after, and so have > after-1, which is actually number 327 overall, etc.), plus the ability to > set and hold ten different generic filters (so you can mark a given time > period as having contained links to news, for example, using filter_1, and > store a count of matches within the period in match_count_1, or use > filter_2 and match_count_2 to note a particular time-series period had X > posts related to a particular topic), and base methods for doing a lookup > and retrieving an instance. > > > Then, for a particular data set, you extend this abstract class and add in > columns you want to track within a given time period (see > https://github.com/jonathanmorgan/reddit_data for an example - the model > Subreddit_Time_Series_Data is a much more complete example than > Domain_Time_Series_Data). > > An abstract base class like this would give you the common elements of > time-series data on which you could start to build a generic visualizer, > and also then give you the flexibility to add data as needed. You could > probably just use a simple naming convention to reveal added columns as > data ("tsdata_"). > > You could also probably add generic data fields to this abstract model, > but it didn't seem to be useful for me when I built this, since I was > capturing many fields per time series, and I kept having to add fields as > our research led to more questions. > > > Also worth noting, if you go with a relational database (I personally like > relational databases for complex, relational data) and might have millions > of rows, postgresql is much easier to get reasonable performance out of > than mysql. > > On Tuesday, March 11, 2014 10:42:24 AM UTC-5, RLange wrote: >> >> I'm currently working on an app for browsing and visualizing time-series >> data. Each point of time-series data may be a mix of Strings, Floats, and >> Ints. In my current design, I have a separate model for each of my data >> types, and I have been writing a new view for each one. In other words, my >> app is strongly coupled to the structure of the models. *Ideally*, my >> app would use some sort of generic abstraction of a time-series model such >> that adding new types of data is a simple settings.py configuration, and >> the views for browse/visualize would be free. >> >> I see a few possible avenues to accomplish this. None seems clearly >> better than any other. My database is MySQL. Any feedback is helpful! >> >> 1. Use django-mutant (or equivalent) to make new models on the fly. >> However I would really only need to make models at initialization time, >> not >> at runtime. >> 2. Instead of a column for each dimension of the data, use a Blob or >> Text type for headers and one for data (a hack for variable-length data. >> makes intelligent queries nearly impossible) >> 3. Forget about pluggability and continue developing under the >> assumption that this app will never leave "in-house" >> >> >> This is a deliberately vague question since it is about a vague topic: >> abstraction. I can provide more details about my specific use case, but >> that seems to defeat the purpose of writing a generic app. >> >> Thank you, >> RLange >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/176c83d1-0a15-480e-9777-3f3e91bef587%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
