On Sun, Apr 1, 2018 at 2:10 PM, Stefano Zacchiroli <z...@upsilon.cc> wrote:
> On Sun, Apr 01, 2018 at 01:53:19AM -0400, Martin Blais wrote:
> > - rewriting bean-web to be a dumber, more generic web interface that
> > basically renders SQL queries (using the new query engine) without any
> > special treatment (just tables and tree-tables)
>
> Given Fava does that too, and also provides useful reporting out of the
> box, would it be worth to have another implementation of the same "web
> based bean-query console" thing? (Maybe yes, if, say, you consider that
> Fava has too many additional dependencies. Asking just to understand
> which design constraints you're considering.)
>
It would be different; all the pages would have a single query at the top
(which would be visible and which the user could tweak), and rendering
would have simple generic rules (e.g. match account name patterns) to
create links to e.g. journal pages which, again, would just be queries, but
otherwise just render generically (e.g. no special rendering for journals
to show postings and such, for instance, which I've always found ugly
anyway), even the first version would just render the text in a <pre> tag,
really bare bones. The idea is definitely to avoid replicating Fava or
bean-web, but instead provide something that's doing 60~80% of that
functionality with an incredibly simple implementation (like, one file,
calling out to bean-query library code). Basically, a web-based interface
to a more powerful and generic bean-query.
Now the twist in the plot: bean-query itself would expand in scope to go
beyond Beancount. I'm thinking of generalizing this to make it a row-based
(slow) query engine on in-memory tables, something that would be used akin
to the "re" library, and that would fit in the space that Pandas is
currently in, as well as supporting various input data formats (e.g. you
could use it to make queries on a CSV table, for example, or on a
filesystem directory, or e.g. on the EXIF files of a list of JPEG files,
whatever data sources are implemented, I can imagine many). It would be a
generic library that you could instantiate and customize to provide
SQL-like functionality to any (Python) program. It would be extensible
(allowing you to add new datatypes and control the set of available
functions), and Beancount would basically become its first use case of
customization (providing Amount, Position and Inventory data types and
associated aggregation functions, and a few data sources/tables,
"transactions" being the main one, but also one table type for each other
directive, e.g. "commodities", etc.). It would support joins and structured
fields (e.g. so you could join with per-commodity or per-account metadata,
to implement e.g. your own section categorization), and it would provide a
command-line tool (to fill in those cases where awk leaves you wanting),
but also find a lot of usage from within Python as a library (e.g., like
one uses Pandas, on in-memory data tables). It would automatically infer
schemas by e.g. looking at the contents of a CSV file, you can usually
automatically infer the data types and column names from the header, so it
would become an ultimate CSV manipulation tool (I've surveyed the other
ones out there, what I've seen falls short of what I'd like to have).
Personally I think this could make general data manipulation easier than
many of the libraries out there (e.g. Pandas), at least for those cases
where performance isn't a concern (simplistic row-based implementation, at
least at first) and doing aggregations and pivots and such isn't as easy
with just the unix tools than what you can do in a single SQL expression. I
spent some time in the past prototyping something like this using sqlite3 -
it looks like it's designed to be extensible - but it has proved too
difficult and not customizable enough to replace bean-query, we still need
something a more customizable than what this offers.
bean-web currently uses the convoluted bean-report code, which I would love
to delete (it annoys me as it's not super generalized, e.g. not all of the
reports support the same sets of outputs, and it's very ugly OO-style
code). Doing this would force me to beef up the SQL functionality a bit to
support more, and make the "default" interface to Beancount just that. I
think Fava is already covering the terrain in terms of providing a nice web
interface for Beancount and to make that out of scope for it wouldn't make
me sad at all (I'd rather let Dominik/Jakob and the rest of that team
continue to take over that space, there's a lot of changes going on there,
it's very dynamic, I see all the discussions, there's lots going on every
day).
This would result in less code for me to maintain, which in the long run is
probably a good thing for this project. I want to move toward a smaller
codebase and a more streamlined core set of functionalities. Focus on
maintaining the core, focus on solving new problems, e.g. transaction vs.
settlement dates, automating the trading accounts idea for currencies,
finally getting to properly calculating returns, stuff like that. My vision
for a simplified future currently looks like this:
- beancount.query -> transitions gradually to a new project, but which is
cutomizable enough to support Beancount's data types, aggregation functions
and data "tables" synthesized from its contents.
- beancount -> remains core data structures & schema, grammar/parser,
booking algorithm (the special sauce), prices, some scripts, as well as
custom data types and table "data sources" for the query library
- beancount.reports, beancount.web -> deprecated / deleted in the favor of
Fava, or if you're a minimalist you'd use the simplistic new web-based
interface to the query tool
So in short the impetus is this:
I would like to get rid of custom report code and the Holdings code,
which is what bean-web is implemented with, therefore I'd like to get
rid of bean-web as well,
which is fine because Fava provides something better,
but bean-bake relies on bean-web, and so I'm still not sure if
zipping up Fava to a static web site is possible.
(So far this is just an idea, nothing has been removed/done yet.)
--
You received this message because you are subscribed to the Google Groups
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to beancount+unsubscr...@googlegroups.com.
To post to this group, send email to beancount@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/beancount/CAK21%2BhMQCQekJgZ0qeWLiqD9F3%3DQC7jKBkDSwDpeye_ydDaLrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
