Re: python GnuCash interface to SQL backend

[Sébastien de Menten]

On Wednesday, November 12, 2014, John Ralls  wrote:

>
> > On Nov 11, 2014, at 1:10 PM, Sébastien de Menten  > wrote:
> > 
> >
> > I would be genuinely interested to have more specific documentation on
> the
> > risks of going the SQL way.
>
> There's nothing wrong with reading the database to generate reports. That
> is indeed easier for many people via SQL query than writing custom report
> plugins in Scheme.
>
> It may also be easier to go with the SQL than with the std python binding
for reporting but also to change the GnuCash book.


> The risk of writing to the database outside of GnuCash, whether in SQL or
> XML, is that unless you are very careful and have a deep understanding of
> how GnuCash works that you will irretrievably corrupt your accounting data.
> There is no business logic encoded in the SQL database, so your code must
> replicate the GnuCsah engine code to ensure that all required fields are
> computed and stored correctly. Much of GnuCash is neither straightforward
> nor obvious and some critical data are stored outside of the primary
> tables, usually to preserve backward compatibility with previous versions.
>
> Regards,
> John Ralls
>
>
>
I have mainly used the basic objects from GnuCash required for
basic personal finance (so no invoice, no budget, ...) and did not found
any issues while handling lot of accounts/transactions/splits and stock
prices. I had the impression that GnuCash does indeed calculations when the
book is opened but that it does not save them in the SQL backend. Hence, if
we access the book when it's not opened by GnuCash at the same time, risks
are quite reduced, would this be a "wrong" impression ?

Where could I find detailed documentation on the GnuCash engine (and the
constrains/invariants GnuCash enforces) ? Or would there be some
code/program to check a GnuCash file is "sane/consistent" ?

Regards

Sebastien
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[Sébastien de Menten]

On Thursday, November 13, 2014, John Ralls  wrote:

>
> On Nov 12, 2014, at 12:08 PM, Sébastien de Menten  > wrote:
>
> On Wednesday, November 12, 2014, John Ralls  > wrote:
>
>>
>> > On Nov 11, 2014, at 1:10 PM, Sébastien de Menten 
>> wrote:
>> > 
>> >
>> > I would be genuinely interested to have more specific documentation on
>> the
>> > risks of going the SQL way.
>>
>> There's nothing wrong with reading the database to generate reports. That
>> is indeed easier for many people via SQL query than writing custom report
>> plugins in Scheme.
>>
>> It may also be easier to go with the SQL than with the std python binding
> for reporting but also to change the GnuCash book.
>
>
> It might be, but I doubt it. You won’t be able to implement the business
> logic in SQL alone.
>
>
The python interface uses SQLAlchemy (an ORM) only to handle the backend
(retrieve and save objects), all business logic is in the python code. For
instance, when creating a transaction and the related splits, it is the
python code that ensures the business logic (for instance that the sum of
the splits = 0). This is close to what GnuCash does.
Moreover, there are some basic SQL integrity constrains (we cannot remove a
split without removing the related transaction) that are added in the ORM
layer as they do not exist in the SQL backend.

>
>
>>
>> I have mainly used the basic objects from GnuCash required for
> basic personal finance (so no invoice, no budget, ...) and did not found
> any issues while handling lot of accounts/transactions/splits and stock
> prices. I had the impression that GnuCash does indeed calculations when the
> book is opened but that it does not save them in the SQL backend. Hence, if
> we access the book when it's not opened by GnuCash at the same time, risks
> are quite reduced, would this be a "wrong" impression ?
>
> I’m not sure I follow you about calculations when the book is opened that
> aren’t saved.
>
I was thinking about temporary results/cached calculations/etc that are not
saved to the back ends (if there are any).


> With the SQL backend, everything is written back to the database when a
> change is committed, so the *results* of the calculations are immediately
> saved. What GnuCash doesn’t do is *read* the database after the initial
> load, nor does it use any database concurrency control, so there are two
> potential ways to screw things up with two programs (even two instances of
> GnuCash) using the same database: A change made by one instance could be
> overwritten by a change made in the other or, much worse, the two instances
> could try writing the same records at the same time, corrupting those
> records.
>
> Indeed, if GnuCash has opened the file and is using it (ie there is a lock
in the table gnc_lock), we are almost 100% sure to have the issues you
mentions if we change the file in parallel through SQL.  There is a check
in pyscash that raises an exception in this case (it can be overruled but
at user's own risk.
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Invoices and Bill Reports

[JP Liew]

Hi all,

After searching through most of the old messages and online, I can't seems to 
find the right path to the information. Hopefully someone will be able to point 
me to the right path.


Basically I need to generate two monthly or quarterly reports, either in html 
or csv


1. Invoices Report that has the following fields

Date, Customer, Total, Subtotal, Tax

2. Bills Report that has the following fields

Date, Vendor, Total, Subtotal, Tax


Thank you so much.


Cheers
JP
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Working example of kvp acess in Python

[Christoph Holtermann]

Hello,

Am 13.11.2014 um 03:05 schrieb John Ralls:
>> On Nov 12, 2014, at 3:56 PM, Christoph Holtermann  
>> wrote:
>>
>> Hello,
>>
>> for my part i answered some of the questions myself:
>>
>> Am 12.11.2014 um 18:46 schrieb Christoph Holtermann:
>>> Hello,
>>>
>>> so KVPs should be accessed by objects they belong to and not from the 
>>> outside.
>>> Some questions:
>>> * Is it legitimate to have KVP representation in Python at all or is this 
>>> low-level und
>>> should remain to the c-api ?
>> It is legitimate but use should be restricted. Maybe warning docs or putting 
>> them in a submodule
>> called low_level.
> In 2.6 and before KVP is part of the public API and is accessed directly from 
> all over the program. If you're working with 2.6 you have no alternative to 
> using KVP directly if you need access to some items stored there because the 
> classes that the KVP is associated with have no code that manipulates those 
> KVP items.
>
> In master, KVP is a private implementation detail. The KVP accesses outside 
> of src/engine have been assigned GObject Properties on their associated 
> objects and should be accessed that way (except for one: I discovered a KVP 
> access done in Scheme that had escaped my notice and which is still on the 
> loose). Since the API is going to change rather a lot as we convert to C++ 
> you probably want to ignore all of that and stick with the 2.6 API until we 
> start the 2.7 releases.
>
>>> * The information that I'm interested in is the company data for invoices. 
>>> As far as
>>> I' ve seen there is just KVP access to that.
>>> * It seems to me that there should be an object Company that is structured 
>>> similar
>>> to Customer and has an assigned object Address identical to Customer.
>>> * When only the object should be able to access KVP, who is this object for 
>>> the
>>> company address ? At the moment it's book, I guess.
>>> * I could create a python object Company and add getter functions that 
>>> access the
>>> KVPs. But it seems better to me to not introduce objects in python that do 
>>> not exist
>>> in c.
>>> * On the other hand it can still be changed afterwards if an object is 
>>> introduced in
>>> c.
>> I decide to create an object Company.
>> https://github.com/c-holtermann/gnucash/blob/master/src/optional/python-bindings/example_scripts/gncCompany.py
>>
>> This I can easily be put to jinja in my invoice_export script.
>> I'm quite happy to be able to submit all the information by this simple call:
>> output = template.render(invoice=invoice, locale=locale, 
>> company=Company(book))
>>
>> I posted a pull request for the jinja2 approach. The inclusion of access to 
>> the Company
>> data remains head of my development fork:
>> https://github.com/c-holtermann/gnucash
>>
>> Maybe you can have a look and it and we can come to an agreement if the 
>> Company
>> and KVP approach is adequate or how it can be improved.
>>
>> Further steps would be a closer integration into the GUI.
> To date our policy on the Python bindings is that they're for users to query 
> the GC database for their own use. We're don't accept Python extensions of 
> GnuCash into GnuCash itself and have no intention or interest in adding 
> Python interpreters to the Mac and Windows AIO bundles. I don't see any 
> reason to change that policy. I haven't yet looked at your pull request, but 
> if it conflicts with that policy you can do me a favor and withdraw it.
No. It's clean outside GnuCash's GUI. So no policy violation here ;-)
It's just some additions to the already existing script to export invoices to 
LaTeX using templating engine jinja2.
So I think there should be no problem with this.

The 2nd pull request will be to access the Company data for which KVP access is 
necessary at the moment.
>
> My intention for KVP is to make it a backend implementation detail for 
> storage only and completely invisible to the rest of the program. There are 
> quite a few chunks of KVP that don't fit very well with the objects they're 
> currently associated with. I expect that in the course of the C++ rewrite 
> we'll be creating new classes to deal with them. Company data is one of 
> those, the import account matching data is another, and there are more that 
> don't come immediately to mind.
I'll put these bits of information into the documentation of the KVP access. 
Especially the thing about the API to change in 2.7.
>
> Regards,
> John Ralls
>
>
>
regards,

Christoph Holtermann

-- 
--- Nachricht gesendet von C. Holtermann ---
-  -
-  Verschlüsselte Nachrichten können über  -
- den öffentlichen Schlüssel auf folgendem -
- Keyserver an mich gesendet werden:   -
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4DD9CF0482B0620B



___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

New mailing list for Russian Translation discussions

[Dmitry Pavlov]

Hi!

Is it possible to create a new mailing list for discussing translations to
russian?

The reason to do it: developer list is mostly related to code things and
currently there is no active list for translation discussions.

As I don't participate in development it would be great to have specific
channel for translation related questions, and discussions.

Current contact information about Translation group in ru.po file is
obsolete, mail is dead and as the last translator I propose to create new
list.

This page  says there are already
a number of regional mailing list already like de, pt etc.

If it'll take a lot of time, then I can create list at googlegroups with
updating Translation group information in po file.

-- 
Regards,
Dmitry Pavlov
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Would like to convert provider_list to std::list

[Chenxiong Qi]

Hi all,

I would like to convert provider_list in qofsession.cpp to std::list.
Make a small change (step) to C++. What do you say? :)

-- 
A Python developer living in Beijing
http://about.me/tkdchen
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Would like to convert provider_list to std::list

[Aaron Laws]

On Thu, Nov 13, 2014 at 9:46 AM, Chenxiong Qi  wrote:

> Hi all,
>
> I would like to convert provider_list in qofsession.cpp to std::list.
> Make a small change (step) to C++. What do you say? :)
>

Is this your first forray into contributing to gnucash (C or C++)? I think
the hope is that qofsession will become 'class QofSessionImpl' or something
like that, but I don't know that part of the code yet. It is probably a
useful exercise for yourself to go ahead and do what you propose even if
you're not interested in going "whole-hog" yet. I wouldn't hold your breath
that this would be accepted, though (I don't do the accepting, by the way),
because a global std::list may not be seen as an improvement in itself ^_^ .

You asked "What do you say?", and this is what I say. Do keep in mind that
I don't make any decisions like this for GnuCash development :-)

In Christ,
Aaron Laws
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[John Ralls]

> On Nov 12, 2014, at 10:12 PM, Sébastien de Menten  wrote:
> 
> On Thursday, November 13, 2014, John Ralls  > wrote:
> 
>> On Nov 12, 2014, at 12:08 PM, Sébastien de Menten > 
>> wrote:
>> 
>> On Wednesday, November 12, 2014, John Ralls > wrote:
>> 
>> > On Nov 11, 2014, at 1:10 PM, Sébastien de Menten > 
>> > wrote:
>> > 
>> >
>> > I would be genuinely interested to have more specific documentation on the
>> > risks of going the SQL way.
>> 
>> There's nothing wrong with reading the database to generate reports. That is 
>> indeed easier for many people via SQL query than writing custom report 
>> plugins in Scheme.
>> 
>> It may also be easier to go with the SQL than with the std python binding 
>> for reporting but also to change the GnuCash book.
> 
> It might be, but I doubt it. You won’t be able to implement the business 
> logic in SQL alone.
> 
> 
> The python interface uses SQLAlchemy (an ORM) only to handle the backend 
> (retrieve and save objects), all business logic is in the python code. For 
> instance, when creating a transaction and the related splits, it is the 
> python code that ensures the business logic (for instance that the sum of the 
> splits = 0). This is close to what GnuCash does. 
> Moreover, there are some basic SQL integrity constrains (we cannot remove a 
> split without removing the related transaction) that are added in the ORM 
> layer as they do not exist in the SQL backend. 
>>  
>> 
>> I have mainly used the basic objects from GnuCash required for basic 
>> personal finance (so no invoice, no budget, ...) and did not found any 
>> issues while handling lot of accounts/transactions/splits and stock prices. 
>> I had the impression that GnuCash does indeed calculations when the book is 
>> opened but that it does not save them in the SQL backend. Hence, if we 
>> access the book when it's not opened by GnuCash at the same time, risks are 
>> quite reduced, would this be a "wrong" impression ?
>> 
> 
> I’m not sure I follow you about calculations when the book is opened that 
> aren’t saved. 
> I was thinking about temporary results/cached calculations/etc that are not 
> saved to the back ends (if there are any). 
>  
> With the SQL backend, everything is written back to the database when a 
> change is committed, so the *results* of the calculations are immediately 
> saved. What GnuCash doesn’t do is *read* the database after the initial load, 
> nor does it use any database concurrency control, so there are two potential 
> ways to screw things up with two programs (even two instances of GnuCash) 
> using the same database: A change made by one instance could be overwritten 
> by a change made in the other or, much worse, the two instances could try 
> writing the same records at the same time, corrupting those records.
> 
> Indeed, if GnuCash has opened the file and is using it (ie there is a lock in 
> the table gnc_lock), we are almost 100% sure to have the issues you mentions 
> if we change the file in parallel through SQL.  There is a check in pyscash 
> that raises an exception in this case (it can be overruled but at user's own 
> risk.  


What’s your goal here? I don’t think that reimplementing GnuCash in Python with 
GnuCash’s SQL schema is a particularly good approach: It’s not exactly the most 
efficient design. Rather, it’s designed to mirror the XML schema. You’ll have a 
better design if you relegate GnuCash SQL to import/export.

An aside about the name: Pyscash is likely to be pronounced by English speakers 
with a short “i” sound where the y goes.  That conveys a rather unfortunate 
meaning.

Regards,
John Ralls

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: New mailing list for Russian Translation discussions

[John Ralls]

> On Nov 13, 2014, at 4:27 AM, Dmitry Pavlov  wrote:
> 
> Hi!
> 
> Is it possible to create a new mailing list for discussing translations to
> russian?
> 
> The reason to do it: developer list is mostly related to code things and
> currently there is no active list for translation discussions.
> 
> As I don't participate in development it would be great to have specific
> channel for translation related questions, and discussions.
> 
> Current contact information about Translation group in ru.po file is
> obsolete, mail is dead and as the last translator I propose to create new
> list.
> 
> This page  says there are already
> a number of regional mailing list already like de, pt etc.
> 
> If it'll take a lot of time, then I can create list at googlegroups with
> updating Translation group information in po file.

The Dutch, French, German, Italian, and Portuguese mailing lists are the 
equivalent in those languages of gnucash-user. If you think that there's enough 
interest in GnuCash in Russia to make such a list worthwhile and you're willing 
to moderate it and provide support to those users that way, great. 

Those lists are mentioned on the translation page because they're a ready 
source of native speakers of those languages who are also familiar with GnuCash 
who can help with interpretation and proof reading. Translation isn't their 
main focus. I suspect that a list devoted only to translation of a single 
language would be pretty lonely.

Regards,
John Ralls
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Would like to convert provider_list to std::list

[Chenxiong Qi]

On Thu, Nov 13, 2014 at 11:12 PM, Aaron Laws  wrote:
> On Thu, Nov 13, 2014 at 9:46 AM, Chenxiong Qi  wrote:
>>
>> Hi all,
>>
>> I would like to convert provider_list in qofsession.cpp to std::list.
>> Make a small change (step) to C++. What do you say? :)
>
>
> Is this your first forray into contributing to gnucash (C or C++)? I think
> the hope is that qofsession will become 'class QofSessionImpl' or something
> like that, but I don't know that part of the code yet. It is probably a
> useful exercise for yourself to go ahead and do what you propose even if
> you're not interested in going "whole-hog" yet. I wouldn't hold your breath
> that this would be accepted, though (I don't do the accepting, by the way),
> because a global std::list may not be seen as an improvement in itself ^_^ .
>
> You asked "What do you say?", and this is what I say. Do keep in mind that I
> don't make any decisions like this for GnuCash development :-)
>
> In Christ,
> Aaron Laws

Hi Aaron,

I just noticed the provider_list during going through the code in
libqof/qof , and I think it might be a small and quick change to the
migration. A global provider_list seems not a good solution indeed.
emm.. what I mentioned in the first mail is really not a good idea,
ignore it. :)

Regards,
Chenxiong Qi

-- 
A Python developer living in Beijing
http://about.me/tkdchen
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Would like to convert provider_list to std::list

[John Ralls]

> On Nov 13, 2014, at 7:12 AM, Aaron Laws  wrote:
> 
> On Thu, Nov 13, 2014 at 9:46 AM, Chenxiong Qi  wrote:
> 
>> Hi all,
>> 
>> I would like to convert provider_list in qofsession.cpp to std::list.
>> Make a small change (step) to C++. What do you say? :)
>> 
> 
> Is this your first forray into contributing to gnucash (C or C++)? I think
> the hope is that qofsession will become 'class QofSessionImpl' or something
> like that, but I don't know that part of the code yet. It is probably a
> useful exercise for yourself to go ahead and do what you propose even if
> you're not interested in going "whole-hog" yet. I wouldn't hold your breath
> that this would be accepted, though (I don't do the accepting, by the way),
> because a global std::list may not be seen as an improvement in itself ^_^ .
> 
> You asked "What do you say?", and this is what I say. Do keep in mind that
> I don't make any decisions like this for GnuCash development :-)

Qi,

Aaron's right: There's no benefit to that particular change other than removing 
a single GLib dependency out of hundreds. Most of QofSession, including the 
provider list, isn't really used. It was added when the authors of QOF thought 
that QOF might be useful outside of GnuCash and wrote it to be much more 
general than GnuCash needs. Very little of QofSession is likely to survive the 
rewrite.

If you want a good target for conversion from GList, look at QofCollection in 
qofid.cpp. This class is heavily used and bogs down badly with large datasets. 
Study its use-cases and design a re-implementation that is fast and scalable -- 
or if appropriate, multiple implementations for different use-cases. Profile 
the current code and try different implementations profiling each. Be sure to 
consider and test using a custom allocator.

Regards,
John Ralls




___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Working example of kvp acess in Python

[Christian Stimming (mobil)]

>From my understanding, there are two separate issues here:

1. You are proposing a new object "Company" because you have reasons for 
needing new data fields. I think that's a good idea. Why don't you prepare a 
patch to extend the existing objects?

2. You ask for kvp access from python. No, we prefer not to have kvp accessed 
from python. Instead, either use gobject properties (on master), or add a new 
object on the C side as mentioned above and add python wrappers for this.

Regards, Christian
--
Sent from mobile.
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Working example of kvp acess in Python

[Christoph Holtermann]

Am 13.11.2014 um 17:50 schrieb Christian Stimming (mobil):
> From my understanding, there are two separate issues here:
>
> 1. You are proposing a new object "Company" because you have reasons for 
> needing new data fields. I think that's a good idea. Why don't you prepare a 
> patch to extend the existing objects?
>
> 2. You ask for kvp access from python. No, we prefer not to have kvp accessed 
> from python. Instead, either use gobject properties (on master), or add a new 
> object on the C side as mentioned above and add python wrappers for this.
Surely adding an object on the C side would be the more robust approach. Can 
you tell me an object that already exists
and that could be taken as an example ?

I don't really understand the gobject concept though and what the difference to 
my approach is.

It seems some work until we have something we're all happy with, though ;-)
>
> Regards, Christian
> --
> Sent from mobile. 
regards,

Christoph

-- 
--- Nachricht gesendet von C. Holtermann ---
-  -
-  Verschlüsselte Nachrichten können über  -
- den öffentlichen Schlüssel auf folgendem -
- Keyserver an mich gesendet werden:   -
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4DD9CF0482B0620B



___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[John Ralls]

On Nov 13, 2014, at 9:31 AM, Sébastien de Menten  wrote:

> Indeed, it may be worth to explain what are the goals (and the limits).
> 
> I have tried to use the official python bindings and had the following issues:
> - need swig + compilations to make them work => pyscash is pure python and 
> has only sqlalchemy as main dependency (which is rather supported/standard 
> package)
> - python binding is a mapping if C API with a thin layer python friendly 
> layer => I do not find the resulting python scripts very pythonic
> 
> Hence, instead of writing (part of the example to create accounts):
> 
> s = Session("bookname", is_new=False)
> 
> book = s.book
> root = book.get_root_account()
> commod_table = book.get_table()
> CAD = commod_table.lookup('CURRENCY', 'CAD')
> 
> a = Account(book)
> root.append_child(a)
> a.SetName('Assets')
> a.SetType(ACCT_TYPE_ASSET)
> a.SetCommodity(CAD)
> 
> a2 = Account(book)
> a.append_child(a2)
> a2.SetName('Receivables')
> a2.SetType(ACCT_TYPE_RECEIVABLE)
> a2.SetCommodity(CAD)
> 
> a3 = Account(book)
> root.append_child(a3)
> a3.SetName('Income')
> a3.SetType(ACCT_TYPE_INCOME)
> a3.SetCommodity(CAD)
> 
> i can now write (reusing standard SQL Alchemy patterns):
> 
> s = Session("bookname")   # pyscash cannot create a Book from 
> scratch. It relies on gnucash to do this
> 
> # query from the session the Book and the CAD currency
> book = s.query(Book).one()
> CAD = s.query(Commodity).filter_by(name="CAD").one()
> 
> root = book.root_account
> 
> # possibility to create an account in a pythonic way
> acc = Account(name="Assets", type=ACCT_TYPE_ASSET, commodity=CAD, 
> parent=root_account) 
> 
> # possibility to create accounts directly with subaccounts
> acc_inc = Account(name="Income", type=ACCT_TYPE_ASSET, commodity=CAD,
>   parent=root,
>   children=[
>   Account(name="Income subacc 1", type=ACCT_TYPE_INCOME, 
> commodity=CAD),
>   Account(name="Income subacc 2", type=ACCT_TYPE_INCOME, 
> commodity=CAD),
>   Account(name="Income subacc 3", type=ACCT_TYPE_INCOME, 
> commodity=CAD),
> )
> 
> # and then either one of the command
> s.rollback()   # to undo changes
> s.commit()# to save/commit changes
> 
> So, to sum up, goals of pyscash :
> - easy installation as pure python module (no compilation, no swig, ...)
> - easy to contribute as pure python and based on de facto standard ORM for 
> SQL in python (sql alchemy)
> - pythonic implementation by leveraging SQL Alchemy features (transparent 
> handling of guid, free commit/rollback/transaction support, automatic 
> parent-children relation handling, etc)
> - pythonic interface for CRUD operations on a GnuCash Book
> 
> The last point is important as the goal is not at all to reimplement the 
> whole GnuCash engine but only to be able to manipulate (CRUD operations) the 
> different GnuCash entities (book, account, currency, etc) in an easy pythonic 
> way.
> For instance, I do not plan to reimplement functions like GetBalance, 
> GetReconciledBalance, GetPresentBalanceInCurrency, GetAccountBalance, etc
> 
> Is the goal of the project clearer ?
> Do you see complexities in reimplementing the pure CRUD operations (except 
> the basic complexities of leaving a GnuCash book in a consistent status) ?
> 
> On the name issue, you are definitely right ! I had in mind a pie-ess-cash 
> pronounciation (the S for SQL) as pygnucash was already taken. Any suggestion 
> ? pysacash (python sqlalchemy gnucash interface) ? or is it even worse :-) ?
> 
> kind regards,
> 
> Sebastien
> 
> ps: and thank you for clarifying the announce on the user mailing list ! I 
> should have thought about that myself to avoid confusion…

Sébastien,

Please remember to copy the list on all replies.

The goal of the project is clearer but is in my mind severely misguided. Object 
persistence is always a side effect of a non-trivial program, and GnuCash is a 
non-trivial program.

If your complaint about the python bindings is that they’re not pythonic I 
think that your efforts would be better spent improving that. It should become 
easier as we progress through the C++ rewrite which will produce a much more 
object-oriented architecture where that’s appropriate. That said, it’s also 
worth noting that C++ is a far more versatile language than either C or Python 
in that it supports generic and functional programming as well as the 
structured and OO paradigms that Python supports. In moving away from GObject 
we’ll also be enforcing type safety much more rigorously. Since that’s a 
notorious weakness (though casual hackers think it’s a feature) of Python, 
having thin Python wrappers around C++ objects will provide for much safer 
Python add ons than does the current code base.

It’s also very dangerous to attempt to select what parts of GnuCash’s internal 
functions to reimplement. To take your example of balances, it’s an integral 
part of GnuCa

Re: Working example of kvp acess in Python

[Christoph Holtermann]

Am 13.11.2014 um 18:55 schrieb Christoph Holtermann:
> Am 13.11.2014 um 17:50 schrieb Christian Stimming (mobil):
>> From my understanding, there are two separate issues here:
>>
>> 1. You are proposing a new object "Company" because you have reasons for 
>> needing new data fields. I think that's a good idea. Why don't you prepare a 
>> patch to extend the existing objects?
>>
>> 2. You ask for kvp access from python. No, we prefer not to have kvp 
>> accessed from python. Instead, either use gobject properties (on master), or 
>> add a new object on the C side as mentioned above and add python wrappers 
>> for this.
> Surely adding an object on the C side would be the more robust approach. Can 
> you tell me an object that already exists
> and that could be taken as an example ?
I'll try gncAddress as a start.
>
> I don't really understand the gobject concept though and what the difference 
> to my approach is.
>
> It seems some work until we have something we're all happy with, though ;-)
>> Regards, Christian
>> --
>> Sent from mobile. 
> regards,
>
> Christoph
>


-- 
--- Nachricht gesendet von C. Holtermann ---
-  -
-  Verschlüsselte Nachrichten können über  -
- den öffentlichen Schlüssel auf folgendem -
- Keyserver an mich gesendet werden:   -
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4DD9CF0482B0620B

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Working example of kvp acess in Python

[John Ralls]

On Nov 13, 2014, at 10:46 AM, Christoph Holtermann  wrote:

> Am 13.11.2014 um 18:55 schrieb Christoph Holtermann:
>> Am 13.11.2014 um 17:50 schrieb Christian Stimming (mobil):
>>> From my understanding, there are two separate issues here:
>>> 
>>> 1. You are proposing a new object "Company" because you have reasons for 
>>> needing new data fields. I think that's a good idea. Why don't you prepare 
>>> a patch to extend the existing objects?
>>> 
>>> 2. You ask for kvp access from python. No, we prefer not to have kvp 
>>> accessed from python. Instead, either use gobject properties (on master), 
>>> or add a new object on the C side as mentioned above and add python 
>>> wrappers for this.
>> Surely adding an object on the C side would be the more robust approach. Can 
>> you tell me an object that already exists
>> and that could be taken as an example ?
> I'll try gncAddress as a start.
>> 
>> I don't really understand the gobject concept though and what the difference 
>> to my approach is.
>> 
>> It seems some work until we have something we're all happy with, though ;-)

You needn't worry about GObject. We’re replacing it with C++, but we’re just 
starting out and it will be several months at least before we’re ready to start 
on the engine objects. If you must dive in now, sure, GncAddress should be a 
pretty good starting point, but I suggest reading up a bit on GObject [1] so 
that you have some understanding of what you’re doing. In particular, the 
memory management parts of almost all of the GObject-based classes in engine, 
including GncAddress, are implemented incorrectly. Make sure that you 
understand the right way to create and in particular destroy (the functions for 
the latter are dispose and finalize) your object and to use reference counting 
correctly to manage object lifetime. 

Your Company class implements only getters, which for your report-writing code 
is all that you need. The corresponding C/C++ class will need setters as well. 
In the current implementation where it writes to the QofBook KVP, that would be 
all.

Regards,
John Ralls

[1] https://developer.gnome.org/gobject/stable/
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[Sébastien de Menten]

On 2014-11-13 19:25, John Ralls wrote:

On Nov 13, 2014, at 9:31 AM, Sébastien de Menten  wrote:


Indeed, it may be worth to explain what are the goals (and the limits).

I have tried to use the official python bindings and had the following issues:
- need swig + compilations to make them work => pyscash is pure python and has 
only sqlalchemy as main dependency (which is rather supported/standard package)
- python binding is a mapping if C API with a thin layer python friendly layer 
=> I do not find the resulting python scripts very pythonic

[...]
So, to sum up, goals of pyscash :
- easy installation as pure python module (no compilation, no swig, ...)
- easy to contribute as pure python and based on de facto standard ORM for SQL 
in python (sql alchemy)
- pythonic implementation by leveraging SQL Alchemy features (transparent 
handling of guid, free commit/rollback/transaction support, automatic 
parent-children relation handling, etc)
- pythonic interface for CRUD operations on a GnuCash Book

The last point is important as the goal is not at all to reimplement the whole 
GnuCash engine but only to be able to manipulate (CRUD operations) the 
different GnuCash entities (book, account, currency, etc) in an easy pythonic 
way.
For instance, I do not plan to reimplement functions like GetBalance, 
GetReconciledBalance, GetPresentBalanceInCurrency, GetAccountBalance, etc

Is the goal of the project clearer ?
Do you see complexities in reimplementing the pure CRUD operations (except the 
basic complexities of leaving a GnuCash book in a consistent status) ?

On the name issue, you are definitely right ! I had in mind a pie-ess-cash 
pronounciation (the S for SQL) as pygnucash was already taken. Any suggestion ? 
pysacash (python sqlalchemy gnucash interface) ? or is it even worse :-) ?

kind regards,

Sebastien

ps: and thank you for clarifying the announce on the user mailing list ! I 
should have thought about that myself to avoid confusion…

Sébastien,

Please remember to copy the list on all replies.

The goal of the project is clearer but is in my mind severely misguided. Object 
persistence is always a side effect of a non-trivial program, and GnuCash is a 
non-trivial program.
If you see GnuCash from the (limited) perspective of an editor for a 
Book document (as LibreOffice Writer is an editor for a ODT document), 
object persistence is central. And luckily we have both a very clean and 
well designed object model in GnuCash as well as standard persistence 
formats (xml, SQL  and not an obscure binary format).


The GnuCash documentation states that the XML document can be used to 
get the GnuCash data in whatever other format (see "Converting XML 
GnuCash File" 
http://www.gnucash.org/docs/v2.6/C/gnucash-guide/appendixa_xmlconvert1.html) 
through XLST transformation and vice-versa (so generating a 
new/transformed gnucash XML file from a LibreOffice spreadsheet).


PieCash (I like your name !) aims to fulfill exactly this purpose, no 
more, no less. As with the XLST transforms, it allows to do CRUD 
operations on the objects within a GnuCash Book.


Has this stance on the manipulation outside GnuCash of a GnuCash 
document evolved since it was written ? Would this still be supported 
after the C++ rewrite ?

  your complaint about the python bindings is that they’re not pythonic I think 
that your efforts would be better spent improving that. It should become easier 
as we progress through the C++ rewrite which will produce a much more 
object-oriented architecture where that’s appropriate. That said, it’s also 
worth noting that C++ is a far more versatile language than either C or Python 
in that it supports generic and functional programming as well as the 
structured and OO paradigms that Python supports. In moving away from GObject 
we’ll also be enforcing type safety much more rigorously. Since that’s a 
notorious weakness (though casual hackers think it’s a feature) of Python, 
having thin Python wrappers around C++ objects will provide for much safer 
Python add ons than does the current code base.
The python binding are not pure python bindings (and will probably never 
be as they should interface C or C++ code). This makes them not easily 
accessible on Windows platforms (complexity in compilation) and more 
complex to hack (for non C/C++ programmers). However, they offer the 
ability to call any function of the engine and to be independent of the 
backend (XML, SQL,...) as long as it is interfaced so may be the only 
solution in some cases.


On the other side, PieCash is pure python. The code required to 
interface an entity is almost trivial. For instance, the full interface 
to the Commodity entity is


class Commodity(DeclarativeBaseGuid):
__tablename__ = 'commodities'
__table_args__ = {}

# column definitions
cusip = Column('cusip', TEXT(length=2048))
fraction = Column('fraction', INTEGER(), nullable=False)
fullname = Column('f

Submit a payment from somewhere not in the GUI...

[Allen S. Rout]

Is there any art for submitting payments or transactions from anywhere
other than clicking through the GUI?  I am attempting to automate the
submission of payments for our hackerspace from e.g. square, dwolla, etc.

I'd be perfectly delighted with any command-line invocation in which I
submit all of the required bits from the form.


- Allen S. Rout

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: New mailing list for Russian Translation discussions

[Dmitry Pavlov]

Seems reasonable. I'm not sure that there are a lot of users from Russia,
at least who wish to communicate via mailing lists :)

Ok, then I'm going to create mailing list at googlegroups and update wiki
page and ru.po file with it's address.

2014-11-13 18:41 GMT+03:00 John Ralls :

>
> > On Nov 13, 2014, at 4:27 AM, Dmitry Pavlov  wrote:
> >
> > Hi!
> >
> > Is it possible to create a new mailing list for discussing translations
> to
> > russian?
> >
> > The reason to do it: developer list is mostly related to code things and
> > currently there is no active list for translation discussions.
> >
> > As I don't participate in development it would be great to have specific
> > channel for translation related questions, and discussions.
> >
> > Current contact information about Translation group in ru.po file is
> > obsolete, mail is dead and as the last translator I propose to create new
> > list.
> >
> > This page  says there are
> already
> > a number of regional mailing list already like de, pt etc.
> >
> > If it'll take a lot of time, then I can create list at googlegroups with
> > updating Translation group information in po file.
>
> The Dutch, French, German, Italian, and Portuguese mailing lists are the
> equivalent in those languages of gnucash-user. If you think that there's
> enough interest in GnuCash in Russia to make such a list worthwhile and
> you're willing to moderate it and provide support to those users that way,
> great.
>
> Those lists are mentioned on the translation page because they're a ready
> source of native speakers of those languages who are also familiar with
> GnuCash who can help with interpretation and proof reading. Translation
> isn't their main focus. I suspect that a list devoted only to translation
> of a single language would be pretty lonely.
>
> Regards,
> John Ralls




-- 
С уважением,
Дмитрий Павлов
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: Invoices and Bill Reports

[JP Liew]

Thanks guys, exported the data to mysql and worked it out.




 From: JP Liew 
To: "gnucash-devel@gnucash.org"  
Sent: Thursday, November 13, 2014 9:06 PM
Subject: Invoices and Bill Reports
 


Hi all,

After searching through most of the old messages and online, I can't seems to 
find the right path to the information. Hopefully someone will be able to point 
me to the right path.


Basically I need to generate two monthly or quarterly reports, either in html 
or csv


1. Invoices Report that has the following fields

Date, Customer, Total, Subtotal, Tax

2. Bills Report that has the following fields

Date, Vendor, Total, Subtotal, Tax


Thank you so much.


Cheers
JP
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[John Ralls]

On Nov 13, 2014, at 12:44 PM, Sébastien de Menten  wrote:

> 
> On 2014-11-13 19:25, John Ralls wrote:
>> On Nov 13, 2014, at 9:31 AM, Sébastien de Menten  wrote:
>> 
>>> Indeed, it may be worth to explain what are the goals (and the limits).
>>> 
>>> I have tried to use the official python bindings and had the following 
>>> issues:
>>> - need swig + compilations to make them work => pyscash is pure python and 
>>> has only sqlalchemy as main dependency (which is rather supported/standard 
>>> package)
>>> - python binding is a mapping if C API with a thin layer python friendly 
>>> layer => I do not find the resulting python scripts very pythonic
>>> 
>>> [...]
>>> So, to sum up, goals of pyscash :
>>> - easy installation as pure python module (no compilation, no swig, ...)
>>> - easy to contribute as pure python and based on de facto standard ORM for 
>>> SQL in python (sql alchemy)
>>> - pythonic implementation by leveraging SQL Alchemy features (transparent 
>>> handling of guid, free commit/rollback/transaction support, automatic 
>>> parent-children relation handling, etc)
>>> - pythonic interface for CRUD operations on a GnuCash Book
>>> 
>>> The last point is important as the goal is not at all to reimplement the 
>>> whole GnuCash engine but only to be able to manipulate (CRUD operations) 
>>> the different GnuCash entities (book, account, currency, etc) in an easy 
>>> pythonic way.
>>> For instance, I do not plan to reimplement functions like GetBalance, 
>>> GetReconciledBalance, GetPresentBalanceInCurrency, GetAccountBalance, etc
>>> 
>>> Is the goal of the project clearer ?
>>> Do you see complexities in reimplementing the pure CRUD operations (except 
>>> the basic complexities of leaving a GnuCash book in a consistent status) ?
>>> 
>>> On the name issue, you are definitely right ! I had in mind a pie-ess-cash 
>>> pronounciation (the S for SQL) as pygnucash was already taken. Any 
>>> suggestion ? pysacash (python sqlalchemy gnucash interface) ? or is it even 
>>> worse :-) ?
>>> 
>>> kind regards,
>>> 
>>> Sebastien
>>> 
>>> ps: and thank you for clarifying the announce on the user mailing list ! I 
>>> should have thought about that myself to avoid confusion…
>> Sébastien,
>> 
>> Please remember to copy the list on all replies.
>> 
>> The goal of the project is clearer but is in my mind severely misguided. 
>> Object persistence is always a side effect of a non-trivial program, and 
>> GnuCash is a non-trivial program.
> If you see GnuCash from the (limited) perspective of an editor for a Book 
> document (as LibreOffice Writer is an editor for a ODT document), object 
> persistence is central. And luckily we have both a very clean and well 
> designed object model in GnuCash as well as standard persistence formats 
> (xml, SQL  and not an obscure binary format).

I don’t share either opinion, especially about the object model in GnuCash. 

> 
> The GnuCash documentation states that the XML document can be used to get the 
> GnuCash data in whatever other format (see "Converting XML GnuCash File" 
> http://www.gnucash.org/docs/v2.6/C/gnucash-guide/appendixa_xmlconvert1.html) 
> through XLST transformation and vice-versa (so generating a new/transformed 
> gnucash XML file from a LibreOffice spreadsheet).

No matter what the GnuCash documentation might say, only someone with a deep 
understanding of GnuCash could successfully create a correct GnuCash data file 
from a spreadsheet document using XSLT. I’m not familiar with LibreOffice’s ODS 
format, but I expect that it would take deep knowledge of that format to 
successfully extract useful data from it with XSLT as well.

> 
> PieCash (I like your name !) aims to fulfill exactly this purpose, no more, 
> no less. As with the XLST transforms, it allows to do CRUD operations on the 
> objects within a GnuCash Book.

CRUD operations on a GnuCash database will corrupt it. The GnuCash schema is 
not normalized, and not all of the necessary data is stored in the table 
associated with the objects.

> 
> Has this stance on the manipulation outside GnuCash of a GnuCash document 
> evolved since it was written ? Would this still be supported after the C++ 
> rewrite ?

It isn’t supported now. It never has, and it is unlikely that it ever will be, 
even if we are able, after several development cycles, to actually migrate to a 
3N database schema. There is too much logic that is encoded in the program and 
which cannot be portably encoded in a SQL database, to make that feasible.

>>  your complaint about the python bindings is that they’re not pythonic I 
>> think that your efforts would be better spent improving that. It should 
>> become easier as we progress through the C++ rewrite which will produce a 
>> much more object-oriented architecture where that’s appropriate. That said, 
>> it’s also worth noting that C++ is a far more versatile language than either 
>> C or Python in that it supports generic and functional programming as well 
>>

Re: Submit a payment from somewhere not in the GUI...

[John Ralls]

On Nov 13, 2014, at 1:15 PM, Allen S. Rout  wrote:

> 
> Is there any art for submitting payments or transactions from anywhere
> other than clicking through the GUI?  I am attempting to automate the
> submission of payments for our hackerspace from e.g. square, dwolla, etc.
> 
> I'd be perfectly delighted with any command-line invocation in which I
> submit all of the required bits from the form.
> 

No. You’ll need to script your interaction in Scheme or Python, or write a 
separate application that generates something like QIF or OFX that GnuCash can 
import.

Regards,
John Ralls


___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[Derek Atkins]

Sébastien de Menten  writes:

> Where could I find detailed documentation on the GnuCash engine (and the
> constrains/invariants GnuCash enforces) ? Or would there be some
> code/program to check a GnuCash file is "sane/consistent" ?

Only in the implementation.  There is no documentation per se on this,
because we do not support modification of the database from outside the
GnuCash APIs.  This also allows us to change the underlying storage
mechanisms without breaking things, because it's all abstracted.  By
re-implementing it you're basically binding yourself to a particular
version of the database schema, which can (and will) change over time,
requiring you to duplicate the effort already happening in the gnucash
code.

Moreover, you're also tied to a particular backend, which isn't very
nice.

If you want to modify the gnucash database, you really should use the
exported GnuCash APIs.

If the current python bindings aren't pythonic enough for you, then I
urge you to spend the time to fix that instead of reimplementing
something that will absolutely break some time in the future.  When it
will break I cannot tell you, but I can assure you it WILL break at some
point.

> Regards
>
> Sebastien

-derek

-- 
   Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
   Member, MIT Student Information Processing Board  (SIPB)
   URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH
   warl...@mit.eduPGP key available

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: python GnuCash interface to SQL backend

[Derek Atkins]

John Ralls  writes:

> What’s your goal here? I don’t think that reimplementing GnuCash in
> Python with GnuCash’s SQL schema is a particularly good approach: It’s
> not exactly the most efficient design. Rather, it’s designed to mirror
> the XML schema. You’ll have a better design if you relegate GnuCash
> SQL to import/export.

I agree wholeheartedly.  Don't do it this way.  Use the GnuCash Python
bindings.  If you don't like the current structure of them, then fix
that.  This way your apps will always work because the bindings will
stay in lockstep with any changes that get made.

> An aside about the name: Pyscash is likely to be pronounced by English
> speakers with a short “i” sound where the y goes.  That conveys a
> rather unfortunate meaning.

Indeed, I would pronounce is "piss cash".

> Regards,
> John Ralls

-derek
-- 
   Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
   Member, MIT Student Information Processing Board  (SIPB)
   URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH
   warl...@mit.eduPGP key available

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: New mailing list for Russian Translation discussions

[Derek Atkins]

You should look to the translation project if you want a list purely for
translation purposes.

If you want a Russian Language gnucash-user equivalent then we can work
with you to set one up, as John suggested.

-derek

Dmitry Pavlov  writes:

> Seems reasonable. I'm not sure that there are a lot of users from Russia,
> at least who wish to communicate via mailing lists :)
>
> Ok, then I'm going to create mailing list at googlegroups and update wiki
> page and ru.po file with it's address.
>
> 2014-11-13 18:41 GMT+03:00 John Ralls :
>
>>
>> > On Nov 13, 2014, at 4:27 AM, Dmitry Pavlov  wrote:
>> >
>> > Hi!
>> >
>> > Is it possible to create a new mailing list for discussing translations
>> to
>> > russian?
>> >
>> > The reason to do it: developer list is mostly related to code things and
>> > currently there is no active list for translation discussions.
>> >
>> > As I don't participate in development it would be great to have specific
>> > channel for translation related questions, and discussions.
>> >
>> > Current contact information about Translation group in ru.po file is
>> > obsolete, mail is dead and as the last translator I propose to create new
>> > list.
>> >
>> > This page  says there are
>> already
>> > a number of regional mailing list already like de, pt etc.
>> >
>> > If it'll take a lot of time, then I can create list at googlegroups with
>> > updating Translation group information in po file.
>>
>> The Dutch, French, German, Italian, and Portuguese mailing lists are the
>> equivalent in those languages of gnucash-user. If you think that there's
>> enough interest in GnuCash in Russia to make such a list worthwhile and
>> you're willing to moderate it and provide support to those users that way,
>> great.
>>
>> Those lists are mentioned on the translation page because they're a ready
>> source of native speakers of those languages who are also familiar with
>> GnuCash who can help with interpretation and proof reading. Translation
>> isn't their main focus. I suspect that a list devoted only to translation
>> of a single language would be pretty lonely.
>>
>> Regards,
>> John Ralls

-- 
   Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
   Member, MIT Student Information Processing Board  (SIPB)
   URL: http://web.mit.edu/warlord/PP-ASEL-IA N1NWH
   warl...@mit.eduPGP key available
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel