FYI, I have importers for banks and credit cards that support ofx, based on my beancount_reds_importers library. I'm happy to add that to my python package that you're using if there's interest. I wrap it with smart importer, which is very well designed and written, and gets it right well over 90% of the time.
You're right: I've found that building dependable automation around my import/ingest made took that part from a 3 hour task to a 3min task, and was a key piece in making my beancount/fava experience fruitful. This is why I get my imports to extract balances and prices and convert them to balance check and price entries. Taking automation a step further, I also have some scripts to auto-download my statements for a few banks/cards/brokerages that support quicken direct connect. This automates the downloading step as well, and stores your passwords securely (encrypted with a master password). On Thursday, June 25, 2020 at 10:21:58 PM UTC-7, Jonathan Goldman wrote: > > Thanks this is very helpful. I was not aware of the smart importer and > will try it out. And I’ll check out beancounttools. Thanks! > > On Thu, 25 Jun 2020 at 5:41 PM, 'Patrick Ruckstuhl' via Beancount < > bean...@googlegroups.com <javascript:>> wrote: > >> Hi Jonathan, >> >> Q1 >> >> I would say whenever you have both (or more) sides of a transactions then >> it's the best to import all legs. The thing is that in most cases you don't >> have them. On thing that I also do quite often is, that if I can import >> both sides of a transaction form two different sources (e.g. two different >> banks), then I use the transaction ids of each bank to add them to the same >> transaction, so I know that I have it imported from both sides. >> >> >> Q2 >> >> Not sure if you've come across smart importer >> https://github.com/beancount/smart_importer, this is a wrapper around >> your importer that uses existing transactions to guess what the missing >> other leg could be. For me this works extremely well. >> >> I use it together with the web ui fava https://github.com/beancount/fava >> >> when you then run an importer you get basically the list of all your >> transactions and you can still modify if the smart importer guess wrong. >> >> What I also started doing more and more (and also works quite nicely), is >> to actually not even have a file to import but instead directly use an >> available api (e.g. truelayer, revolut, transferwise, interactive brokers, >> ...), you can see some examples in my beancounttools >> https://github.com/tarioch/beancounttools >> >> >> Q3 >> >> There were some enhancements done for hooks: >> https://github.com/beancount/beancount/issues/458 >> >> >> Regards, >> >> Patrick >> >> >> >> On 25.06.2020 07:15, Jonathan Goldman wrote: >> >> Hi, >> >> I have done a bunch of exploration of beancount over the last few months >> and really like its capabilities and now ready to get serious about getting >> it working for me. So far this is what I have done: >> >> >> - Used the sample files to familiarize myself with the commands and >> beancount's capabilities. >> - Created accounts and manually entered some real data for my >> personal finances. >> - Tried this <https://github.com/jbms/beancount-import> >> sem-auto-categorization web-based import tool. It's good but I think I'll >> mostly go the manual route. >> - Tried various import methods >> - Tried importing a quicken file...have had some trouble with this. >> - Tried a csv2ofx <https://github.com/reubano/csv2ofx> tool >> - Used reds_importers >> >> <https://github.com/redstreet/beancount_reds_importers/blob/master/README.md> >> >> for Vanguard. This has worked successfully. This generates output that >> has >> two-sided. >> - Wrote my own csv importer for my bank that is based on this csv >> >> <https://gist.github.com/mterwill/7fdcc573dc1aa158648aacd4e33786e8#file-importers-chase-py> >> >> importer.. (I'm not a software engineer by training so this took me >> some >> time but I'm very happy with this now and understand the code much >> more). >> This generates output that is only one-sided. >> >> My own conclusion is that I want go the route suggested by Martin and >> effectively have my own importers for each source and with text-based >> editor with auto-completion do the categorization. For the brokerages there >> are a limited number of accounts/categories you can have so doing the >> two-sided entry can be done here so it is auto-categorization if you will. >> (e.g. dividends from account xyz always go to this income account, etc). >> For credit cards and banks it rarely helps as there are so many expense >> categories. Perhaps a subset (e.g. interest income from the bank can be >> autocategorized or mortgage payment of specific amount) can be automated. >> >> *Q1. Is doing two-side entry for brokerages and single-side for >> banks/credit cards a good approach the general practice that most here >> use? * >> *Q2. Is there other import approaches to consider? I think I'm finding >> this the hardest part to get going on and that is why other tools like >> Quicken/Mint/GnuCash are easier to get going. So not sure how, but perhaps >> data collection and import is something already solved that is not >> beancount specific?* >> >> I also noticed that Martin mentioned this in the import documentation >> <https://docs.google.com/document/d/11EwQdujzEo2cxqaF5PgxCEZXWfKKQCYSMfdJowp_1S8/edit#> >> : >> >>> A list of things I’d really want to add, beyond fortifying what’s >>> already there: >>> >>> - >>> >>> A generic, configurable CSV importer which you can instantiate. I >>> plan to play with this a bit and build a sniffer that could >>> automatically >>> figure out the role of each column. >>> - >>> >>> A hook to allow you to register a callback for post-processing >>> transactions that works across all importers. >>> >>> >> *Q3. Is there any update/development on these two items?* >> >> thanks, >> Jonathan >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to bean...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/b2b52ac6-b019-4375-adcb-e3f7e6ab0c3bo%40googlegroups.com >> >> <https://groups.google.com/d/msgid/beancount/b2b52ac6-b019-4375-adcb-e3f7e6ab0c3bo%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to bean...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/10207a15-f6f1-d6d6-994c-33e9520e2c83%40ch.tario.org >> >> <https://groups.google.com/d/msgid/beancount/10207a15-f6f1-d6d6-994c-33e9520e2c83%40ch.tario.org?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/12d98f89-eb7d-49d9-98f1-a0ec65a794dco%40googlegroups.com.
