I'm excited for Beancount v3's importer changes, it will hopefully mean that we can all merge our efforts on importers into one compatible format. -Ben hi, Could you please tell what importer changes are you talking about? (i am v2 user, not yet familiar with v3)
On Sunday, February 7, 2021 at 12:55:56 PM UTC-5 b...@bben.us wrote: > > Importing is time consuming > I use github.com/jbms/beancount-import and am quite happy with it for > the import UI, transaction dedupe logic, transaction clearing, and ML-based > categorization. Admittedly I've spent tens of hours programming importers > and scrapers and possibly have not made any net time savings. But it was a > fun project! > I signed up for a plaid developer account and wrote a small script to > download transactions. Then I wrote a beancount-import source for it. > I'm excited for Beancount v3's importer changes, it will hopefully mean > that we can all merge our efforts on importers into one compatible format. > I'd be happy to collaborate with you on design for that Martin. > > > Journal file being too large to navigate > I share this frustration, though mine is more of a frustration that it's > difficult to keep it organized in any kind of sane way. Beancount v3's > ability to rewrite the ledger file in code will hopefully make it trivial > to make a beancount formatter, which is super exciting. > The beancount emacs plugin is /by far/ the best transaction entry system > I've used so far. I too am a longtime vim user and have been learning doom > emacs for a few months. So far I am thrilled with it and am increasingly > using it full-time. I don't think it'll replace Jetbrains IDEs for heavy > coding but it's replacing all my adhoc editing / light coding needs. > I am using this forked version <https://github.com/cnsunyour/beancount.el> of > the beancount emacs plugin since I am currently using > jbms/beancount-import's journal format which splits accounts and > transactions into separate files. I haven't learned elisp yet or I'd > consider trying to get this patch upstreamed. I should probably adopt the > 'one journal file' approach instead. > > > petl, visidata, data frames and beancount queries > I love Martin's principles around promoting interop with powerful > recomposable tools like this. I look forward to seeing this workflow > realized and collaborating to make a new cookbook of useful queries. > > On Sun, Feb 7, 2021 at 9:28 AM Dustin Farris <dustin...@gmail.com> wrote: > >> Wow, thanks for the fast and thorough response. It is very encouraging, >> and really what I needed for my mental health haha. >> >> A couple reactions at random: >> >> Yodlee, or Plaid >> >> ๐๐๐ will check these out >> >> >> Sure, you can split. It won't make it much faster though. >> >> I am excited about performance coming with the C++ rewrite, but this was >> not so much about addressing performance as my ability to find things. >> (but to be clear). The headers Iโve created in my journal are >> per-institution-account (e.g. fidelity 401k) with sub-headers by year and >> then by month. These headers with their folding in vim (similar to emacs >> outline mode) are helpful, but I think I can do better. It is also a >> less-than-nice experience for on the fly entries such as invoices, tracking >> kidsโ allowance, and so on. I need to just set aside a few hours to ponder >> this and come up with the right organization and workflow. >> >> >> There is a final stage of awakening called "ace-jump-mode", kept as a >> closely guarded secret >> >> Is there a vim equivalent of nirvana? :P Because of beancount, I gave >> emacs+orgmode a real try โ a month with spacemacs, and a month with doom, >> but ultimately reverted back to what I know (vim). Iโm hoping to get my >> beancount vim+VSCode experience leveled up. >> >> >> new rewriting capabilities will make it possible to rewrite the actual >> file by changing the AST and writing it back out. >> >> This sounds really exciting. >> >> >> in the next version. With a more generic SQL or providing a library that >> works with petl >> >> Being able to reason about my financials using dataframes sounds >> empowering. Looking forward to it. >> >> >> -- >> Dustin Farris >> (646) 671-2007 >> >> >> >> On Feb 7, 2021, at 8:06 AM, Martin Blais <bl...@furius.ca> wrote: >> >> On Sun, Feb 7, 2021 at 10:24 AM Dustin Farris <dustin...@gmail.com> >> wrote: >> >>> I've been using beancount for a little over a year to manage my personal >>> finances as well as some side self-employment accounting. I have twice >>> gotten frustrated to the point of trying other products (specifically >>> Personal Capital, Mint, Quicken, and QuickBooks) but give up and come back >>> to command-line accounting for reasons probably familiar to everyone in >>> this group. >>> >>> So here are my frustrations and what I'm going to attempt to do for >>> myself to address them. I'm mainly venting here, but if anyone has any >>> suggestions I'd really appreciate them. >>> >>> >>> - I update my journal every month. Getting updated transactions >>> from 25+ different accounts every month is very time consuming. >>> - Todo: Research programmatic downloads of transactions from all >>> banks (has this been done already?) >>> >>> Look for Yodlee, or Plaid. >> If you're willing to give up your passwords, those could save you time by >> lumping together a bunch of imports. >> (I tend to only update the most important ones - checking, credit card, >> investing - frequently, and the others lag behind a bit.) >> >> Also, there are simplifications coming up to the importing framework >> which is going to make it substantially simpler to use, though I'm not sure >> if those changes will address your specific needs. >> >> >>> >>> - The beancount file is getting overwhelmingly large after just 1 >>> year. This is making it hard for me to jump around and find/fix things. >>> I >>> often have personal transactions that cross equity accounts into our >>> rental >>> business, or my software engineering self employment. >>> - Todo: split personal.beancount into smaller journals (by month?) >>> >>> Sure, you can split. It won't make it much faster though. (The ongoing >> rewrite in C++ is going to address that.) >> >> I use outline-minor-mode. Another clever trick is to realize that like in >> life, you only ever go to a few places in that file - depending how you >> organize it - and insert unique comment tags and just i-search for them to >> go there directly (*). Huge time saver. The new import framework might even >> auto-insert in front of those tags. >> >> I think if you have the patience, you could refine your previous >> transaction detection so that it's nearly flawless, automatically >> categorize your transactions, and implement auto-insert in your file and >> trust it. Would save some time, but it'll be time-consuming to get it right >> IMO (you'll spend the time coding instead of updating; it's unclear to me >> in that situation which will be better). beangulp will also provide more >> hooks for you to refine previous-import-detection, even per importer. (For >> more on upcoming changes on that look here >> https://docs.google.com/document/d/1O42HgYQBQEna6YpobTqszSgTGnbRX7RdjmzR2xumfjs/edit >> ) >> >> >> (*) In fact, I would say if you're using emacs this is the fourth stage >> of evolution of Emacs users about cursor movement: first you use the cursor >> keys, then you learn to use the relative word and paragraph movement, then >> you graduate to sexp movements (e.g. beginning of function), and when you >> nearly achieve enlightenment you are nearly always moving via >> interactive-search ( >> https://www.masteringemacs.org/article/effective-editing-movement). >> There is a final stage of awakening called "ace-jump-mode", kept as a >> closely guarded secret by those of use experiencing a permanent state of >> non-duality ;-). >> >> >> >>> >>> - omni-complete in vim is an awkward keyboard chord and account >>> completions in vim are sometimes broken depending on what you last typed >>> - Todo: try VSCode which can now embed neovim and has it's own >>> beancount extension that should make the editing experience nicer >>> - except VSCode folding is weird so this might actually make >>> things worse >>> - I still donโt understand how reporting works, and part of that >>> is because I don't use it enough. I feel like I'm relearning Beancount >>> Query Language every time I do need something. >>> - Todo: keep a list of commonly-used queries >>> >>> Write your queries as code, and run your own scripts. There's a neat API. >> You can also use the Query directive to save pre-made queries in your >> Beancount file and execute them by name. >> >> Easier reporting is something I'd like to refine in the next version. >> With a more generic SQL or providing a library that works with petl. >> >> >> >>> - reconciling receipts / splitting transactions is time consuming, >>> although vim macros help here >>> - Todo: Improve importer recognition of payees and likely expense >>> accounts >>> >>> Yep. >> >> (v3's new parser will bring another dimension to this: instead of having >> to write code to fixup the payees in memory only, the new rewriting >> capabilities will make it possible to rewrite the actual file by changing >> the AST and writing it back out.) >> >> >>> >>> - no way to attach receipt pictures to transactions? >>> - Todo: Research beancount tooling or other apps to capture >>> receipts >>> >>> See the ingest framework's filling ability. It renames files that have >> been identified prepending a date and can stash them in a repository under >> the same account hierarchy. bean-web used to serve those directives right >> next to the transactions and you could insert a common link to link them >> together (or write a plugin that attempts to match them up automatically) >> >> >>> >>> - fava is nice, but could be nicer >>> - Todo: Research writing a new frontend reporting/visualization >>> tool and/or contribute to fava >>> >>> Go for it! >> Custom one-off renderings are also useful (e.g. like in beangrow). >> >> >> >> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Beancount" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to beancount+...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to beancount+...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to beancount+...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com >> >> <https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/d8f4f5a3-ae5f-4f6b-8e68-6e363590923en%40googlegroups.com.
