Sorry it's been a rather crazy time and I don't remember. There are several nice importers in github.com/jbms/beancount-import in addition to a nice webUI for merging / reviewing postings. beancount-import supports using beancount native importers, so in a sense the merging I described is already there in one direction.
On Tue, Nov 16, 2021 at 2:14 PM Bman Q <mplo...@gmail.com> wrote: > "I'm excited for Beancount v3's importer changes, it will hopefully mean > that we can all merge our efforts on importers into one compatible format. > I'd be happy to collaborate with you on design for that Martin." > -could you please explain what changes are you talking about? (i am using > v2 and didn't look into v3 yet) > > On Sunday, February 7, 2021 at 12:55:56 PM UTC-5 b...@bben.us wrote: > >> > Importing is time consuming >> I use github.com/jbms/beancount-import and am quite happy with it for >> the import UI, transaction dedupe logic, transaction clearing, and ML-based >> categorization. Admittedly I've spent tens of hours programming importers >> and scrapers and possibly have not made any net time savings. But it was a >> fun project! >> I signed up for a plaid developer account and wrote a small script to >> download transactions. Then I wrote a beancount-import source for it. >> I'm excited for Beancount v3's importer changes, it will hopefully mean >> that we can all merge our efforts on importers into one compatible format. >> I'd be happy to collaborate with you on design for that Martin. >> >> > Journal file being too large to navigate >> I share this frustration, though mine is more of a frustration that it's >> difficult to keep it organized in any kind of sane way. Beancount v3's >> ability to rewrite the ledger file in code will hopefully make it trivial >> to make a beancount formatter, which is super exciting. >> The beancount emacs plugin is /by far/ the best transaction entry system >> I've used so far. I too am a longtime vim user and have been learning doom >> emacs for a few months. So far I am thrilled with it and am increasingly >> using it full-time. I don't think it'll replace Jetbrains IDEs for heavy >> coding but it's replacing all my adhoc editing / light coding needs. >> I am using this forked version >> <https://github.com/cnsunyour/beancount.el> of the beancount emacs >> plugin since I am currently using jbms/beancount-import's journal format >> which splits accounts and transactions into separate files. I haven't >> learned elisp yet or I'd consider trying to get this patch upstreamed. I >> should probably adopt the 'one journal file' approach instead. >> >> > petl, visidata, data frames and beancount queries >> I love Martin's principles around promoting interop with powerful >> recomposable tools like this. I look forward to seeing this workflow >> realized and collaborating to make a new cookbook of useful queries. >> >> On Sun, Feb 7, 2021 at 9:28 AM Dustin Farris <dustin...@gmail.com> wrote: >> >>> Wow, thanks for the fast and thorough response. It is very encouraging, >>> and really what I needed for my mental health haha. >>> >>> A couple reactions at random: >>> >>> Yodlee, or Plaid >>> >>> ๐๐๐ will check these out >>> >>> >>> Sure, you can split. It won't make it much faster though. >>> >>> I am excited about performance coming with the C++ rewrite, but this was >>> not so much about addressing performance as my ability to find things. >>> (but to be clear). The headers Iโve created in my journal are >>> per-institution-account (e.g. fidelity 401k) with sub-headers by year and >>> then by month. These headers with their folding in vim (similar to emacs >>> outline mode) are helpful, but I think I can do better. It is also a >>> less-than-nice experience for on the fly entries such as invoices, tracking >>> kidsโ allowance, and so on. I need to just set aside a few hours to ponder >>> this and come up with the right organization and workflow. >>> >>> >>> There is a final stage of awakening called "ace-jump-mode", kept as a >>> closely guarded secret >>> >>> Is there a vim equivalent of nirvana? :P Because of beancount, I gave >>> emacs+orgmode a real try โ a month with spacemacs, and a month with doom, >>> but ultimately reverted back to what I know (vim). Iโm hoping to get my >>> beancount vim+VSCode experience leveled up. >>> >>> >>> new rewriting capabilities will make it possible to rewrite the actual >>> file by changing the AST and writing it back out. >>> >>> This sounds really exciting. >>> >>> >>> in the next version. With a more generic SQL or providing a library that >>> works with petl >>> >>> Being able to reason about my financials using dataframes sounds >>> empowering. Looking forward to it. >>> >>> >>> -- >>> Dustin Farris >>> (646) 671-2007 >>> >>> >>> >>> On Feb 7, 2021, at 8:06 AM, Martin Blais <bl...@furius.ca> wrote: >>> >>> On Sun, Feb 7, 2021 at 10:24 AM Dustin Farris <dustin...@gmail.com> >>> wrote: >>> >>>> I've been using beancount for a little over a year to manage my >>>> personal finances as well as some side self-employment accounting. I have >>>> twice gotten frustrated to the point of trying other products (specifically >>>> Personal Capital, Mint, Quicken, and QuickBooks) but give up and come back >>>> to command-line accounting for reasons probably familiar to everyone in >>>> this group. >>>> >>>> So here are my frustrations and what I'm going to attempt to do for >>>> myself to address them. I'm mainly venting here, but if anyone has any >>>> suggestions I'd really appreciate them. >>>> >>>> >>>> - I update my journal every month. Getting updated transactions >>>> from 25+ different accounts every month is very time consuming. >>>> - Todo: Research programmatic downloads of transactions from all >>>> banks (has this been done already?) >>>> >>>> Look for Yodlee, or Plaid. >>> If you're willing to give up your passwords, those could save you time >>> by lumping together a bunch of imports. >>> (I tend to only update the most important ones - checking, credit card, >>> investing - frequently, and the others lag behind a bit.) >>> >>> Also, there are simplifications coming up to the importing framework >>> which is going to make it substantially simpler to use, though I'm not sure >>> if those changes will address your specific needs. >>> >>> >>>> >>>> - The beancount file is getting overwhelmingly large after just 1 >>>> year. This is making it hard for me to jump around and find/fix >>>> things. I >>>> often have personal transactions that cross equity accounts into our >>>> rental >>>> business, or my software engineering self employment. >>>> - Todo: split personal.beancount into smaller journals (by >>>> month?) >>>> >>>> Sure, you can split. It won't make it much faster though. (The ongoing >>> rewrite in C++ is going to address that.) >>> >>> I use outline-minor-mode. Another clever trick is to realize that like >>> in life, you only ever go to a few places in that file - depending how you >>> organize it - and insert unique comment tags and just i-search for them to >>> go there directly (*). Huge time saver. The new import framework might even >>> auto-insert in front of those tags. >>> >>> I think if you have the patience, you could refine your previous >>> transaction detection so that it's nearly flawless, automatically >>> categorize your transactions, and implement auto-insert in your file and >>> trust it. Would save some time, but it'll be time-consuming to get it right >>> IMO (you'll spend the time coding instead of updating; it's unclear to me >>> in that situation which will be better). beangulp will also provide more >>> hooks for you to refine previous-import-detection, even per importer. (For >>> more on upcoming changes on that look here >>> https://docs.google.com/document/d/1O42HgYQBQEna6YpobTqszSgTGnbRX7RdjmzR2xumfjs/edit >>> ) >>> >>> >>> (*) In fact, I would say if you're using emacs this is the fourth stage >>> of evolution of Emacs users about cursor movement: first you use the cursor >>> keys, then you learn to use the relative word and paragraph movement, then >>> you graduate to sexp movements (e.g. beginning of function), and when you >>> nearly achieve enlightenment you are nearly always moving via >>> interactive-search ( >>> https://www.masteringemacs.org/article/effective-editing-movement). >>> There is a final stage of awakening called "ace-jump-mode", kept as a >>> closely guarded secret by those of use experiencing a permanent state of >>> non-duality ;-). >>> >>> >>> >>>> >>>> - omni-complete in vim is an awkward keyboard chord and account >>>> completions in vim are sometimes broken depending on what you last typed >>>> - Todo: try VSCode which can now embed neovim and has it's own >>>> beancount extension that should make the editing experience nicer >>>> - except VSCode folding is weird so this might actually make >>>> things worse >>>> - I still donโt understand how reporting works, and part of that >>>> is because I don't use it enough. I feel like I'm relearning Beancount >>>> Query Language every time I do need something. >>>> - Todo: keep a list of commonly-used queries >>>> >>>> Write your queries as code, and run your own scripts. There's a neat >>> API. >>> You can also use the Query directive to save pre-made queries in your >>> Beancount file and execute them by name. >>> >>> Easier reporting is something I'd like to refine in the next version. >>> With a more generic SQL or providing a library that works with petl. >>> >>> >>> >>>> - reconciling receipts / splitting transactions is time consuming, >>>> although vim macros help here >>>> - Todo: Improve importer recognition of payees and likely >>>> expense accounts >>>> >>>> Yep. >>> >>> (v3's new parser will bring another dimension to this: instead of having >>> to write code to fixup the payees in memory only, the new rewriting >>> capabilities will make it possible to rewrite the actual file by changing >>> the AST and writing it back out.) >>> >>> >>>> >>>> - no way to attach receipt pictures to transactions? >>>> - Todo: Research beancount tooling or other apps to capture >>>> receipts >>>> >>>> See the ingest framework's filling ability. It renames files that have >>> been identified prepending a date and can stash them in a repository under >>> the same account hierarchy. bean-web used to serve those directives right >>> next to the transactions and you could insert a common link to link them >>> together (or write a plugin that attempts to match them up automatically) >>> >>> >>>> >>>> - fava is nice, but could be nicer >>>> - Todo: Research writing a new frontend reporting/visualization >>>> tool and/or contribute to fava >>>> >>>> Go for it! >>> Custom one-off renderings are also useful (e.g. like in beangrow). >>> >>> >>> >>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Beancount" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to beancount+...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com >>>> <https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Beancount" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to beancount+...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com >>> <https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Beancount" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to beancount+...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com >>> <https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to beancount+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/beancount/da9b922d-fa61-4eec-8ab6-e0cb7b55fc5dn%40googlegroups.com > <https://groups.google.com/d/msgid/beancount/da9b922d-fa61-4eec-8ab6-e0cb7b55fc5dn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CACGEkZsRAaU2pzsyeLyE1CJD7r%3DzuyYJ9yed_FJGdJJc3SphNg%40mail.gmail.com.
