I agree with Blais that there really isn't a better solution to all of this. I think there is a misconception that because it is conceivable that all of your transactions can be automated (and they are for individual banks & institutions) that it should be easy to automate or that there is already a solution. It sucks to have to do it all manually, but the pain of entering hundreds of "Expenses:Coffee 3.00 USD" transactions now makes me think twice about it! (or to just use cash and book them all in a single pad entry :P) It also has really made me appreciate the complexity of some financial instruments, which in my case has made me less afraid of them since I know that I can keep track of them all as I want to start doing crazy things to make money :)
I really can't imagine having to do all of the importers myself and maintain them since the banks are always changing and no two are alike. I think its just easier to do by hand as of now since I know after working a few hours I will have a product, rather than spending a few hours automating and it just sucks at the end anyways. I would definitely pay someone to provide up to date importers and a UI for quickly categorizing them (since the banks never get this right for my purposes). I have heard of people outsourcing their bookkeeping to traditional book keepers (I'm guessing they only do mainstream software like Quick Books) and also to overseas virtual assistants which will do anything you pay them for. I am betting that if you find one you can work with long enough that they would be able to do most of the drudge work if you just give them your statements. On Tuesday, November 16, 2021 at 5:30:14 PM UTC-5 b...@bben.us wrote: > Sorry it's been a rather crazy time and I don't remember. > There are several nice importers in github.com/jbms/beancount-import in > addition to a nice webUI for merging / reviewing postings. beancount-import > supports using beancount native importers, so in a sense the merging I > described is already there in one direction. > > On Tue, Nov 16, 2021 at 2:14 PM Bman Q <mpl...@gmail.com> wrote: > >> "I'm excited for Beancount v3's importer changes, it will hopefully mean >> that we can all merge our efforts on importers into one compatible format. >> I'd be happy to collaborate with you on design for that Martin." >> > -could you please explain what changes are you talking about? (i am using >> v2 and didn't look into v3 yet) >> > >> On Sunday, February 7, 2021 at 12:55:56 PM UTC-5 b...@bben.us wrote: >> >>> > Importing is time consuming >>> I use github.com/jbms/beancount-import and am quite happy with it for >>> the import UI, transaction dedupe logic, transaction clearing, and ML-based >>> categorization. Admittedly I've spent tens of hours programming importers >>> and scrapers and possibly have not made any net time savings. But it was a >>> fun project! >>> I signed up for a plaid developer account and wrote a small script to >>> download transactions. Then I wrote a beancount-import source for it. >>> I'm excited for Beancount v3's importer changes, it will hopefully mean >>> that we can all merge our efforts on importers into one compatible format. >>> I'd be happy to collaborate with you on design for that Martin. >>> >>> > Journal file being too large to navigate >>> I share this frustration, though mine is more of a frustration that it's >>> difficult to keep it organized in any kind of sane way. Beancount v3's >>> ability to rewrite the ledger file in code will hopefully make it trivial >>> to make a beancount formatter, which is super exciting. >>> The beancount emacs plugin is /by far/ the best transaction entry system >>> I've used so far. I too am a longtime vim user and have been learning doom >>> emacs for a few months. So far I am thrilled with it and am increasingly >>> using it full-time. I don't think it'll replace Jetbrains IDEs for heavy >>> coding but it's replacing all my adhoc editing / light coding needs. >>> I am using this forked version >>> <https://github.com/cnsunyour/beancount.el> of the beancount emacs >>> plugin since I am currently using jbms/beancount-import's journal format >>> which splits accounts and transactions into separate files. I haven't >>> learned elisp yet or I'd consider trying to get this patch upstreamed. I >>> should probably adopt the 'one journal file' approach instead. >>> >>> > petl, visidata, data frames and beancount queries >>> I love Martin's principles around promoting interop with powerful >>> recomposable tools like this. I look forward to seeing this workflow >>> realized and collaborating to make a new cookbook of useful queries. >>> >>> On Sun, Feb 7, 2021 at 9:28 AM Dustin Farris <dustin...@gmail.com> >>> wrote: >>> >>>> Wow, thanks for the fast and thorough response. It is very >>>> encouraging, and really what I needed for my mental health haha. >>>> >>>> A couple reactions at random: >>>> >>>> Yodlee, or Plaid >>>> >>>> ๐๐๐ will check these out >>>> >>>> >>>> Sure, you can split. It won't make it much faster though. >>>> >>>> I am excited about performance coming with the C++ rewrite, but this >>>> was not so much about addressing performance as my ability to find things. >>>> (but to be clear). The headers Iโve created in my journal are >>>> per-institution-account (e.g. fidelity 401k) with sub-headers by year and >>>> then by month. These headers with their folding in vim (similar to emacs >>>> outline mode) are helpful, but I think I can do better. It is also a >>>> less-than-nice experience for on the fly entries such as invoices, >>>> tracking >>>> kidsโ allowance, and so on. I need to just set aside a few hours to >>>> ponder >>>> this and come up with the right organization and workflow. >>>> >>>> >>>> There is a final stage of awakening called "ace-jump-mode", kept as a >>>> closely guarded secret >>>> >>>> Is there a vim equivalent of nirvana? :P Because of beancount, I gave >>>> emacs+orgmode a real try โ a month with spacemacs, and a month with doom, >>>> but ultimately reverted back to what I know (vim). Iโm hoping to get my >>>> beancount vim+VSCode experience leveled up. >>>> >>>> >>>> new rewriting capabilities will make it possible to rewrite the actual >>>> file by changing the AST and writing it back out. >>>> >>>> This sounds really exciting. >>>> >>>> >>>> in the next version. With a more generic SQL or providing a library >>>> that works with petl >>>> >>>> Being able to reason about my financials using dataframes sounds >>>> empowering. Looking forward to it. >>>> >>>> >>>> -- >>>> Dustin Farris >>>> (646) 671-2007 >>>> >>>> >>>> >>>> On Feb 7, 2021, at 8:06 AM, Martin Blais <bl...@furius.ca> wrote: >>>> >>>> On Sun, Feb 7, 2021 at 10:24 AM Dustin Farris <dustin...@gmail.com> >>>> wrote: >>>> >>>>> I've been using beancount for a little over a year to manage my >>>>> personal finances as well as some side self-employment accounting. I >>>>> have >>>>> twice gotten frustrated to the point of trying other products >>>>> (specifically >>>>> Personal Capital, Mint, Quicken, and QuickBooks) but give up and come >>>>> back >>>>> to command-line accounting for reasons probably familiar to everyone in >>>>> this group. >>>>> >>>>> So here are my frustrations and what I'm going to attempt to do for >>>>> myself to address them. I'm mainly venting here, but if anyone has any >>>>> suggestions I'd really appreciate them. >>>>> >>>>> >>>>> - I update my journal every month. Getting updated transactions >>>>> from 25+ different accounts every month is very time consuming. >>>>> - Todo: Research programmatic downloads of transactions from >>>>> all banks (has this been done already?) >>>>> >>>>> Look for Yodlee, or Plaid. >>>> If you're willing to give up your passwords, those could save you time >>>> by lumping together a bunch of imports. >>>> (I tend to only update the most important ones - checking, credit card, >>>> investing - frequently, and the others lag behind a bit.) >>>> >>>> Also, there are simplifications coming up to the importing framework >>>> which is going to make it substantially simpler to use, though I'm not >>>> sure >>>> if those changes will address your specific needs. >>>> >>>> >>>>> >>>>> - The beancount file is getting overwhelmingly large after just 1 >>>>> year. This is making it hard for me to jump around and find/fix >>>>> things. I >>>>> often have personal transactions that cross equity accounts into our >>>>> rental >>>>> business, or my software engineering self employment. >>>>> - Todo: split personal.beancount into smaller journals (by >>>>> month?) >>>>> >>>>> Sure, you can split. It won't make it much faster though. (The ongoing >>>> rewrite in C++ is going to address that.) >>>> >>>> I use outline-minor-mode. Another clever trick is to realize that like >>>> in life, you only ever go to a few places in that file - depending how you >>>> organize it - and insert unique comment tags and just i-search for them to >>>> go there directly (*). Huge time saver. The new import framework might >>>> even >>>> auto-insert in front of those tags. >>>> >>>> I think if you have the patience, you could refine your previous >>>> transaction detection so that it's nearly flawless, automatically >>>> categorize your transactions, and implement auto-insert in your file and >>>> trust it. Would save some time, but it'll be time-consuming to get it >>>> right >>>> IMO (you'll spend the time coding instead of updating; it's unclear to me >>>> in that situation which will be better). beangulp will also provide more >>>> hooks for you to refine previous-import-detection, even per importer. (For >>>> more on upcoming changes on that look here >>>> https://docs.google.com/document/d/1O42HgYQBQEna6YpobTqszSgTGnbRX7RdjmzR2xumfjs/edit >>>> ) >>>> >>>> >>>> (*) In fact, I would say if you're using emacs this is the fourth stage >>>> of evolution of Emacs users about cursor movement: first you use the >>>> cursor >>>> keys, then you learn to use the relative word and paragraph movement, then >>>> you graduate to sexp movements (e.g. beginning of function), and when you >>>> nearly achieve enlightenment you are nearly always moving via >>>> interactive-search ( >>>> https://www.masteringemacs.org/article/effective-editing-movement). >>>> There is a final stage of awakening called "ace-jump-mode", kept as a >>>> closely guarded secret by those of use experiencing a permanent state of >>>> non-duality ;-). >>>> >>>> >>>> >>>>> >>>>> - omni-complete in vim is an awkward keyboard chord and account >>>>> completions in vim are sometimes broken depending on what you last >>>>> typed >>>>> - Todo: try VSCode which can now embed neovim and has it's own >>>>> beancount extension that should make the editing experience nicer >>>>> - except VSCode folding is weird so this might actually make >>>>> things worse >>>>> - I still donโt understand how reporting works, and part of >>>>> that is because I don't use it enough. I feel like I'm relearning >>>>> Beancount Query Language every time I do need something. >>>>> - Todo: keep a list of commonly-used queries >>>>> >>>>> Write your queries as code, and run your own scripts. There's a neat >>>> API. >>>> You can also use the Query directive to save pre-made queries in your >>>> Beancount file and execute them by name. >>>> >>>> Easier reporting is something I'd like to refine in the next version. >>>> With a more generic SQL or providing a library that works with petl. >>>> >>>> >>>> >>>>> - reconciling receipts / splitting transactions is time consuming, >>>>> although vim macros help here >>>>> - Todo: Improve importer recognition of payees and likely >>>>> expense accounts >>>>> >>>>> Yep. >>>> >>>> (v3's new parser will bring another dimension to this: instead of >>>> having to write code to fixup the payees in memory only, the new rewriting >>>> capabilities will make it possible to rewrite the actual file by changing >>>> the AST and writing it back out.) >>>> >>>> >>>>> >>>>> - no way to attach receipt pictures to transactions? >>>>> - Todo: Research beancount tooling or other apps to capture >>>>> receipts >>>>> >>>>> See the ingest framework's filling ability. It renames files that have >>>> been identified prepending a date and can stash them in a repository under >>>> the same account hierarchy. bean-web used to serve those directives right >>>> next to the transactions and you could insert a common link to link them >>>> together (or write a plugin that attempts to match them up automatically) >>>> >>>> >>>>> >>>>> - fava is nice, but could be nicer >>>>> - Todo: Research writing a new frontend reporting/visualization >>>>> tool and/or contribute to fava >>>>> >>>>> Go for it! >>>> Custom one-off renderings are also useful (e.g. like in beangrow). >>>> >>>> >>>> >>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Beancount" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to beancount+...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/beancount/4bd97aa7-ef37-4f08-9b65-d9555acbe7fbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Beancount" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to beancount+...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/beancount/CAK21%2BhNsEmRrdxnDCbrZXqxam2pphcGysM4WpEmb0CS-wd3R6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Beancount" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to beancount+...@googlegroups.com. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com >>>> >>>> <https://groups.google.com/d/msgid/beancount/AC64FB84-AF87-4AA7-AB57-CF0667DE1728%40gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to beancount+...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/da9b922d-fa61-4eec-8ab6-e0cb7b55fc5dn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/beancount/da9b922d-fa61-4eec-8ab6-e0cb7b55fc5dn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/a8539316-c6ac-4358-a3e3-2b3ae78d0eeen%40googlegroups.com.
