You’re welcome, and glad to hear!
Found this <https://stackoverflow.com/questions/63723478/default-output-file-in-vs-code-being-stored-in-utf-16-le-whereas-default-is-set> if it helps. On Saturday, September 21, 2024 at 8:06:20 AM UTC-7 Joshua Cabrera wrote: > Thank you so much! Everything seems to be working now, so I just have to > tweak my.import and catch up on recording in my ledger. > > Once I'm caught up, I'll come back to automating the download and extract > workflow. > > Regarding the encoding issue, my first guess was it's because I'm doing > all this on Windows and using the Command Prompt shell, instead of bash. > That theory appeared to be incorrect after creating some test files from > the shell, which resulted in utf-8 encoding. > > Next, I tried creating test files using the integrated terminal in VS Code > since technically that's where I've been running my commands, and it turns > out that was the culprit. ¯\_(ツ)_/¯ > > On Friday, September 20, 2024 at 9:42:32 PM UTC-5 Red S wrote: > >> Thanks for the file! I can’t tell whether it was an artifact of how the >> ofx file you attached got created or whether the original ofx file suffers >> from this problem, but your ofx file is encoded in UTF-16 even though the >> header claims it’s UTF-8. ofxparse fails because of this. Running this >> allows it to work normally: >> iconv -f UTF-16 -t UTF-8 fidelity_X12345678.ofx -o a.ofx ofx-summarize >> a.ofx Total number of accounts: 1 >> ---------------- >> Account info: X12345678 fidelity.com >> Statement info: 2024-07-01 00:00:00 -- 2024-07-31 00:00:00. Bal: [] >> Types: {'cash', 'check', 'dep', 'payment'} >> >> 2024-07-30 04:00:00 cash THIS IS FUN Note -15.0000 >> 2024-07-30 04:00:00 dep THIS IS FUN Note -15.0000 >> 2024-07-29 04:00:00 check THIS IS FUN Note -15.0000 >> 2024-07-26 04:00:00 cash THIS IS FUN Note -15.0000 >> 2024-07-26 04:00:00 cash THIS IS FUN Note -15.0000 >> >> It's not the same stack trace as you got originally though. Either way, >> try this? >> >> On Friday, September 20, 2024 at 6:38:29 PM UTC-7 Joshua Cabrera wrote: >> >>> Thanks, Red. The file itself looks okay after another look. >>> >>> I tried the new line trick, but it didn't change the results. >>> >>> The simple parse threw the same unpacking value error on my ofx while >>> working on the sample ofx. >>> >>> I'm sharing a sanitized version of my ofx (replaced account number, >>> transaction details, etc.) and my import file. >>> >>> In the meantime, I'll try a different Fidelity account and smaller >>> date/transaction ranges. >>> On Friday, September 20, 2024 at 5:09:18 PM UTC-5 Red S wrote: >>> >>>> This is uncommon, but typically because there was an issue downloading >>>> the ofx. Usually because of authentication problems. Have you examined the >>>> ofx to see if it’s valid? >>>> >>>> Try this to make it more readable: sed 's/>/>\n/g' xyz.ofx >>>> >>>> >>>> You should see something like this on the top: >>>> <STATUS> <CODE> 0 </CODE> <SEVERITY> INFO </SEVERITY> <MESSAGE> Success >>>> </MESSAGE> </STATUS> >>>> >>>> >>>> If you see that, then try a really simple parse: >>>> #!/usr/bin/env python3 import ofxparse ofx_file_path = >>>> "your_ofx_file.ofx" with open(ofx_file_path, 'r') as ofx_file: ofx = >>>> ofxparse.OfxParser.parse(ofx_file) print("Account type:", >>>> ofx.account.account_type) print("Account ID:", ofx.account.account_id) >>>> >>>> >>>> On Friday, September 20, 2024 at 2:54:15 PM UTC-7 Joshua Cabrera wrote: >>>> >>>>> So, I've got my .ofx file and am not worried about the download task >>>>> anymore. That's the good news. >>>>> >>>>> The bad news is now I'm stuck on the import task. >>>>> bean-extract my.import fidelity_1234_july2024.ofx >>>>> >>>>> >>>>> Results >>>>> ERROR:root:Importer >>>>> beancount_reds_importers.importers.fidelity.Importer.identify() raised an >>>>> unexpected error: not enough values to unpack (expected 2, got 1) >>>>> Traceback (most recent call last): >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\beancount\ingest\identify.py" >>>>> , line 63, in find_imports >>>>> matched = importer.identify(file) >>>>> ^^^^^^^^^^^^^^^^^^^^^^^ >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\beancount_reds_importers\libreader\reader.py" >>>>> , line 26, in identify >>>>> self.initialize_reader(file) >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\beancount_reds_importers\libreader\ofxreader.py" >>>>> , line 26, in initialize_reader >>>>> self.ofx = self.read_file(file) >>>>> ^^^^^^^^^^^^^^^^^^^^ >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\beancount_reds_importers\libreader\ofxreader.py" >>>>> , line 57, in read_file >>>>> return ofxparse.OfxParser.parse(fh) >>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\ofxparse\ofxparse.py" >>>>> , line 396, in parse >>>>> ofx_file = OfxPreprocessedFile(file_handle) >>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\ofxparse\ofxparse.py" >>>>> , line 155, in __init__ >>>>> super(OfxPreprocessedFile, self).__init__(fh) >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\ofxparse\ofxparse.py" >>>>> , line 82, in __init__ >>>>> self.read_headers() >>>>> File >>>>> "C:\Users\joshd\AppData\Local\Programs\Python\Python312\Lib\site-packages\ofxparse\ofxparse.py" >>>>> , line 95, in read_headers >>>>> header, value = line.split(six.b(":")) >>>>> ^^^^^^^^^^^^^ >>>>> ValueError: not enough values to unpack (expected 2, got 1) >>>>> ;; -*- mode: beancount -*- >>>>> >>>>> >>>>> If I run ofx-summarize, it throws the same error, so I found a sample >>>>> .ofx online and tested ofx-summarize with it. >>>>> >>>>> And of course, the example works, so it looks like Fidelity is >>>>> providing an incorrectly formatted file? I noticed it doesn't have a >>>>> single >>>>> line break until the end of the file. >>>>> >>>>> On Monday, September 16, 2024 at 11:01:14 PM UTC-5 Red S wrote: >>>>> >>>>>> Glad you got it working with ofxget. bean-download is just a wrapper >>>>>> around ofxget or whatever command you have to parallelize all your >>>>>> downloads. If you don’t have several accounts you’re using it with, it’s >>>>>> of >>>>>> limited use. The built in template for fidelity is here, and lets >>>>>> you store your password securely in pass so it’s not stored in >>>>>> plaintext. You could simply paste your working ofxget command into the >>>>>> bean-download config file to get it to work if the parallelization >>>>>> is useful for you. Either way, that’s the last thing I’d do after I got >>>>>> everything else in the import workflow working first. >>>>>> >>>>>> And oh wow, I’d forgotten that Fidelity stopped >>>>>> <https://www.reddit.com/r/fidelityinvestments/comments/17tpgfr/why_doesnt_fidelity_support_ofx_downloads_anymore/> >>>>>> letting >>>>>> you download ofx via their website, and only let you do it via direct >>>>>> download. >>>>>> >>>>>> >>>>>> On Monday, September 16, 2024 at 7:31:34 PM UTC-7 Joshua Cabrera >>>>>> wrote: >>>>>> >>>>>>> Quick update: >>>>>>> >>>>>>> Installed ofxget and manage to get my ofxget.cfg updated using >>>>>>> ofxget acctinfo. Not sure if this required for bean-download. >>>>>>> >>>>>>> Also downloaded a .ofx file using: >>>>>>> ofxget stmt fidelity > c:\users\joshd\fidelity.ofx >>>>>>> >>>>>>> Struggling to my download.cfg and bean-download download working - >>>>>>> keep getting: >>>>>>> 1/1 sites failed: fidelity >>>>>>> >>>>>>> Will keep trying whenever I have time this week. >>>>>>> On Sunday, September 15, 2024 at 9:24:33 PM UTC-5 Red S wrote: >>>>>>> >>>>>>>> My reason for using .csv over .ofx is simply familiarity and ease >>>>>>>> of starting up. >>>>>>>> >>>>>>>> I have no experience with ofx and thought the learning curve would >>>>>>>> be much steeper, and I already feel like I'm drowning a bit as a >>>>>>>> layperson. >>>>>>>> >>>>>>>> I know it may seem like that, especially given ofx is not a human >>>>>>>> readable format. But might I suggest giving that a shot first, and you >>>>>>>> might be pleasantly surprised. Because there is nothing to do, no >>>>>>>> importer >>>>>>>> to write, since it is a machine format with an official spec, unlike >>>>>>>> csv. >>>>>>>> The learning curve should be close to zero. There are other >>>>>>>> benefits >>>>>>>> <https://reds-rants.netlify.app/personal-finance/a-word-about-input-formats-use-ofx-when-you-can/> >>>>>>>> >>>>>>>> too. >>>>>>>> >>>>>>>> I'll look into the csvreader changes you mentioned to try making a >>>>>>>> replacement for prepare_raw_columns() and also look further into ofx. >>>>>>>> >>>>>>>> If I were you, I’d definitely spend five minutes with ofx first. >>>>>>>> And reds-importers ships with ofx-summarize which you can use to >>>>>>>> peek inside and poke around an ofx file, which makes it a lot less >>>>>>>> opaque >>>>>>>> and easy on a layperson. >>>>>>>> >>>>>>>> Thanks for the help here and confirming I'm doing alright, and also >>>>>>>> thank you for maintaining it! >>>>>>>> >>>>>>>> Happy to, and glad its helpful! >>>>>>>> >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/ec40ebc8-1a09-4eda-8635-4d908d1a5eaan%40googlegroups.com.
