Jean, I would love to see how you did it with Python. That's actually on the shortlist of languages I want to learn.
I realized that at least in so far as the Citi produced OFX file is concerned, I will always need to delete a fixed number of lines. If I download only one account at a time, I must delete the first 30 lines. I have been using sed to do this: sed -i '1,30d' /tmp/ofx.log I realized just now (after grabbing some sample downloads of various sizes) that if there is more than one account I need to find that first </OFX> tag and then delete that line and the 31 that follow. I tried using sed with regex but can't get it quite right as it only deletes the </OFX> line; the next 31 lines are left alone: sed -i '/<[^>]*\/OFX>/,32d' /tmp/ofx.log Much appreciation for your help! Stu On Wed, Apr 15, 2020 at 6:22 PM Jean Laroche <rip...@gmail.com> wrote: > I hacked some python code to do that if you're interested. I'm taking > ofx files, reading them and combining them just the way you describe it. > It's not beautiful code but it works. > What would be awesome would be if you could select multiple files in GC, > and then do the import! > > On 4/15/20 2:58 PM, Stu Perlman wrote: > > I did some experimenting today and I think I am seeing something similar. > > > > If I take the </OFX> tag that ends the first account’s transaction data > > and then select all text up to and including the <OFX> tag that starts > > the next account’s transaction it will work perfectly. > > > > There is a bunch of stuff in between those two tags, including another > > pair of <OFX> </OFX> tags that wrap some header info that appears before > > the second account’s transactions. > > > > I’d rather not have to manipulate the file each time so if I can figure > > out a script that can do what I am doing by hand I should be set. If > > it’s not the same number of lines or characters each time my script > > will need to be able to detect the correct tags as start and stop points > > before removing the text. I plan on working on this a little more now > > that I know it is possible to import multiple accounts using a single > file. > > > > On Tue, Apr 14, 2020 at 10:11 PM jean laroche <rip...@gmail.com > > <mailto:rip...@gmail.com>> wrote: > > > > Well, combined OFX files *are* supported, at least when the > > originate from the same bank. > > I just created a combined OFX file by joining two separate OFX > > files, and putting the two data in between the BANKMSGSRSV1 tags in > > a row. > > > > <BANKMSGSRSV1> > > .... > > </BANKMSGSRSV1> > > <BANKMSGSRSV1> > > .... > > </BANKMSGSRSV1> > > > > In other words, I took everything that was between <BANKMSGSRSV1> > > and </BANKMSGSRSV1> (including these) from one file, and copied it > > right after the > > </BANKMSGSRSV1> from the other file, saved and imported. I get the > > transactions from both accounts in one shot. > > That's pretty cool. > > So if that's not working for you, you should look at your OFX file > > and try to identify where the problem originates... > > Jean > > > > > > On 4/14/2020 1:29 PM, David Carlson wrote: > >> Stu, > >> > >> Are you doing online banking through GnuCash or importing OFX > >> files that you downloaded earlier? > >> > >> I am importing OFX files that I imported directly from my bank's > >> website. > >> > >> I think that the same OFX format is used either way but a > >> different procedure is used to get the data into GnuCash. > >> > >> When I mentioned three tags that appear in OFX files that I > >> receive I only gave selected examples not a complete definition. > >> One would need to get a copy of the OFX standard for completeness. > >> > >> > >> > >> > >> On Tue, Apr 14, 2020, 3:14 PM Stu Perlman <sgperl...@gmail.com > >> <mailto:sgperl...@gmail.com>> wrote: > >> > >> Jean, I always assumed it was supported because if you use GNC > >> to configure > >> AqB, you only need to set up one User per financial > >> institution even though > >> you can set up multiple accounts for that single user. > >> > >> - Stu > >> > >> On Tue, Apr 14, 2020 at 4:01 PM Jean Laroche <rip...@gmail.com > >> <mailto:rip...@gmail.com>> wrote: > >> > >> > BTW, that's a feature I was interested in (importing > >> combined OFX > >> > files). If there are others interested, I'll look at a way > >> to support > >> > that. Another thing that I miss is the ability to > automatically > >> > reconcile after an OFX import (not to highjack this thread). > >> > J. > >> > > >> > On 4/14/20 12:53 PM, Ove Grunnér wrote: > >> > > Ah, I was talking of QIF, not QFX, sorry about that. > >> > > yes the java code writes QIF. > >> > > Ove. > >> > > > >> > > On Tue, 14 Apr 2020 at 20:45, Stu Perlman > >> <sgperl...@gmail.com <mailto:sgperl...@gmail.com>> wrote: > >> > > > >> > >> Ove, > >> > >> > >> > >> Thanks for the info that you shared. Is your Java > >> program writing to > >> > the > >> > >> QIF format or the OFX format? > >> > >> > >> > >> David - my apologies, I never noticed your reply to my > >> original email > >> > last > >> > >> month! I'm happy to read that it should work. > >> > >> > >> > >> I'm still running GNC 3.8 from a mid-march build on my > >> WIndows system. I > >> > >> have yet to get GNC, AqBanking 6.x and Citi's OFX API to > >> all play well > >> > with > >> > >> each other. I am able to get my data from Citi by using > >> an older AqB > >> > >> release that I have running under Cygwin to pull the data > >> from the > >> > Citi's > >> > >> web server. I have found that in order to import the > >> files that are > >> > >> download into GNC that I need to make one file for each > >> account at Citi > >> > (by > >> > >> running aqbanking-cli for one account at a time). I then > >> strip out the > >> > 1st > >> > >> dozen or so lines in the response received. I have > >> managed to script > >> > most > >> > >> of this so it's not nearly as cumbersome as it may > >> appear, but needing > >> > to > >> > >> enter the password once for each account at Citi is > >> annoying as is > >> > needing > >> > >> to import multiple files into GNC instead of one > >> consolidated file. I > >> > tried > >> > >> to eliminate the repeating password step with the Expect > >> tool, but I > >> > could > >> > >> not get it to work in the Cygwin environment. > >> > >> > >> > >> Unfortunately, the other financial institutions where I > >> have multiple > >> > >> accounts don't even support OFX (or at least if they do, > >> the info is not > >> > >> available on OFXhome.net) so I don't know if the issues > >> I have run into > >> > >> are caused by Citi, GNC, or AqB. These issues are not so > >> annoying that I > >> > >> would go back to Quicken! LOL I may mess around with > >> trying to > >> > automate > >> > >> my calls to aqbanking-cli using PowerShell so that I can > >> take Cygwin > >> > out of > >> > >> the mix and/or just try moving over to my Ubuntu system > >> for GNC. > >> > >> > >> > >> - Stu > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> On Tue, Apr 14, 2020 at 2:40 PM David Carlson < > >> > david.carlson....@gmail.com > >> <mailto:david.carlson....@gmail.com>> > >> > >> wrote: > >> > >> > >> > >>> Ove, > >> > >>> > >> > >>> I believe !Clear:AutoSwitch is only used in QIF > >> imports. My OFX > >> > imports > >> > >>> have tags similar to > >> > >>> > >> > >>> <BANKID> <ACCTID> and <ACCTTYPE>. > >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> On Tue, Apr 14, 2020 at 12:12 PM Ove Grunnér > >> <write2...@gmail.com <mailto:write2...@gmail.com>> > >> > wrote: > >> > >>> > >> > >>>> it works for me as well, but I know I need to put a > >> specific line in > >> > the > >> > >>>> beginning of the file to let gnuCash know it is multi > >> account > >> > >>>> > >> > >>>> open up your file and check if the firs line is: > >> > >>>> !Clear:AutoSwitch > >> > >>>> > >> > >>>> Try to add the line if it is missing. > >> > >>>> > >> > >>>> I use the below java code to write my import files. > >> > >>>> > >> > >>>> br > >> > >>>> > >> > >>>> > >> > >>>> try { > >> > >>>> FileWriter fstream = new > >> FileWriter(targetDirectoryPointer + > >> > targetQifFileName); > >> > >>>> BufferedWriter info = new BufferedWriter(fstream); > >> > >>>> > >> > >>>> // WRITE TO FILE > >> > >>>> // indicator for multi account import > >> > >>>> info.write("!Clear:AutoSwitch"+"\n"); > >> > >>>> > >> > >>>> for (int acc=0; acc< maxAcc; acc++) { > >> > >>>> > >> > >>>> int transactionTableMaxRows = > >> > accountTable.getTransactionTableMaxRows(acc); > >> > >>>> DateFormat dateFormat = new > >> SimpleDateFormat("yyyy-MM-dd"); > >> > >>>> > >> > >>>> // get data for this account > >> > >>>> String targetFileAccountName = > >> > accountTable.getTransactionTableAccountName(acc); > >> > >>>> String[] transactionDescription = > >> > accountTable.getTransactionDescription(acc); > >> > >>>> float[] transactionAmount = > >> > accountTable.getTransactionAmount(acc); > >> > >>>> float[] transactionBalance = > >> > accountTable.getTransactionBalance(acc); > >> > >>>> Date[] transactionDate = > >> > accountTable.getTransactionDate(acc); > >> > >>>> > >> > >>>> // WRITE TO FILE > >> > >>>> // New Account > >> > >>>> String account = accountDetailsGnuCashName[acc] > ; > >> > >>>> info.write("!Account" + "\n" + "N" + account + > >> "\n" + "^" + > >> > "\n"); > >> > >>>> > >> > >>>> > >> > >>>> for (int l = 0; l < transactionTableMaxRows; > >> l++) { > >> > >>>> > >> > >>>> // Transaction > >> > >>>> String type = "Bank"; > >> > >>>> String date = > >> dateFormat.format(transactionDate[l]); > >> > >>>> String amount = > >> String.valueOf(transactionAmount[l]); > >> > >>>> String description = > >> transactionDescription[l]; > >> > >>>> String category = transactionDescription[l]; > >> > >>>> > >> > >>>> info.write("!Type:" + type + "\n"); > >> > >>>> info.write("D" + date + "\n"); > >> > >>>> info.write("T" + amount + "\n"); > >> > >>>> info.write("P" + description +"\n"); > >> > >>>> info.write("L" + category + "\n"); > >> > >>>> info.write("^" + "\n"); > >> > >>>> } > >> > >>>> } > >> > >>>> > >> > >>>> // Close File > >> > >>>> info.close(); > >> > >>>> > >> > >>>> } > >> > >>>> catch (IOException e) { > >> > >>>> System.out.println("A write error has occurred"); > >> > >>>> e.printStackTrace(); > >> > >>>> } > >> > >>>> > >> > >>>> > >> > >>>> On Fri, 13 Mar 2020 at 15:21, David Carlson < > >> > david.carlson....@gmail.com > >> <mailto:david.carlson....@gmail.com>> > >> > >>>> wrote: > >> > >>>> > >> > >>>>> Stu, > >> > >>>>> > >> > >>>>> I have been importing OFX files from my bank for years > >> with three or > >> > >>>>> four > >> > >>>>> checking or savings accounts in one file, so I know > >> GnuCash release > >> > >>>>> 2.6.19 > >> > >>>>> can import OFX files with multiple accounts. It is > >> possible but > >> > >>>>> unlikely > >> > >>>>> that GnuCash has lost that ability in recent > >> releases. You may have > >> > >>>>> something else happening in your case. > >> > >>>>> > >> > >>>>> On Fri, Mar 13, 2020 at 9:58 AM Stu Perlman > >> <sgperl...@gmail.com <mailto:sgperl...@gmail.com>> > >> > >>>>> wrote: > >> > >>>>> > >> > >>>>>> Hello, > >> > >>>>>> > >> > >>>>>> I am using GNC on Windows 10. I have a fairly recent > >> build, dated > >> > >>>>> March 9 > >> > >>>>>> from the maintenance branch. > >> > >>>>>> > >> > >>>>>> I noticed that I'm still unable to use AqBanking to > >> get OFX data > >> > from > >> > >>>>> Citi > >> > >>>>>> into GNC. I Did a little exploring and discovered > >> that if I > >> > >>>>> manipulate the > >> > >>>>>> file in C:\tmp\ofx.log and break it into separate > >> parts for each > >> > >>>>> account > >> > >>>>>> that was downloaded that GNC will import the data > >> just fine if it's > >> > >>>>> limited > >> > >>>>>> to one account per file. I also believe, but I need > >> to confirm that > >> > >>>>> I can > >> > >>>>>> leave all of the data in a single file but that I > >> need to eliminate > >> > a > >> > >>>>> lot > >> > >>>>>> lines starting with the closing tags for the first > >> account's > >> > response > >> > >>>>> body > >> > >>>>>> and up to and including the opening tags for the > >> subsequent > >> > accounts' > >> > >>>>>> response bodies. > >> > >>>>>> > >> > >>>>>> Has anyone else encountered this issue and if so, do > >> you have any > >> > >>>>> ideas how > >> > >>>>>> to fix this? Is it a defect? > >> > >>>>>> > >> > >>>>>> Thanks in advance for your help. > >> > >>>>>> _______________________________________________ > >> > >>>>>> gnucash-user mailing list > >> > >>>>>> gnucash-user@gnucash.org > >> <mailto:gnucash-user@gnucash.org> > >> > >>>>>> To update your subscription preferences or to > >> unsubscribe: > >> > >>>>>> > https://lists.gnucash.org/mailman/listinfo/gnucash-user > >> > >>>>>> If you are using Nabble or Gmane, please see > >> > >>>>>> https://wiki.gnucash.org/wiki/Mailing_Lists for more > >> information. > >> > >>>>>> ----- > >> > >>>>>> Please remember to CC this list on all your replies. > >> > >>>>>> You can do this by using Reply-To-List or Reply-All. > >> > >>>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> -- > >> > >>>>> David Carlson > >> > >>>>> _______________________________________________ > >> > >>>>> gnucash-user mailing list > >> > >>>>> gnucash-user@gnucash.org <mailto: > gnucash-user@gnucash.org> > >> > >>>>> To update your subscription preferences or to > unsubscribe: > >> > >>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user > >> > >>>>> If you are using Nabble or Gmane, please see > >> > >>>>> https://wiki.gnucash.org/wiki/Mailing_Lists for more > >> information. > >> > >>>>> ----- > >> > >>>>> Please remember to CC this list on all your replies. > >> > >>>>> You can do this by using Reply-To-List or Reply-All. > >> > >>>>> > >> > >>>> > >> > >>> > >> > >>> -- > >> > >>> David Carlson > >> > >>> > >> > >> > >> > > _______________________________________________ > >> > > gnucash-user mailing list > >> > > gnucash-user@gnucash.org <mailto:gnucash-user@gnucash.org> > >> > > To update your subscription preferences or to unsubscribe: > >> > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > >> > > If you are using Nabble or Gmane, please see > >> > https://wiki.gnucash.org/wiki/Mailing_Lists for more > >> information. > >> > > ----- > >> > > Please remember to CC this list on all your replies. > >> > > You can do this by using Reply-To-List or Reply-All. > >> > > > >> > > >> _______________________________________________ > >> gnucash-user mailing list > >> gnucash-user@gnucash.org <mailto:gnucash-user@gnucash.org> > >> To update your subscription preferences or to unsubscribe: > >> https://lists.gnucash.org/mailman/listinfo/gnucash-user > >> If you are using Nabble or Gmane, please see > >> https://wiki.gnucash.org/wiki/Mailing_Lists for more > information. > >> ----- > >> Please remember to CC this list on all your replies. > >> You can do this by using Reply-To-List or Reply-All. > >> > > > _______________________________________________ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. ----- Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
