[GNC-dev] New OFX Requirements For USAA FSB
Not sure if this is the proper channel, but USAA Federal Savings Bank has deprecated older QWIN OFX support. I have confirmed with the bank this occurred on 27 Jan 2021. Using a trial subscription for Quicken for Mac, I have an OFX log for new syncing sequence with USAA. Here's a snippet: 2021-01-28 16:50:44 +: Request to: https://df3cx-services.1fsapi.com/casm/usaa/access.ofx (BID 67811) Full request body string: OFXHEADER:100 DATA:OFXSGML VERSION:103 SECURITY:NONE ENCODING:USASCII CHARSET:1252 COMPRESSION:NONE OLDFILEUID:NONE NEWFILEUID:NONE 20210128115044.978[-5:EST] XX N ENG USAA Federal Savings Bank 67811 QMOFX 2300 -DB64-4AC0-A835-xx840640 -8F48-47B5-A984-xx03392E MSGSET 0101 I have edited the online banking user and account files to the point where I think they should reflect the above settings, but to no avail. What is the proper way to debug new OFX setup? Can it be done here or should I be talking specifically with the aqbanking team? Thanks, Bob White ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
Thanks, John, Not mentioned in your emails is the response from USAA: A webpage reporting a server error instead of the usual 50x HTTP response code. I do see a 400 in the Online Banking Transaction Window when attempting to download transactions in GNC: AqBanking v6.2.5.0stable Sending jobs to the bank(s) Sorting commands by account Sorting commands by account Sorting commands by provider Send commands to providers Send commands to provider "aqofxconnect" Locking customer "4563" Sending request... Connecting to server... Resolving hostname "df3cx-services.1fsapi.com" ... IP address is "45.60.151.211" Connecting to "df3cx-services.1fsapi.com" Connected to "df3cx-services.1fsapi.com" Using GnuTLS default ciphers. TLS: SSL-Ciphers negotiated: TLS1.3:ECDHE-RSA-AES-128-GCM:AEAD Connected. Sending message... Message sent. Waiting for response... Receiving response... HTTP-Status: 400 (Bad Request) Unlocking customer "4563" Also not mentioned in your emails: I suppose that you were able to download your transactions successfully with Quicken. Do you think you could install Wireshark (https://www.wireshark.org/#download) and collect what Quicken is sending? It's been a while since I used Wireshark, but I did install install it. Everything captured is encrypted. I've never decrypted TLS in Wireshark before. Is there a tutorial available that doesn't require the use of Chrome or Netscape so I can capture while using the Quicken app? If not, I guess I could try the Quicken Web interface via Chrome or Netscape and capture things that way. Bob ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
The Quicken web interface is I think different from OFX Direct Connect. If it's OFX Web Connect then it handles authentication differently and that's probably at least part of the problem. I found a quicken community discussion that suggests that Quicken for Windows used IE to connect, so I'd imagine that Quicken for Mac would use WebKit. I don't know if Apple's installed WebKit uses openssl, but it might, in which case it might be possible to get a key log for the Quicken session. Total speculation, I've never done anything remotely like this. I did capture and decrypt enough to see that a REST API is in use for Quicken for Web so that's no help. I have an OFX log file contenting a small number of sessions from Quicken for Mac containing initial setup, successful accounts download, successful and unsuccessful account update requests. Many of the OFX interaction are with an Intuit URL, but the USAA account interactions are clearly independent to a new URL: https://df3cx-services.1fsapi.com/casm/usaa/access.ofx Are the OFX interactions with USAA enough to update aqbanking or do we need the HTTP interactions as well? Regards, Bob ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
I can confirm the redirect to USAA for creds is a one-time issue. You also need a CUSTOMUID. I haven’t deciphered if it is generated sever or client side. I am away from my Mac at the moment but can upload the OFX log when I get back to it. Is there someplace to upload to or would attachment to a follow-on email work? Regards, Bob Sent from my iPhone > On Jan 30, 2021, at 12:17 PM, Martin Preuss wrote: > > Am 30.01.21 um 18:06 schrieb John Ralls: >> >> On Jan 30, 2021, at 6:50 AM, Bob White wrote: >> [...] >> "The credentials that Quicken prompts you for will vary based on your >> software version and the type of account you want to download. With the >> recommended Direct Connect method in Quicken follows these steps: >> >>• You'll select "Get Access ID and PIN" and we'll direct you to usaa.com >> to log on. >>• USAA will provide a unique ID and PIN that you'll use only in Quicken. >>• To add your accounts in Quicken, you'll connect using these credentials. >>• If you forget your Access ID and PIN, you can get new ones at any time >> using the same process" >> >> Not good news. > [...] > > Hmm, that sounds like you only need to retrieve those credentials once and > use them in following DirectConnect sessions? > > The redirect should be a HTTP redirect, AqBanking doesn't follow redirects > but prints them to the log window. It might be enough to copy that URL, open > it with Firefox and get the new credentials from there. > > I would then need to know in what fields those credentials are expected by > the server... > > > Regards > Martin > > > > > -- > "Things are only impossible until they're not" ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] USAA Additional Info
Scott, Thanks for the shout out. I just saw the new thread this morning and I want to confirm that the CLIENTUID is user specific--the one I was given by Quicken differs from the one you mention. I also noticed that APPVER has two values [2300, 2400] in the Quicken logs. I can't decipher when and why it changes. But if "QMOFX" works then it's easier to got with that. I hope to use your recap to try and get valid requests working through the aqbanking-cli app, but not sure how soon I can get to it--got to get to work right now. Regards, Bob ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] USAA
Scott,
To expand a bit more on your research, it turns I got myself locked out working
via command line (had a typo in my password,) but I got a new response:
=
HTTP/2 200
date: Mon, 08 Feb 2021 19:21:46 GMT
content-type: application/x-ofx
content-length: 661
vary: Origin
vary: Access-Control-Request-Method
vary: Access-Control-Request-Headers
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
cache-control: no-cache, no-store, max-age=0, must-revalidate
pragma: no-cache
expires: 0
strict-transport-security: max-age=31536000 ; includeSubDomains
x-frame-options: DENY
set-cookie:
visid_incap_2454689=VL3kNBr2SlSW0WJCqhj8bUqPIWAAQUIPAAAItMvflIE03TK597RZBrSG;
expires=Tue, 08 Feb 2022 16:54:56 GMT; HttpOnly; path=/; Domain=.1fsapi.com;
Secure; SameSite=None
set-cookie: nlbi_2454689=GUGEXaoMYhyU6CN+hXBnAwCJYu0m0lufamEaMlpy8lh6;
path=/; Domain=.1fsapi.com; Secure; SameSite=None
set-cookie:
incap_ses_1286_2454689=sP1YdCv3cxxFphfCS8rYEUqPIWAADjWNiSpDNqhTw/8uqYqZkA==;
path=/; Domain=.1fsapi.com; Secure; SameSite=None
x-cdn: Incapsula
x-iinfo: 3-12758011-12758012 CT(8 5 0) RT(1612812105972 0) q(0 0 0 -1) r(1
1) U6
OFXHEADER:100
DATA:OFXSGML
VERSION:103
SECURITY:NONE
ENCODING:USASCII
CHARSET:NONE
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE
2000ERRORYou're locked out due to multiple tries, you'll need to verify your information. Get Access ID and PIN here
https://df3cx-services.1fsapi.com/casm/usaa/enroll20210208142146.714[-5:EST]ENGUSAA Federal Savings
Bank67811096F9A5D-A1A6-4FF7-A75E-222F55F834CFc15500ERROR
=
I am even getting this response in the Quicken OFX logs at the moment.
Followed the link in the MESSAGE above and it just showed me the creds page
with the user/passwd I'd preciously setup.
I was able to automate some of the changing fields (DTCLIENT and TRNUID) in the
request:
=
echo -en "OFXHEADER:100\r\nDATA:OFXSGML\r\nVERSION:103\r\nSECURITY:NONE\r\nENCODING:USASCII\r\nCHARSET:NONE\r\nCOMPRESSION:NONE\r\nOLDFILEUID:NONE\r\nNEWFILEUID:NONE\r\n\r\n\r\n\r\n\r\n`date
+%Y%m%d%H%M%S`\r\n362\r\n66\r\nENG\r\n\r\nUSAA Federal Savings
Bank\r\n67811\r\n\r\nQMOFX\r\n2300\r\n-DB64-4AC0-A835-\r\n\r\n\r\n\r\n\r\n`uuidgen
| id=_ && echo ${id^^}`c\r\n\r\n19900101\r\n\r\n\r\n\r\n\r\n" | curl -isS -X POST -H "Content-Type: application/x-ofx" -A
InetClntApp/3.0 --data-binary @- https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
=
Not sure how long I will be locked out, but will get back to it when I can.
Regards,
Bob
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] gnucash-devel Digest, Vol 215, Issue 9
Scott, After about 45 minutes on the phone with USAA being transferred to no less than 3 reps it turns out all one needs to do to reset the lock on Quicken access for USAA is visit: www.usaa.com/accessid And resubmit the form. Also, once unlocked I was able to get two successful requests submitted via curl: ACCTINFOTRNRQ: OFXHEADER:100 DATA:OFXSGML VERSION:103 SECURITY:NONE ENCODING:USASCII CHARSET:NONE COMPRESSION:NONE OLDFILEUID:NONE NEWFILEUID:NONE 20210210010544.000 XXX XXX ENG USAA Federal Savings Bank 67811 QMOFX 2300 -17AE-42CC-9AE4- 25B63B5A-7E11-4B65-8D3C-EABB4407B7EE 19900101 And STMTTRNRQ: OFXHEADER:100 DATA:OFXSGML VERSION:103 SECURITY:NONE ENCODING:USASCII CHARSET:1252 COMPRESSION:NONE OLDFILEUID:NONE NEWFILEUID:12F4539B-75E1-4B37-9A82-D97C0EB4E566 20210210010359.000 XXX XXX ENG USAA Federal Savings Bank 67811 QMOFX 2300 -17AE-42CC-9AE4- A86DD084-6ADF-42EB-A410-F52CFD6333AA 1 314074269 000XXX CHECKING 2021012000.000 2021020800.000 Y The results lead me to believe NEWFILEUID is not required and can be NONE. What files need to be updated under the GnuCash.app to use the latest aqbanking lib with Martin's recent changes? Regards, Bob On February 9, 2021 at 12:00 PM, gnucash-devel-requ...@gnucash.org wrote: You can try unlinking Quicken from your USAA account settings page (under Apps) and then re-activate it? On Mon, Feb 8, 2021 at 11:47 AM Bob White wrote: Scott, To expand a bit more on your research, it turns I got myself locked out working via command line (had a typo in my password,) but I got a new response: ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
Martin, With further curl testing I have discovered that the NEWFILEUID apparently now requires an actual UUID or the value NONE. You can see my working examples in my previous post to the list. Any attempt to use time-based NEWFILEUID, e.g. 20210209212246.000, results in: Error encountered. See details below: java.io.IOException: Server returned HTTP response code: 400 for URL: https://usint.pz.finance-api.services/eftx/usaa/access.ofx I was able to clone and build the latest aqbanking from GitHub. I replaced the user edit dialog within GnuCash.app and saw the new httpUserAgent field, but the data was not persisted to the user config file. I expect this was due to not having all the pieces correctly overlaid on GnuCash. Do you know which files need to be replaced within GnuCash.app on macOS from a fresh build of aqbanking to fully replace the 6.2.5 that was distributed with it? Regards, Bob ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
Martin, Bob, Am 10.02.21 um 04:09 schrieb Bob White via gnucash-devel: [...] With further curl testing I have discovered that the NEWFILEUID apparently now requires an actual UUID or the value NONE. You can see my working examples in my previous post to the list. [...] Thanks, I changed it to "NONE" in current GIT. I pulled the latest (on master) but didn't see any commits beyond the uuid fix. I made the following changes, ran 'make && make install', but don't see the NONE reflected in aqbanking-cli requests (they still have time-base NEWFILEUID): diff --git a/src/libs/plugins/backends/aqofxconnect/v1/n_header.c b/src/libs/plugins/backends/aqofxconnect/v1/n_header.c index aa933df..f335a20 100644 --- a/src/libs/plugins/backends/aqofxconnect/v1/n_header.c +++ b/src/libs/plugins/backends/aqofxconnect/v1/n_header.c @@ -58,10 +58,11 @@ int AO_V1_AddOfxHeaders(AB_PROVIDER *pro, AB_USER *u, GWEN_BUFFER *buf, const ch GWEN_Buffer_AppendString(buf, "CHARSET:1252\r\n" "COMPRESSION:NONE\r\n" - "OLDFILEUID:NONE\r\n"); - GWEN_Buffer_AppendString(buf, "NEWFILEUID:"); - GWEN_Time_toString(ti, "MMDDhhmmss.000", buf); - GWEN_Buffer_AppendString(buf, "\r\n"); + "OLDFILEUID:NONE\r\n" + "NEWFILEUID:NONE\r\n"); + // GWEN_Buffer_AppendString(buf, "NEWFILEUID:"); + // GWEN_Time_toString(ti, "MMDDhhmmss.000", buf); + // GWEN_Buffer_AppendString(buf, "\r\n"); /* header finished */ GWEN_Buffer_AppendString(buf, "\r\n"); Am I assuming the make process is more thorough than it is or am I missing something else? I would love to be able to get a successful download with aqbanking-cli to demonstrate we've isolated all the issues with the new OFX interface at USAA. Regards, Bob ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] New OFX Requirements For USAA FSB
Martin,
Thanks. I integrated the 'make clean' and I can now run locally modified code.
It turns out I am getting a 400 on receive from:
aqbanking-cli request --account=00075X --transactions --fromdate=20210120
--todate=20210208
Debug output:
...
= Enter Password =
Please enter the password for user XX
Input: **
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 175: Saving OFX log to
"/Users/bobwhite/aqbanking/logs/aqofx.log" ...
Saving communication log to /Users/bobwhite/aqbanking/logs/aqofx.log
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 128: RBW
_createConnection
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 136: RBW addr
https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 145: RBW HttpVMajor 0
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 147: RBW HttpVMinor 0
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 151: RBW userAgent
InetClntApp/3.0
6:2021/02/11 21-39-57:aqbanking(3280):httpsession.c: 167: Extending TLS SyncIo
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 65: RBW Connect here (0)
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 75: RBW POST (0)
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 88: RBW Recv (400)
3:2021/02/11 21-39-57:aqofxconnect(3280):io_network.c: 99: here (400)
6:2021/02/11 21-39-57:aqbanking(3280):./provider_user.c: 252: Unlocking customer
"4563"
8:2021/02/11 21-39-57:aqbanking(3280):provider.c: 82: Destroying AB_PROVIDER
(aqofxconnect)
accountInfoList {
} #accountInfoList
securityList {
} #securityList
messageList {
} #messageList
Not being familiar with the gwenhywfar lib I haven't been able to see the
actual buffer with the content of the POST request.
The curl version of the request specified in the aqbanking-cli request above is working.
I am including the verbose curl output from the request in the hopes it might provide
clues as to whether gwenhywfar supports the interaction it describes, like the switchover
to HTTP/2. (In the aqbanking-cli request I also did get a prompt to accept the cert due
to "Status : Certificate owner does not match hostname" so it makes me wonder
if the subjectAltName is supported.)
* Trying 45.60.151.211...
* TCP_NODELAY set
* Connected to df3cx-services.1fsapi.com (45.60.151.211) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=imperva.com
* start date: Jan 20 17:31:41 2021 GMT
* expire date: Jul 22 08:26:17 2021 GMT
* subjectAltName: host "df3cx-services.1fsapi.com" matched cert's
"*.1fsapi.com"
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign Atlas R3 DV TLS CA 2020
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x8c3bb0)
POST /casm/usaa/access.ofx HTTP/2
Host: df3cx-services.1fsapi.com
User-Agent: InetClntApp/3.0
Accept: */*
Content-Type: application/x-ofx
Content-Length: 753
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* We are completely uploaded and fine
< HTTP/2 200
HTTP/2 200
< date: Fri, 12 Feb 2021 02:51:19 GMT
date: Fri, 12 Feb 2021 02:51:19 GMT
< content-type: application/x-ofx
content-type: application/x-ofx
< content-length: 5097
content-length: 5097
< vary: Origin
vary: Origin
< vary: Access-Control-Request-Method
vary: Access-Control-Request-Method
< vary: Access-Control-Request-Headers
vary: Access-Control-Request-Headers
< x-content-type-options: nosniff
x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< cache-control: no-cache, no-store, max-age=0, must-revalidate
cache-control: no-cache, no-store, max-age=0, must-revalidate
< pragma: no-cache
pragma: no-cache
< expires: 0
expires: 0
< strict-transport-security: max-age=31536000 ; includeSubDomains
strict-transport-security: max-age=31536000 ; includeSubDomains
< x-frame-options: DENY
x-frame-options: DENY
< set-cookie:
visid_incap_2454689=MRSRHr7fT6qOqlWtxfrzWSb
Re: [GNC-dev] New OFX Requirements For USAA FSB
Martin, Thanks! Being able to see the header cleared things right up--there was a spurious "%0A" hanging off the end of the my 'serverAddr' in the user settings file, how it got there I do not know. As it turns out, with a proper user config, the current master branches of gwenhywfar and aqbanking/aqbanking-cli are good to download from USAA's new OFX URL https://df3cx-services.1fsapi.com/casm/usaa/access.ofx. For the list--when is the next release of GnuCash? Regards, Bob On February 12, 2021 at 9:50 AM, Martin Preuss wrote: Bob, Am 12.02.21 um 04:40 schrieb Bob White: [...] Not being familiar with the gwenhywfar lib I haven't been able to see the actual buffer with the content of the POST request. [...] You could increase the logelvel of libgwen to "debug", that should show the header sent (among many, many other things): GWEN_LOGLEVEL=debug aqbanking-cli ... However, you could also call the OFXDirectConnect backend with a account list request: X8 aqbanking-cli --control=aqofxconnect listusers X8 This shows the list of OFX users setup for AqBanking, the needed user id is at the end of the resulting line. You can then request a list of accounts for that bank: X8 GWEN_LOGLEVEL=debug aqbanking-cli --control=aqofxconnect getaccounts -u USERID X8 This will produce a ton of logs on the console and possibly contain sensitive information not to post unfiltered... Regards Martin ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
Re: [GNC-dev] [GNC] USAA FSB requires newer QWIN support
I got new USAA OFX download working with GNC 4.4.1.
It's mainly a configuration task.
-- User config --
int uniqueId=""
char backendName="aqofxconnect"
char userName=""
char userId=""
char customerId=""
char country="us"
char bankCode="314074269"
int lastSessionId="0"
data {
backend {
char bankName="USAA Federal Savings Bank"
char org="USAA Federal Savings Bank"
char fid="67811"
char
serverAddr="https%3A%2F%2Fdf3cx-services.1fsapi.com%2Fcasm%2Fusaa%2Faccess.ofx"
char appId="QMOFX"
char appVer="2300"
char headerVer="103"
char clientUid=""
int httpVMajor="0"
int httpVMinor="0"
char httpUserAgent="InetClntApp%2F3.0"
} #backend
} #data
-
Notes:
-- 'httpUserAgent' is currently not configurable through GNC v4.4.1, but it
works when included in the file. It has been added in newer AqBanking, look
for support in next GNC release.
-- 'CLIENTUID' can be found in OFXLog.txt generated by Quicken. Signup for
trial and cancel within 30 days no charge.
-- 'your new USAA access id' can be found at https://www.usaa.com/accessid
You will need to disconnect accounts from within the AqBanking Wizard, download
accounts with new user configuration, then reconnect accounts. In my case, the
new credit card account numbers in the accounts list are not a 100% match to
the number found on your credit card. There were enough common digits to know
which was which, just be aware. I was able to successfully download and update
2 checking accounts, a savings account, and a credit card account.
I hope this helps.
Regards,
Bob
PS. I did clone the AqBanking repo and build and install the latest libraries
on the same machine I am running GNC. I am not sure if the GNC used the
libraries distributed with it or the ones I built so YMMV.
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
[GNC-dev] R
Regards, Bob Sent from my iPhonei v ___ gnucash-devel mailing list gnucash-devel@gnucash.org https://lists.gnucash.org/mailman/listinfo/gnucash-devel
