Oh cool!
Thanks for the pointer.
One more question: is the ofx data encrypted on the way back to your
side of things? It does not look like it is since you're able to
download your data once you know all the parameter of the "traditional"
ofx query, is that right?
J.
On 2/7/2021 7:28 PM, Scott McRae wrote:
I'm you want something a bit more automated, I came across mitm-proxy
in searches:
https://mitmproxy.org/
This should take care of generating certificates automatically and
actually do the forwarding, etc. You'll need to generate a CA cert for
it and install that in your trusted certificates. Their docs should
explain how. I didn't actually use it, since my manual method ended up
working, but this sounds better suited for your use case.
On Sun, Feb 7, 2021 at 7:23 PM Jean L <rip...@gmail.com
<mailto:rip...@gmail.com>> wrote:
Wow, that's really cool. I would love to replicate that to be able
to connect to my bank as I'm sure many would. I wonder if there
would be a way to make that a bit easier than completely manually.
At the moment, I have a python script that logs into my bank, make
the right clicks and downloads the OFX files. Definitely NOT
robust so I would love to be able to go back to downloading ofx
files directly.
Could you possibly write a small blurb on how to do this, from
start to finish? That would be super useful for me. On the other
hand, I'm not sure whether this is 100% within the law, not sure
whether the DMCA has something to say about this or not :(
Jean
On 2/7/2021 7:06 PM, Scott McRae wrote:
>>/So I decided to give the devil his due and temporarily got a
Quicken />>/subscription and setup an SSL man-in-the-middle. />Sure, you can have a man-in-the-middle setup, but if you don't have the
>keys that quicken and the bank use to communicate and communications are
>encoded, you can't get any data from being in the middle, unless I'm
>missing something.
I generated a self-signed cert and added to the trust store on my Mac OS
keychain. I was actually able to get away with a very manual
man-in-the-middle
using an "openssl s_server" command running on 433, modifying the /etc/hosts
file to point back to my machine, and copy-pasting the request to curl, then
copy-pasting the response.
On Sat, Feb 6, 2021 at 8:45 PM Scott McRae <smc...@parax.com
<mailto:smc...@parax.com>> wrote:
I got this working in my software with some help for the info
on this list. Here is a write-up:
USAA's changes to their OFX interface
-------------------------------------
On 2020-01-26, USAA's previous OFX interface
(https://service2.usaa.com/ofx/OFXServlet) stopped working.
It seems like they switched to a new interface through a tech
provider to replace their previous login method (with your
website credentials) to an app-specific ID and password. This
is a good move for security, but it was done without notice,
it seems, to anyone but Quicken.
From some internet searches, I found some people on the right
track to fixing this on the GNU Cash development mailing list:
https://lists.gnucash.org/pipermail/gnucash-devel/2021-January/045664.html
They were able to determine that USAA was:
- using a new OFX endpoint:
https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
- using a new OFX Org ID: USAA Federal Savings Bank
- using a new OFX FID: 67811
Additionally, someone on the USAA forums was about to extract
and post the link to generate an App ID and PIN:
source:
https://communities.usaa.com/t5/Finances/USAA-Creates-Quicken-Monopoly/td-p/243850/highlight/false
Authorization link:
https://df3cx-services.1fsapi.com/casm/usaa/enroll
However, with a lot of trial and error I still wasn't able to
hit this new endpoint successfully. So I decided to give the
devil his due and temporarily got a Quicken subscription and
setup an SSL man-in-the-middle.
The new OFX interface is *very* finicky, so you basically
have to input everything exactly the way it expects it. Here
is an example of an account listing query that works:
echo -en
"OFXHEADER:100\r\nDATA:OFXSGML\r\nVERSION:103\r\nSECURITY:NONE\r\nENCODING:USASCII\r\nCHARSET:NONE\r\nCOMPRESSION:NONE\r\nOLDFILEUID:NONE\r\nNEWFILEUID:NONE\r\n\r\n<OFX>\r\n<SIGNONMSGSRQV1>\r\n<SONRQ>\r\n<DTCLIENT>20210207031942\r\n<USERID>XXXXX\r\n<USERPASS>NNNNN\r\n<LANGUAGE>ENG\r\n<FI>\r\n<ORG>USAA
Federal Savings
Bank\r\n<FID>67811\r\n</FI>\r\n<APPID>QMOFX\r\n<APPVER>2300\r\n<CLIENTUID>1955A543-B071-455E-A31E-73CC7C493D68\r\n</SONRQ>\r\n</SIGNONMSGSRQV1>\r\n<SIGNUPMSGSRQV1>\r\n<ACCTINFOTRNRQ>\r\n<TRNUID>e39add7d-2085-4504-b9ee-be37927de39c\r\n<ACCTINFORQ>\r\n<DTACCTUP>19900101\r\n</ACCTINFORQ>\r\n</ACCTINFOTRNRQ>\r\n</SIGNUPMSGSRQV1>\r\n</OFX>\r\n"
| curl -isS -X POST -H "Content-Type: application/x-ofx" -A
InetClntApp/3.0 --data-binary @-
https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
Note you have to change the XXXXX and NNNNN to the App ID and
PIN you get from the link above.
Some things I've found through trial and error:
- The OFX elements must be separated with "\r\n". This is
dumb, but true. No spaces. No simple "\n". Exactly "\r\n".
- The APPID "QMOFX" and APPVER "QMOFX" work. Others I tried
did not.
- The CLIENTUID "1955A543-B071-455E-A31E-73CC7C493D68" works
for me. It must be uppercase. This might be particular to
your account. If so, you can find it looking at the OFX logs
from Quicken.
- TRNUID must be present, but an UUID will do.
- DTACCTUP: The value "19900101" works. The value "19700101"
does not. The value "19900101000000" does not.
- You need the "Content-Type: application/x-ofx" header
- You need the User-Agent "InetClntApp/3.0". This is what
Quicken for Mac sends.
It also seems their gateway will under some conditions put
your IP on a ban list. If you are testing, you may want to
spin up an AWS instance or something. When you get on it,
you'll start seeing an empty HTML page response, like:
<html>
<head>
<META NAME="robots" CONTENT="noindex,nofollow">
<script
src="/_Incapsula_Resource?SWJIYLWA=5074a744e2e3d891814e9a2dace20bd4,719d34d31c8e3a6e6fffd425f7e032f3">
</script>
<body>
</body></html>
Valid queries will work from different source IPs when this
happens.
Thanks to Bob White on the GNU Cash list and RDD! on the USAA
Forums for the breadcrumbs. No thanks to USAA for swapping
out their functional interface with absolutely no notice or
documentation and pretending like Quicken users are the only
customers of any importance. Please just don't break our
software again... at least for awhile.
- Scott McRae
_______________________________________________
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
