OK I get it, nothing on top of https.
Thanks for all this great info.
J.
On 2/7/2021 7:53 PM, Scott McRae wrote:
The encryption is all standard HTTPS (which is HTTP over TLS). It is
encrypted in both directions on the network. But if you are
terminating the TLS (a.k.a. SSL) connection, you get to see the
unencrypted data from both directions. This is what a
man-in-the-middle does.
On Sun, Feb 7, 2021 at 7:51 PM Jean L <rip...@gmail.com
<mailto:rip...@gmail.com>> wrote:
Oh cool!
Thanks for the pointer.
One more question: is the ofx data encrypted on the way back to
your side of things? It does not look like it is since you're able
to download your data once you know all the parameter of the
"traditional" ofx query, is that right?
J.
On 2/7/2021 7:28 PM, Scott McRae wrote:
I'm you want something a bit more automated, I came across
mitm-proxy in searches:
https://mitmproxy.org/
This should take care of generating certificates
automatically and actually do the forwarding, etc. You'll need to
generate a CA cert for it and install that in your trusted
certificates. Their docs should explain how. I didn't actually
use it, since my manual method ended up working, but this sounds
better suited for your use case.
On Sun, Feb 7, 2021 at 7:23 PM Jean L <rip...@gmail.com
<mailto:rip...@gmail.com>> wrote:
Wow, that's really cool. I would love to replicate that to be
able to connect to my bank as I'm sure many would. I wonder
if there would be a way to make that a bit easier than
completely manually.
At the moment, I have a python script that logs into my bank,
make the right clicks and downloads the OFX files. Definitely
NOT robust so I would love to be able to go back to
downloading ofx files directly.
Could you possibly write a small blurb on how to do this,
from start to finish? That would be super useful for me. On
the other hand, I'm not sure whether this is 100% within the
law, not sure whether the DMCA has something to say about
this or not :(
Jean
On 2/7/2021 7:06 PM, Scott McRae wrote:
>>/So I decided to give the devil his due and temporarily got a
Quicken />>/subscription and setup an SSL man-in-the-middle. />Sure, you can have a man-in-the-middle setup, but if you don't have the
>keys that quicken and the bank use to communicate and communications are
>encoded, you can't get any data from being in the middle, unless I'm
>missing something.
I generated a self-signed cert and added to the trust store on my Mac OS
keychain. I was actually able to get away with a very manual
man-in-the-middle
using an "openssl s_server" command running on 433, modifying the
/etc/hosts
file to point back to my machine, and copy-pasting the request to curl,
then
copy-pasting the response.
On Sat, Feb 6, 2021 at 8:45 PM Scott McRae <smc...@parax.com
<mailto:smc...@parax.com>> wrote:
I got this working in my software with some help for the
info on this list. Here is a write-up:
USAA's changes to their OFX interface
-------------------------------------
On 2020-01-26, USAA's previous OFX interface
(https://service2.usaa.com/ofx/OFXServlet) stopped
working. It seems like they switched to a new interface
through a tech provider to replace their previous login
method (with your website credentials) to an
app-specific ID and password. This is a good move for
security, but it was done without notice, it seems, to
anyone but Quicken.
From some internet searches, I found some people on the
right track to fixing this on the GNU Cash development
mailing list:
https://lists.gnucash.org/pipermail/gnucash-devel/2021-January/045664.html
They were able to determine that USAA was:
- using a new OFX endpoint:
https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
- using a new OFX Org ID: USAA Federal Savings Bank
- using a new OFX FID: 67811
Additionally, someone on the USAA forums was about to
extract and post the link to generate an App ID and PIN:
source:
https://communities.usaa.com/t5/Finances/USAA-Creates-Quicken-Monopoly/td-p/243850/highlight/false
Authorization link:
https://df3cx-services.1fsapi.com/casm/usaa/enroll
However, with a lot of trial and error I still wasn't
able to hit this new endpoint successfully. So I decided
to give the devil his due and temporarily got a Quicken
subscription and setup an SSL man-in-the-middle.
The new OFX interface is *very* finicky, so you
basically have to input everything exactly the way it
expects it. Here is an example of an account listing
query that works:
echo -en
"OFXHEADER:100\r\nDATA:OFXSGML\r\nVERSION:103\r\nSECURITY:NONE\r\nENCODING:USASCII\r\nCHARSET:NONE\r\nCOMPRESSION:NONE\r\nOLDFILEUID:NONE\r\nNEWFILEUID:NONE\r\n\r\n<OFX>\r\n<SIGNONMSGSRQV1>\r\n<SONRQ>\r\n<DTCLIENT>20210207031942\r\n<USERID>XXXXX\r\n<USERPASS>NNNNN\r\n<LANGUAGE>ENG\r\n<FI>\r\n<ORG>USAA
Federal Savings
Bank\r\n<FID>67811\r\n</FI>\r\n<APPID>QMOFX\r\n<APPVER>2300\r\n<CLIENTUID>1955A543-B071-455E-A31E-73CC7C493D68\r\n</SONRQ>\r\n</SIGNONMSGSRQV1>\r\n<SIGNUPMSGSRQV1>\r\n<ACCTINFOTRNRQ>\r\n<TRNUID>e39add7d-2085-4504-b9ee-be37927de39c\r\n<ACCTINFORQ>\r\n<DTACCTUP>19900101\r\n</ACCTINFORQ>\r\n</ACCTINFOTRNRQ>\r\n</SIGNUPMSGSRQV1>\r\n</OFX>\r\n"
| curl -isS -X POST -H "Content-Type: application/x-ofx"
-A InetClntApp/3.0 --data-binary @-
https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
Note you have to change the XXXXX and NNNNN to the App
ID and PIN you get from the link above.
Some things I've found through trial and error:
- The OFX elements must be separated with "\r\n". This
is dumb, but true. No spaces. No simple "\n". Exactly
"\r\n".
- The APPID "QMOFX" and APPVER "QMOFX" work. Others I
tried did not.
- The CLIENTUID "1955A543-B071-455E-A31E-73CC7C493D68"
works for me. It must be uppercase. This might be
particular to your account. If so, you can find it
looking at the OFX logs from Quicken.
- TRNUID must be present, but an UUID will do.
- DTACCTUP: The value "19900101" works. The value
"19700101" does not. The value "19900101000000" does not.
- You need the "Content-Type: application/x-ofx" header
- You need the User-Agent "InetClntApp/3.0". This is
what Quicken for Mac sends.
It also seems their gateway will under some conditions
put your IP on a ban list. If you are testing, you may
want to spin up an AWS instance or something. When you
get on it, you'll start seeing an empty HTML page
response, like:
<html>
<head>
<META NAME="robots" CONTENT="noindex,nofollow">
<script
src="/_Incapsula_Resource?SWJIYLWA=5074a744e2e3d891814e9a2dace20bd4,719d34d31c8e3a6e6fffd425f7e032f3">
</script>
<body>
</body></html>
Valid queries will work from different source IPs when
this happens.
Thanks to Bob White on the GNU Cash list and RDD! on the
USAA Forums for the breadcrumbs. No thanks to USAA for
swapping out their functional interface with absolutely
no notice or documentation and pretending like Quicken
users are the only customers of any importance. Please
just don't break our software again... at least for awhile.
- Scott McRae
_______________________________________________
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
