Michael, I find the descriptions in [ https://mojolicious.org/perldoc/Mojo/UserAgent/Transactor#tx | https://mojolicious.org/perldoc/Mojo/UserAgent/Transactor#tx ] pretty helpful
cheers tobi ----- On May 14, 2020, at 2:06 PM, 'Michael Lackhoff' via Mojolicious <mojolicious@googlegroups.com> wrote: > Well, the easiest solution would be to just use curl. > But M::U is a powerful tool I find myself using it more and more and I would > like to better understand such tools. > It looks great but if it has some severe shortcomings I want to know before > doing lots of mission critical stuff with it. > But thanks for your suggestion. It is always good to know further options. > -Michael > Am Donnerstag, 14. Mai 2020 13:47:41 UTC+2 schrieb Felipe Gasper: >> FWIW, you can use libcurl with Mojo via Net::Curl::Promiser::Mojo. That may >> be >> your easiest solution? >> -FG >>> On May 14, 2020, at 07:38, 'Michael Lackhoff' via Mojolicious < >>> mojol...@googlegroups.com > wrote: >>> Am 14.05.2020 um 05:08 schrieb Stefan Adams: >>>> > I'm certain you'll be able to accomplish this with M::UA. Can you share >>> Just to reduce complexity I changed everything to plain HTTP, so no SSL, >>> keys or certificates. >>>> > what the HTML error produced is? >>> It is not really a normal error page, just a page from the UI. The most >>> relevant lines are these: >>> <p class="ErrorMsg">Invalid variable name.</p> >>> <p>Wiederholen Sie das Update oder starten Sie die FRITZ!Box neu.</p> >>> [German for: repeat the update(sic! I wanted to backup the box, not >>> update) or restart your FRITZ!Box] >>>> > Can you share the output that you are comparing that you are noticing is >>>> > so >>>> > similar between curl and M::UA? You're using -v for curl. Also set the >>>> > MOJO_CLIENT_DEBUG env variable to 1. >>> Here is what curl gives (with --trace-ascii): >>> => Send header, 200 bytes (0xc8) >>> 0000: POST /cgi-bin/firmwarecfg HTTP/1.1 >>> 0024: Host: fritz.box >>> 0035: User-Agent: curl/7.58.0 >>> 004e: Accept: */* >>> 005b: Content-Length: 370 >>> 0070: Content-Type: multipart/form-data; boundary=-------------------- >>> 00b0: ----565a580ebeb0c567 >>> 00c6: >>> => Send data, 370 bytes (0x172) >>> 0000: --------------------------565a580ebeb0c567 >>> 002c: Content-Disposition: form-data; name="sid" >>> 0058: >>> 005a: 4e240384c30c909d >>> 006c: --------------------------565a580ebeb0c567 >>> 0098: Content-Disposition: form-data; name="ImportExportPassword" >>> 00d5: >>> 00d7: backup >>> 00df: --------------------------565a580ebeb0c567 >>> 010b: Content-Disposition: form-data; name="ConfigExport" >>> 0140: >>> 0142: >>> 0144: --------------------------565a580ebeb0c567-- >>> <= Recv header, 17 bytes (0x11) >>> 0000: HTTP/1.1 200 OK >>>> > See how it works comparing curl to: >>>> > $ env MOJO_CLIENT_DEBUG=1 mojo get -v -M POST -f sid=$SID -f >>>> > ImportExportPassword=$BAKPWD -f ConfigExport= http >>>>> ://fritz.box/cgi-bin/firmwarecfg [ http://fritz.box/cgi-bin/firmwarecfg | >>>> > <http://fritz.box/cgi-bin/firmwarecfg> ] >>> The empty ConfigExport parameter gets lost with this command and the >>> Content-Type is different: >>> -- Blocking request ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> -- Connect 7c84d2d7396d9d2ff9571982f73240cb ( [ http://fritz.box/ | >>> http://fritz.box:80 ] ) >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> POST /cgi-bin/firmwarecfg HTTP/1.1\x0d >>> Content-Length: 48\x0d >>> User-Agent: Mojolicious (Perl)\x0d >>> Accept-Encoding: gzip\x0d >>> Content-Type: application/x-www-form-urlencoded\x0d >>> Host: fritz.box\x0d >>> \x0d >>> ImportExportPassword=backup&sid=4e240384c30c909d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> -- Client <<< Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> HTTP/1.1 200 OK\x0d >>> With DEBUG on and my code (s. my original message and below) I get this: >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> POST /cgi-bin/firmwarecfg HTTP/1.1\x0d >>> Accept-Encoding: gzip, deflate\x0d >>> Content-Type: multipart/form-data; boundary=tyUPX\x0d >>> Accept: */*\x0d >>> Host: fritz.box\x0d >>> Content-Length: 230\x0d >>> User-Agent: Mojolicious (Perl)\x0d >>> \x0d >>> --tyUPX\x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> Content-Disposition: form-data; name="ConfigExport"\x0d >>> \x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> \x0d >>> --tyUPX\x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> Content-Disposition: form-data; name="ImportExportPassword"\x0d >>> \x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> backup >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> \x0d >>> --tyUPX\x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> Content-Disposition: form-data; name="sid"\x0d >>> \x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> 33cb272bcd34940b >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> \x0d >>> --tyUPX--\x0d >>> -- Client >>> Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> -- Client <<< Server ( [ http://fritz.box/cgi-bin/firmwarecfg | >>> http://fritz.box/cgi-bin/firmwarecfg ] ) >>> HTTP/1.1 200 OK\x0d >>> which for me looks very much the same as the curl version but as I said, >>> curl gives the backup file whereas M::U just gives the HTML page back, >>> indicating that something went wrong. >>> I can only spot very minor differences that is why I am running out of >>> ideas: >>> - the boundary used by M::U is quite short >>> - the form parameters are not in the same order >>> - Accept-Encoding header (is also sent from Firefox where it works) >>> - the different SID is normal (my script always makes a fresch login) >>> Looks very strange to me >>> -Michael >>>> > On Wed, May 13, 2020, 4:56 PM 'Michael Lackhoff' via Mojolicious < >>>> > mojol...@googlegroups.com > wrote: >>>>> >> I had quite a bit of success recently using Mojo::UserAgent so I tried >>>>> >> to >>>>> >> replace a curl command to do a backup of my Fritz.box router with M::U. >>>>> >> Here is the curl command: >>>>> >> curl -s -k -o $OUT --form sid=$SID --form ImportExportPassword=$BAKPWD >>>>> >> \ >>>>>>> --form ConfigExport= [ http://fritz.box/cgi-bin/firmwarecfg | >>>>> >> http://fritz.box/cgi-bin/firmwarecfg ] >> >>>>> >> It should be equivalent to this M::U request: >>>>> >> my $tx = $ua->build_tx( >>>>>>> POST => ' [ http://fritz.box/cgi-bin/firmwarecfg | >>>>> >> http://fritz.box/cgi-bin/firmwarecfg ] ' => >>>>> >> { >>>>> >> 'Accept' => '*/*', >>>>> >> 'Content-Type' => 'multipart/form-data', >>>>> >> } => form => { >>>>> >> sid => $SID, >>>>> >> ImportExportPassword => $BAKPWD, >>>>> >> ConfigExport => '', >>>>> >> } >>>>> >> ); >>>>> >> # for debugging: >>>>> >> print $tx->req->to_string; >>>>> >> $tx = $ua->start($tx); >>>>> >> $tx->res->save_to($OUT); >>>>> >> As far as I can tell both the headers and the POST body is very much >>>>> >> the >>>>> >> same (except the boundary value to separate the form fields) but to my >>>>> >> surprise the curl command works ($OUT is the backup file) but with the >>>>> >> M::U >>>>> >> version $OUT consists of some HTML output indicating an error. >>>>> >> If I could see a difference I could try to better adjust my script but >>>>> >> as >>>>> >> I said, they look very much the same (I compared it with the -v and >>>>> >> --trace-ascii output of curl), so I run out of ideas what could >>>>> >> trigger the >>>>> >> differnt response of my Fritz.box. >>>>> >> Any ideas? At the moment I just solve it by using the curl command with >>>>> >> "system" but I would prefer a Perl-only solution and what is even more >>>>> >> important to me: I want to understand what is going on here. >>>>> >> -Michael >>>>> >> -- >>>>> >> You received this message because you are subscribed to the Google >>>>> >> Groups >>>>> >> "Mojolicious" group. >>>>> >> To unsubscribe from this group and stop receiving emails from it, send >>>>> >> an >>>>> >> email to mojol...@googlegroups.com . >>>>> >> To view this discussion on the web visit >>>>>>> [ >>>>>>> https://groups.google.com/d/msgid/mojolicious/14df452c-e7a4-4b7c-90a2-9e1f5e15becc%40googlegroups.com >>>>>>> | >>>>>>> https://groups.google.com/d/msgid/mojolicious/14df452c-e7a4-4b7c-90a2-9e1f5e15becc%40googlegroups.com >>>>>>> ] >> [ >>>>>>> https://groups.google.com/d/msgid/mojolicious/14df452c-e7a4-4b7c-90a2-9e1f5e15becc%40googlegroups.com?utm_medium=email&utm_source=footer >>>>>>> | >>>>>>> <https://groups.google.com/d/msgid/mojolicious/14df452c-e7a4-4b7c-90a2-9e1f5e15becc%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> >> ] >> . >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Mojolicious" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email >>> to mojol...@googlegroups.com . >>> To view this discussion on the web visit [ >>> https://groups.google.com/d/msgid/mojolicious/6bd73f2c-36ba-4a21-992d-b699fae6f158%40googlegroups.com?utm_medium=email&utm_source=footer >>> | >>> https://groups.google.com/d/msgid/mojolicious/6bd73f2c-36ba-4a21-992d-b699fae6f158%40googlegroups.com >>> ] . > -- > You received this message because you are subscribed to the Google Groups > "Mojolicious" group. > To unsubscribe from this group and stop receiving emails from it, send an > email > to [ mailto:mojolicious+unsubscr...@googlegroups.com | > mojolicious+unsubscr...@googlegroups.com ] . > To view this discussion on the web visit [ > https://groups.google.com/d/msgid/mojolicious/03a573c0-5df4-448c-9256-bb851abf7285%40googlegroups.com?utm_medium=email&utm_source=footer > | > https://groups.google.com/d/msgid/mojolicious/03a573c0-5df4-448c-9256-bb851abf7285%40googlegroups.com > ] . -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland www.oetiker.ch t...@oetiker.ch +41 62 775 9902 -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/mojolicious/1185346809.40923.1589458649675.JavaMail.zimbra%40oetiker.ch.
