The F5 issues were just due to poor environment configuration. Each F5 VIP was sending traffic to the same pool and that pool was only configured for 1 member. That 1 member IP/port was used by several apache virtual hosts. So basically I never knew which virtual host was getting the request which meant some requests were going to listeners not running mod_jk. Always fun joining a new company and having to piece together what someone else has done.
Please remember that over 90% of these configs were created by the Adobe CF webserver config utility so I question some of it as well. Here is what's being used in the uriworkermap.properties: /cfformgateway/* = cfusion /CFFormGateway/* = cfusion /flex2gateway/* = cfusion /flex2gateway = cfusion /cffileservlet/* = cfusion /CFFileServlet/* = cfusion /cfform-internal/* = cfusion /flashservices/gateway/* = cfusion /flex-internal/* = cfusion /rest/* = cfusion /*.cfml/* = cfusion /*.mxml = cfusion /*.as = cfusion /*.cfm = cfusion /*.cfm/* = cfusion /*.swc = cfusion /*.cfml = cfusion /*.cfc = cfusion /*.cfc/* = cfusion /*.cfr = cfusion /*.cfswf = cfusion /*.sws = cfusion /*.jsp = cfusion /*.hbmxml = cfusion This is what I'm seeing in the mod_jk log. I've cut out a few sections of the "Attempting to map context URI" so there's less clutter. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1131): Attempting to map URI '/' from 24 maps [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] jk_translate::mod_jk.c (3723): no match for / found [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1131): Attempting to map URI '/' from 24 maps [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] jk_map_to_storage::mod_jk.c (3798): no match for / found [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] map_uri_to_worker_ext::jk_uri_worker_map.c (1131): Attempting to map URI '/index.cfm' from 24 maps [ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] find_match::jk_uri_worker_map.c (958): Found a wildchar match '/*.cfm=cfusion' [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] jk_handler::mod_jk.c (2621): Into handler jakarta-servlet worker=cfusion r->proxyreq=0 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] wc_get_worker_for_name::jk_worker.c (115): found a worker cfusion [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] wc_get_name_for_type::jk_worker.c (292): Found worker type 'ajp13' [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] init_ws_service::mod_jk.c (1097): Service protocol=HTTP/0.9 method=GET ssl=false host=(null) addr=192.168.253.3 name=app1.dev5.abc.com port=80 auth=(null) user=(null) laddr=192.168.253.61 raddr=192.168.253.3 uri=/ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_get_endpoint::jk_ajp_common.c (3161): acquired connection pool slot=0 after 0 retries [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_marshal_into_msgb::jk_ajp_common.c (626): ajp marshaling done [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_service::jk_ajp_common.c (2450): processing cfusion with 2 retries [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): sending to ajp13 pos=4 len=185 max=8192 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0000 12 34 00 B5 02 02 00 08 48 54 54 50 2F 30 2E 39 - .4......HTTP/0.9 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0010 00 00 01 2F 00 00 0D 31 39 32 2E 31 36 38 2E 32 - .../...192.168.2 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0020 35 33 2E 33 00 FF FF 00 22 63 6F 6D 6D 65 72 63 - 53.3...."app1 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0030 65 2E 64 65 76 35 2E 6C 69 66 65 74 65 63 68 6E - dev5.abc [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0040 6F 6C 6F 67 69 65 73 2E 63 6F 6D 00 00 50 00 00 - .com..P.. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0050 02 A0 09 00 27 54 4C 54 53 49 44 3D 31 38 36 35 - ....'TLTSID=1865 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0060 37 36 41 32 41 33 45 35 31 30 41 33 30 30 30 33 - 76A2A3E510A30003 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0070 45 43 39 33 42 35 31 31 39 39 42 43 00 A0 08 00 - EC93B51199BC.... [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0080 01 30 00 0A 00 0F 41 4A 50 5F 52 45 4D 4F 54 45 - .0....AJP_REMOTE [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 0090 5F 50 4F 52 54 00 00 05 34 31 36 33 30 00 0A 00 - _PORT...41630... [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 00a0 10 4A 4B 5F 4C 42 5F 41 43 54 49 56 41 54 49 4F - .JK_LB_ACTIVATIO [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (1184): 00b0 4E 00 00 03 41 43 54 00 FF 00 00 00 00 00 00 00 - N...ACT......... [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_send_request::jk_ajp_common.c (1673): (cfusion) Statistics about invalid connections: connect check (0), cping (0), send (0) [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_send_request::jk_ajp_common.c (1684): (cfusion) request body to send 0 - request body to resend 0 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): received from ajp13 pos=0 len=474 max=8192 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0000 04 01 2E 00 11 4D 6F 76 65 64 20 54 65 6D 70 6F - .....Moved.Tempo [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0010 72 61 72 69 6C 79 00 00 08 A0 07 00 43 43 46 49 - rarily......CCFI [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0020 44 3D 36 39 38 37 31 3B 20 45 78 70 69 72 65 73 - D=69871;.Expires [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0030 3D 54 68 75 2C 20 32 35 2D 46 65 62 2D 32 30 34 - =Thu,.25-Feb-204 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0040 34 20 32 31 3A 33 36 3A 35 30 20 47 4D 54 3B 20 - 4.21:36:50.GMT;. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0050 50 61 74 68 3D 2F 3B 20 48 74 74 70 4F 6E 6C 79 - Path=/;.HttpOnly [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0060 00 A0 07 00 75 43 46 54 4F 4B 45 4E 3D 33 39 65 - ....uCFTOKEN=39e [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0070 61 33 32 37 38 62 30 65 39 32 31 33 37 2D 39 36 - a3278b0e92137-96 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0080 33 41 43 43 35 42 2D 35 30 35 36 2D 39 32 30 31 - 3ACC5B-5056-9201 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0090 2D 36 32 44 32 38 41 43 35 46 30 35 34 30 35 45 - -62D28AC5F05405E [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00a0 34 3B 20 45 78 70 69 72 65 73 3D 54 68 75 2C 20 - 4;.Expires=Thu,. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00b0 32 35 2D 46 65 62 2D 32 30 34 34 20 32 31 3A 33 - 25-Feb-2044.21:3 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00c0 36 3A 35 30 20 47 4D 54 3B 20 50 61 74 68 3D 2F - 6:50.GMT;.Path=/ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00d0 3B 20 48 74 74 70 4F 6E 6C 79 00 A0 07 00 81 43 - ;.HttpOnly.....C [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00e0 4B 5F 55 52 4C 54 4F 4B 45 4E 3D 43 46 49 44 25 - K_URLTOKEN=CFID% [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 00f0 33 44 36 39 38 37 31 25 32 36 43 46 54 4F 4B 45 - 3D69871%26CFTOKE [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0100 4E 25 33 44 33 39 65 61 33 32 37 38 62 30 65 39 - N%3D39ea3278b0e9 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0110 32 31 33 37 25 32 44 39 36 33 41 43 43 35 42 25 - 2137%2D963ACC5B% [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0120 32 44 35 30 35 36 25 32 44 39 32 30 31 25 32 44 - 2D5056%2D9201%2D [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0130 36 32 44 32 38 41 43 35 46 30 35 34 30 35 45 34 - 62D28AC5F05405E4 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0140 3B 20 44 6F 6D 61 69 6E 3D 2E 69 6E 76 69 74 72 - ;.Domain=.abc [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0150 6F 67 65 6E 2E 63 6F 6D 3B 20 50 61 74 68 3D 2F - .com;.Path=/ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0160 00 00 0C 73 65 72 76 65 72 2D 65 72 72 6F 72 00 - ...server-error. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0170 00 04 74 72 75 65 00 00 08 6C 6F 63 61 74 69 6F - ..true...locatio [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0180 6E 00 00 0A 2F 65 72 72 6F 72 2E 68 74 6D 00 00 - n.../error.htm.. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0190 0D 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 00 00 - .Cache-Control.. [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 01a0 08 6E 6F 2D 63 61 63 68 65 00 00 06 50 72 61 67 - .no-cache...Prag [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 01b0 6D 61 00 00 08 6E 6F 2D 63 61 63 68 65 00 A0 01 - ma...no-cache... [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 01c0 00 17 74 65 78 74 2F 68 74 6D 6C 3B 63 68 61 72 - ..text/html;char [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 01d0 73 65 74 3D 55 54 46 2D 38 00 00 00 00 00 00 00 - set=UTF-8....... [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (681): status = 302 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (688): Number of headers is = 8 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[0] [Set-Cookie] = [CFID=69871; Expires=Thu, 25-Feb-2044 21:36:50 GMT; Path=/; HttpOnly] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[1] [Set-Cookie] = [CFTOKEN=39ea3278b0e92137-963ACC5B-5056-9201-62D28AC5F05405E4; Expires=Thu, 25-Feb-2044 21:36:50 GMT; Path=/; HttpOnly] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[2] [Set-Cookie] = [CK_URLTOKEN=CFID%3D69871%26CFTOKEN%3D39ea3278b0e92137%2D963ACC5B%2D5056%2D9201%2D62D28AC5F05405E4; Domain=.abc.com; Path=/] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[3] [server-error] = [true] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[4] [location] = [/error.htm] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[5] [Cache-Control] = [no-cache] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[6] [Pragma] = [no-cache] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_unmarshal_response::jk_ajp_common.c (744): Header[7] [Content-Type] = [text/html;charset=UTF-8] [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): received from ajp13 pos=0 len=4 max=8192 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0000 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): received from ajp13 pos=0 len=2 max=8192 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1379): 0000 05 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................ [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_process_callback::jk_ajp_common.c (2004): AJP13 protocol: Reuse is OK [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_reset_endpoint::jk_ajp_common.c (786): (cfusion) resetting endpoint with socket 25 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] ajp_done::jk_ajp_common.c (3092): recycling connection pool for worker cfusion and socket 25 [Tue Mar 04 16:36:50 2014] [5763:140265396258560] [debug] jk_handler::mod_jk.c (2772): Service finished with status=302 for worker=cfusion
