can you provide me the service url privately? to try to reproduce it. generally i am not opposed to utf8 as default file encoding, but this touches OJ as a whole. we might break things. so i'd rather check why the WMSRequest does not interpret the charset correctly. maybe it's simply not set?
..ede On 08.01.2020 11:34, Michaud Michael wrote: > Hi Ede, > > I've had a strange problem since a few days which is not related to your fix > but > to WMS and encoding. > > java.io.IOException: org.xml.sax.SAXParseException; lineNumber: 1; > columnNumber: > 49263; Invalid byte 3 of 3-byte UTF-8 sequence. > at > com.vividsolutions.wms.AbstractParser.parseCapabilities(AbstractParser.java:95) > at com.vividsolutions.wms.WMService.initialize(WMService.java:152) > at > com.vividsolutions.jump.workbench.ui.plugin.wms.URLWizardPanel.exitingToRight(URLWizardPanel.java:172) > > A version of OpenJUMP which used to work (1.4.1) suddenly stopped working with > this error. > I get the same error with the last NB (r6216) and I could localize the faulty > string : "ortho‐images" > The character in the middle is an hyphen (U+2010 = 3 hexa bytes : e2 80 90 ). > > What is curious is : > - e2 80 90 seems to be a valid UTF-8 sequence > - GetCapability response String containing this sequence is right (declared in > UTF-8 parsed correctly by any browser or xml editor) > - if I test the last OpenJUMP in dev mode (running OpenJUMP from my IDE), I do > not get the error message ! (and I get the layers from the GetCapability) > > Any idea ? > > Oh, analyzing the starting process in dev mode vs in production mode, I > noticed > that in dev mode, java is started with -Dfile.encoding=UTF-8 > > Now, adding -Dfile.encoding=UTF-8 in my bat file seems to solve the problem. > > Anyway, I don't fully understand the problem. Isn't GetCapabilities parsing > supposed to use the xml encoding declaration ? > > Any hint on this ? > > I the meantime, I will try to add -Dfile.encoding=UTF-8 on different > distribution to check it solves the problem (but I think it may have drawbacks > with plugins using the default platform encoding like csv reader). > > Michaël > >> envoyé : 4 janvier 2020 à 20:00 >> de : edgar.sol...@web.de >> à : OpenJump develop and use <jump-pilot-devel@lists.sourceforge.net> >> objet : [JPP-Devel] Fwd: SVN: [6214] core/trunk >> >> >> Mike, >> >> could you please check the fix and close the ticket if it works as >> expected?.. >> thanks ede >> >> >> -------- Forwarded Message -------- >> Subject: [JPP-Devel] SVN: [6214] core/trunk >> Date: Sat, 04 Jan 2020 18:58:23 +0000 >> From: jump-pilot-svn--- via Jump-pilot-devel >> <jump-pilot-devel@lists.sourceforge.net> >> Reply-To: OpenJump develop and use <jump-pilot-devel@lists.sourceforge.net> >> To: jump-pilot-devel@lists.sourceforge.net >> CC: jump-pilot-...@lists.sourceforge.net >> >> Revision: 6214 >> http://sourceforge.net/p/jump-pilot/code/6214 >> Author: edso >> Date: 2020-01-04 18:58:23 +0000 (Sat, 04 Jan 2020) >> Log Message: >> ----------- >> bugfix #489 "Veneto Region (Italy) WMS service does not work on OpenJump" >> wms now follows http redirections by default >> also >> some better error handling/reporting >> Edit WMS resets connection now on manual url editing >> >> Modified Paths: >> -------------- >> core/trunk/ChangeLog >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/EditWMSQueryPanel.java >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/MapLayerPanel.java >> core/trunk/src/com/vividsolutions/wms/AbstractWMSRequest.java >> core/trunk/src/com/vividsolutions/wms/BasicRequest.java >> core/trunk/src/com/vividsolutions/wms/FeatureInfoRequest.java >> core/trunk/src/com/vividsolutions/wms/MapRequest.java >> core/trunk/src/com/vividsolutions/wms/WMService.java >> core/trunk/src/org/openjump/util/URLConnectionProvider.java >> >> Modified: core/trunk/ChangeLog >> =================================================================== >> --- core/trunk/ChangeLog 2020-01-01 09:21:44 UTC (rev 6213) >> +++ core/trunk/ChangeLog 2020-01-04 18:58:23 UTC (rev 6214) >> @@ -3,6 +3,10 @@ >> # 2. make sure that lines break at 80 chars for constricted display >> situations >> #<-------------------------------- 80 chars >> ----------------------------------># >> >> +2020-01-04 ede >> + * bugfix #489 "Veneto Region (Italy) WMS service does not work on OpenJump" >> + wms now follows http redirections by default >> + >> 2020-01-01 mmichaud <m.michael.mich...@orange.fr> >> * Clean and improve ExtractLayersByAttribute >> * Implements FR #262 : copy info to clipboard (patch from Rashad) >> >> Modified: >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/EditWMSQueryPanel.java >> =================================================================== >> --- >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/EditWMSQueryPanel.java >> 2020-01-01 09:21:44 UTC (rev 6213) >> +++ >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/EditWMSQueryPanel.java >> 2020-01-04 18:58:23 UTC (rev 6214) >> @@ -57,6 +57,7 @@ >> import javax.swing.border.Border; >> import javax.swing.event.DocumentEvent; >> import javax.swing.event.DocumentListener; >> +import javax.swing.text.JTextComponent; >> >> import org.apache.commons.lang3.ArrayUtils; >> import org.openjump.core.ui.plugin.wms.AddWmsLayerWizard; >> @@ -63,6 +64,7 @@ >> import org.openjump.util.UriUtil; >> >> import com.vividsolutions.jump.I18N; >> +import com.vividsolutions.jump.workbench.JUMPWorkbench; >> import com.vividsolutions.jump.workbench.model.WMSLayer; >> import com.vividsolutions.jump.workbench.plugin.EnableCheck; >> import com.vividsolutions.jump.workbench.ui.InputChangedListener; >> @@ -303,7 +305,6 @@ >> private void reset(DocumentEvent e) { >> System.out.println(e); >> resetConnection(); >> - ; >> } >> }; >> >> @@ -329,13 +330,19 @@ >> SwingUtilities.invokeLater(new Runnable() { >> public void run() { >> for (final Component c : cs) { >> - // System.out.println(c); >> + //System.out.println(c); >> if (c instanceof AbstractButton) >> ((AbstractButton) c).addActionListener(ali); >> else if (c instanceof JTextField) >> ((JTextField) c).getDocument().addDocumentListener(doli); >> - else if (c instanceof JComboBox) >> + else if (c instanceof JComboBox) { >> ((JComboBox) c).addActionListener(ali); >> + // also reset if url is edited manually >> + Component editor = ((JComboBox) c).getEditor().getEditorComponent(); >> + if ( editor instanceof JTextComponent) { >> + ((JTextComponent)editor).getDocument().addDocumentListener(doli); >> + } >> + } >> } >> } >> }); >> @@ -357,9 +364,10 @@ >> protected void reinitializeService() { >> String url = urlPanel.getUrl(); >> url = UriUtil >> - .urlAddCredentials(url, urlPanel.getUser(), urlPanel.getPass()); >> + .urlAddCredentials(url, urlPanel.getUser(), urlPanel.getPass()).trim(); >> >> - url = WMService.legalize(url); >> +// [ed] url is now legalized b4 every request >> +// url = WMService.legalize(url); >> // [UT] 20.04.2005 >> WMService service = new WMService(url, URLWizardPanel.wmsVersion); >> >> @@ -369,8 +377,8 @@ >> mapLayerPanel.init(service, layer.getLayerNames()); >> connectButton.setEnabled(false); >> } catch (Exception e) { >> - e.printStackTrace(); >> mapLayerPanel.reset(); >> + JUMPWorkbench.getInstance().getFrame().handleThrowable(e); >> } finally { >> refreshParamCombos(); >> } >> >> Modified: >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/MapLayerPanel.java >> =================================================================== >> --- >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/MapLayerPanel.java >> 2020-01-01 09:21:44 UTC (rev 6213) >> +++ >> core/trunk/src/com/vividsolutions/jump/workbench/ui/plugin/wms/MapLayerPanel.java >> 2020-01-04 18:58:23 UTC (rev 6214) >> @@ -339,7 +339,7 @@ >> } >> >> public void reset() { >> - this.service = null; >> + //this.service = null; >> ((TreeAddRemoveList) addRemovePanel.getLeftList()).setModel(new >> TreeAddRemoveListModel(new DefaultTreeModel(null))); >> addRemovePanel.getRightList().getModel().setItems(new ArrayList()); >> addRemovePanel.updateEnabled(); >> >> Modified: core/trunk/src/com/vividsolutions/wms/AbstractWMSRequest.java >> =================================================================== >> --- core/trunk/src/com/vividsolutions/wms/AbstractWMSRequest.java 2020-01-01 >> 09:21:44 UTC (rev 6213) >> +++ core/trunk/src/com/vividsolutions/wms/AbstractWMSRequest.java 2020-01-04 >> 18:58:23 UTC (rev 6214) >> @@ -70,14 +70,9 @@ >> */ >> protected HttpURLConnection prepareConnection() throws IOException { >> URL requestUrl = getURL(); >> - con = (HttpURLConnection) >> URLConnectionProvider.getJUMP_URLConnectionProvider().getConnection(requestUrl); >> - con = (HttpURLConnection) requestUrl.openConnection(); >> + // by default we follow redirections >> + con = (HttpURLConnection) >> URLConnectionProvider.getJUMP_URLConnectionProvider().getHttpConnection(requestUrl, >> true); >> >> - con.setConnectTimeout(Integer.parseInt( >> - >> ProxySettingsOptionsPanel.getInstance().getSetting(ProxySettingsOptionsPanel.OPEN_TIMEOUT_KEY).toString())); >> - con.setReadTimeout(Integer.parseInt( >> - >> ProxySettingsOptionsPanel.getInstance().getSetting(ProxySettingsOptionsPanel.OPEN_TIMEOUT_KEY).toString())); >> - >> // add this service's auth info >> String userInfo = requestUrl.getUserInfo(); >> if (userInfo != null) { >> @@ -102,6 +97,7 @@ >> public HttpURLConnection getConnection() throws IOException { >> if (con == null) >> con = prepareConnection(); >> + >> return con; >> } >> >> @@ -148,27 +144,32 @@ >> */ >> public String getText() throws IOException { >> HttpURLConnection con = getConnection(); >> - return readConnection(con, 0, false); >> + boolean httpOk = con.getResponseCode() == HttpURLConnection.HTTP_OK; >> + if (!httpOk) >> + readToError(con); >> + >> + return readConnection(con, 0); >> } >> >> protected String readToError(HttpURLConnection con) throws IOException { >> - return readConnection(con, 1024, true); >> + String url = con.getURL().toExternalForm(); >> + String headers = con.getHeaderFields().toString(); >> + String result = readConnection(con, 1024); >> + throw new WMSException( "Request url: " + url + >> + "\nResponse code: " + con.getResponseCode() + "\nHeaders:\n" + headers + >> "\nResponse body:\n" + result); >> } >> >> - protected String readConnection(HttpURLConnection con, long limit, >> - boolean throwError) throws IOException { >> + protected String readConnection(HttpURLConnection con, long limit) throws >> IOException { >> boolean httpOk = con.getResponseCode() == HttpURLConnection.HTTP_OK; >> // get correct stream >> InputStream in = httpOk ? con.getInputStream() : con.getErrorStream(); >> - // limit max chars >> - BoundedInputStream bin = new BoundedInputStream(in, limit > 0 ? limit : >> -1); >> >> - String result = IOUtils.toString(bin); >> - FileUtil.close(bin); >> - >> - if (throwError) { >> - throw new WMSException("Response code: " + con.getResponseCode() >> - + "\nResponse body:\n" + result); >> + String result = ""; >> + if (in!=null) { >> + // limit max chars >> + BoundedInputStream bin = new BoundedInputStream(in, limit > 0 ? limit : >> -1); >> + result = IOUtils.toString(bin); >> + FileUtil.close(bin); >> } >> >> return result; >> >> Modified: core/trunk/src/com/vividsolutions/wms/BasicRequest.java >> =================================================================== >> --- core/trunk/src/com/vividsolutions/wms/BasicRequest.java 2020-01-01 >> 09:21:44 UTC (rev 6213) >> +++ core/trunk/src/com/vividsolutions/wms/BasicRequest.java 2020-01-04 >> 18:58:23 UTC (rev 6214) >> @@ -22,4 +22,5 @@ >> public URL getURL() throws MalformedURLException { >> return url; >> } >> + >> } >> >> Modified: core/trunk/src/com/vividsolutions/wms/FeatureInfoRequest.java >> =================================================================== >> --- core/trunk/src/com/vividsolutions/wms/FeatureInfoRequest.java 2020-01-01 >> 09:21:44 UTC (rev 6213) >> +++ core/trunk/src/com/vividsolutions/wms/FeatureInfoRequest.java 2020-01-04 >> 18:58:23 UTC (rev 6214) >> @@ -46,14 +46,14 @@ >> >> @Override >> public URL getURL() throws MalformedURLException { >> - String featInfoUrl = service.getCapabilities().getFeatureInfoURL(); >> + String featInfoUrl = >> WMService.legalize(service.getCapabilities().getFeatureInfoURL()); >> >> - if (featInfoUrl.contains("?")) { >> - if (!featInfoUrl.endsWith("?")) >> - featInfoUrl += "&"; >> - } else { >> - featInfoUrl += "?"; >> - } >> +// if (featInfoUrl.contains("?")) { >> +// if (!featInfoUrl.endsWith("?")) >> +// featInfoUrl += "&"; >> +// } else { >> +// featInfoUrl += "?"; >> +// } >> >> if (WMService.WMS_1_0_0.equals(version)) { >> featInfoUrl += "REQUEST=feature_info&WMTVER=1.0.0"; >> >> Modified: core/trunk/src/com/vividsolutions/wms/MapRequest.java >> =================================================================== >> --- core/trunk/src/com/vividsolutions/wms/MapRequest.java 2020-01-01 09:21:44 >> UTC (rev 6213) >> +++ core/trunk/src/com/vividsolutions/wms/MapRequest.java 2020-01-04 18:58:23 >> UTC (rev 6214) >> @@ -244,7 +244,8 @@ >> } else if ( WMService.WMS_1_3_0.equals( version ) ){ >> ver = "REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0"; >> } >> - urlBuf.append( service.getCapabilities().getGetMapURL() + ver + "&WIDTH=" + >> imgWidth + "&HEIGHT=" + imgHeight ); >> + urlBuf.append(WMService.legalize(service.getCapabilities().getGetMapURL()) >> + >> ver + "&WIDTH=" + imgWidth + "&HEIGHT=" >> + + imgHeight); >> try { >> urlBuf.append( "&LAYERS=" + encode(listToString( layerNames ), "UTF-8") ); >> } catch (UnsupportedEncodingException e1) { >> >> Modified: core/trunk/src/com/vividsolutions/wms/WMService.java >> =================================================================== >> --- core/trunk/src/com/vividsolutions/wms/WMService.java 2020-01-01 09:21:44 >> UTC (rev 6213) >> +++ core/trunk/src/com/vividsolutions/wms/WMService.java 2020-01-04 18:58:23 >> UTC (rev 6214) >> @@ -50,10 +50,10 @@ >> import javax.net.ssl.*; >> import javax.swing.JOptionPane; >> >> +import org.apache.commons.io.IOUtils; >> import org.openjump.util.UriUtil; >> >> import com.vividsolutions.jump.I18N; >> -import com.vividsolutions.jump.workbench.ui.ErrorDialog; >> >> /** >> * Represents a remote WMS Service. >> @@ -143,13 +143,13 @@ >> } >> >> try { >> - String requestUrlString = this.serverUrl + req; >> + String requestUrlString = WMService.legalize(this.serverUrl.toString()) + >> req; >> URL requestUrl = new URL(requestUrlString); >> >> URLConnection con = new BasicRequest(this, requestUrl).getConnection(); >> >> - // Parser p = new Parser(); >> - cap = parser.parseCapabilities(this, con.getInputStream()); >> + String out = new BasicRequest(this, requestUrl).getText(); >> + cap = parser.parseCapabilities(this, IOUtils.toInputStream(out) ); >> String url1 = cap.getService().getServerUrl(); >> String url2 = cap.getGetMapURL(); >> >> >> Modified: core/trunk/src/org/openjump/util/URLConnectionProvider.java >> =================================================================== >> --- core/trunk/src/org/openjump/util/URLConnectionProvider.java 2020-01-01 >> 09:21:44 UTC (rev 6213) >> +++ core/trunk/src/org/openjump/util/URLConnectionProvider.java 2020-01-04 >> 18:58:23 UTC (rev 6214) >> @@ -3,7 +3,9 @@ >> import com.vividsolutions.jump.I18N; >> import com.vividsolutions.jump.util.Blackboard; >> import com.vividsolutions.jump.workbench.JUMPWorkbench; >> +import com.vividsolutions.jump.workbench.Logger; >> import com.vividsolutions.jump.workbench.plugin.PlugInContext; >> +import >> com.vividsolutions.jump.workbench.ui.network.ProxySettingsOptionsPanel; >> import >> com.vividsolutions.jump.workbench.ui.plugin.PersistentBlackboardPlugIn; >> >> import javax.net.ssl.HttpsURLConnection; >> @@ -12,8 +14,10 @@ >> import javax.net.ssl.X509TrustManager; >> import javax.swing.*; >> import java.io.IOException; >> +import java.net.HttpURLConnection; >> import java.net.URL; >> import java.net.URLConnection; >> +import java.net.URLDecoder; >> import java.security.KeyManagementException; >> import java.security.NoSuchAlgorithmException; >> import java.security.cert.X509Certificate; >> @@ -49,13 +53,63 @@ >> this.authorizedURL = (Set<String>)this.blackboard.get(KEY, new >> HashSet<String>()); >> } >> >> + public HttpURLConnection getHttpConnection(URL url, boolean >> followRedirects) >> throws IOException { >> + HttpURLConnection connection = (HttpURLConnection) getHttpConnection(url); >> + >> + // we handle redirects ourselfs >> + connection.setInstanceFollowRedirects(false); >> + >> + URL prev = null, next = connection.getURL(); >> + while (followRedirects && !next.equals(prev) ) { >> + connection = getHttpConnection(next); >> + // we handle redirects ourselfs >> + connection.setInstanceFollowRedirects(false); >> + >> + switch (connection.getResponseCode()) >> + { >> + case HttpURLConnection.HTTP_MOVED_PERM: >> + case HttpURLConnection.HTTP_MOVED_TEMP: >> + String location = connection.getHeaderField("Location"); >> + location = URLDecoder.decode(location, "UTF-8"); >> + prev = connection.getURL(); >> + next = new URL(prev, location); // compute relative URLs >> + Logger.warn("Follow http redirect to: "+next); >> + continue; >> + } >> + break; >> + } >> + >> + return connection; >> + } >> + >> + /** >> + * @deprecated use getHttpConnection(url,followRedirects) instead >> + * @param url >> + * @return >> + * @throws IOException >> + */ >> + @Deprecated >> public URLConnection getConnection(URL url) throws IOException { >> + return getHttpConnection(url, true); >> + } >> + >> + public HttpURLConnection getHttpConnection(URL url) throws IOException { >> String protocol = url.getProtocol(); >> - if (!protocol.equals("https")) return url.openConnection(); >> - URLConnection connection; >> +// if (!protocol.equals("https")) return url.openConnection(); >> + >> + if (!protocol.matches("^(?i:https?)$")) >> + throw new IOException("Please provide an http(s):// url."); >> + >> + HttpURLConnection connection = (HttpURLConnection)url.openConnection(); >> + >> + // apply timeouts from settings >> + connection.setConnectTimeout(Integer.parseInt( >> + >> ProxySettingsOptionsPanel.getInstance().getSetting(ProxySettingsOptionsPanel.OPEN_TIMEOUT_KEY).toString())); >> + connection.setReadTimeout(Integer.parseInt( >> + >> ProxySettingsOptionsPanel.getInstance().getSetting(ProxySettingsOptionsPanel.READ_TIMEOUT_KEY).toString())); >> + >> try { >> - setTrustOption(false, null); >> - connection = url.openConnection(); >> + setTrustOption(false, url); >> connection.connect(); // try to connect >> return connection; // can connect >> } catch(IOException|KeyManagementException|NoSuchAlgorithmException e) { >> @@ -63,7 +117,7 @@ >> if (authorizedURL.contains(baseURL) || acceptConnection(url)) { >> try { >> setTrustOption(true, url); >> - connection = url.openConnection(); >> + connection = (HttpURLConnection) url.openConnection(); >> authorizedURL.add(baseURL); >> //setTrustOption(false, null); >> return connection; >> @@ -96,15 +150,22 @@ >> }; >> private Set<URL> trustedURLs = new HashSet<>(); >> >> - private void setTrustOption(boolean trust, URL url) >> - throws KeyManagementException, NoSuchAlgorithmException { >> + /** >> + * >> + * @param trust >> + * @param url >> + * @throws KeyManagementException >> + * @throws NoSuchAlgorithmException >> + */ >> + private void setTrustOption(boolean trust, URL url) throws >> KeyManagementException, NoSuchAlgorithmException { >> SSLContext sc = SSLContext.getInstance("SSL"); >> + String host = url != null ? url.getHost() : ""; >> if (trust || (url != null && trustedURLs.contains(url))) { >> - System.out.println("Set the trust manager to not check certificates"); >> - sc.init(null, new TrustManager[]{trm}, null); >> + Logger.info("Certificate verification for trusted host '" + host + "' is >> disabled'"); >> + sc.init(null, new TrustManager[] { trm }, null); >> trustedURLs.add(url); >> } else { >> - System.out.println("Set the trust manager to check certificates"); >> + Logger.info("Using the system trust manager to verify certificate for host >> '"+host+"'."); >> sc.init(null, null, null); >> } >> HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory()); >> >> >> >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> >> _______________________________________________ >> Jump-pilot-devel mailing list >> Jump-pilot-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> > > > > _______________________________________________ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
