Scott, It seems like two bugs fixed.
One is related to missed fillRootPart() in SOAPContext.writeTo() Second is relates on how content type identified in case envelope is set Attaching a patch against current cvs During mime test execution I had one error related to SOAP registry Generated fault: [Attributes={}] [faultCode=SOAP-ENV:Server] [faultString=java.l ang.IllegalArgumentException: No mapping found for 'java.lang.Object' using enco ding style 'http://schemas.xmlsoap.org/soap/encoding/'.] [faultActorURI=/soap/se rvlet/rpcrouter] [DetailEntries=] [FaultEntries=] This should not be related to Mime Part, and I'll look at this Friday (sorry busy today-tomorrow) Best regards, Pavel > -----Original Message----- > From: Scott Nichol [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, November 20, 2002 8:26 AM > To: [EMAIL PROTECTED] > Subject: Re: Using mime parts - huge drawbacks > > > Pavel, > > I applied these patches, but the mime sample does not work. > Please get > this sample working and submit a new patch. > > Thanks. > > Scott Nichol > > ----- Original Message ----- > From: "Pavel Ausianik" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Friday, November 15, 2002 9:59 AM > Subject: RE: Using mime parts - huge drawbacks > > > > > > > > Hi, > > > > I think I managed make it compatible to existing code and yet still > mush > > faster (8-10 sec of 60 I had in the morning ). The Mime part will be > creates > > as soon as it requested, otherwise plain Envelope used > > Also I set up initial buffer in couple of classes... > > > > Please take a look > > > > Also, if comparing to yesterday's picture timeload, we have > 4 big time > > consumers, which I suppose quite logical > > > > 1. DocumentBulder.parse > > 2. RPCMessage.extractFromEnvelope > > 3. EnvelopeMaprshall > > 4. InputStream.read > > > > Pavel > > > > > -----Original Message----- > > > From: Scott Nichol [mailto:[EMAIL PROTECTED]] > > > Sent: Thursday, November 14, 2002 6:54 PM > > > To: [EMAIL PROTECTED] > > > Subject: Re: Using mime parts - huge drawbacks > > > > > > > > > Thanks for yet another valuable contribution! I've committed this > > > patch. Your others will have to wait a little while > since I have to > > > earn some money today (instead of working on Apache SOAP). > > > > > > I have some questions. > > > > > > 1. What do you use to do this profile? I have very little > experience > > > with profilers, mainly with JInsight, but that was over a > year ago. > > > > > > 2. What is "ComplexRequest"? > > > > > > 3. Do you know what version of JavaMail you are using? > > > > > > Something very interesting that I had not noticed before is that > > > provider.invoke gets on the request and response contexts, so > > > that even > > > "plain" RPCs have their SOAP envelope put into SOAPContext and > > > subsequently extracted. I am thinking that the SOAPContext > > > should gain > > > the ability to hold a SOAP envelope other than simply as the root > part > > > to avoid the expense of extracting it. In fact, > SOAPContext should > be > > > able to keep track of whether there are any attachments > versus just > an > > > evelope to optimize the situation where there is only an envelope. > We > > > would use lazy evaluation to stuff it into the root part > if the root > > > part is requested, but otherwise provide shortcuts to just access > the > > > envelope. > > > > > > Scott Nichol > > > > > > ----- Original Message ----- > > > From: "Pavel Ausianik" <[EMAIL PROTECTED]> > > > To: <[EMAIL PROTECTED]> > > > Sent: Thursday, November 14, 2002 9:04 AM > > > Subject: RE: Using mime parts - huge drawbacks > > > > > > > > > > Scott, > > > > > > > > Here is server time picture taken on the Tomcat server , > processing > > > > ComplexRequest. > > > > The red ellipses show that MimePart initialization takes 10-15% > of > > > CPU > > > > load. > > > > The blue ellipses show that ContentType is also quite expensive > for > > > benefits > > > > it provide. I prepared patch for caching ContentType... > > > > > > > > Pavel > > > > > > > > > -----Original Message----- > > > > > From: Scott Nichol [mailto:[EMAIL PROTECTED]] > > > > > Sent: Wednesday, November 13, 2002 5:48 PM > > > > > To: [EMAIL PROTECTED] > > > > > Subject: Re: Using mime parts - huge drawbacks > > > > > > > > > > > > > > > Pavel, > > > > > > > > > > Yes, this is a good observation. In the case where > there are no > > > > > attachments, the process could be streamlined by serializing > > > directly. > > > > > I am still actively working on this part of the code > > > > > (TransportMessage, > > > > > SOAPContext, Call) and will look at sidestepping some of the > > > activity > > > > > where there are no attachments, just a SOAP envelope, which > > > > > as you point > > > > > out is the typical scenario. > > > > > > > > > > Scott Nichol > > > > > > > > > > ----- Original Message ----- > > > > > From: "Pavel Ausianik" <[EMAIL PROTECTED]> > > > > > To: <[EMAIL PROTECTED]> > > > > > Sent: Wednesday, November 13, 2002 9:04 AM > > > > > Subject: Using mime parts - huge drawbacks > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > thinking more on the current code I have found interesting > > > > > thing. Most > > > > > > requests we have a simple, straight SOAP envelopes, without > any > > > > > attachments. > > > > > > Looking how it is processed I have found following (traced > from > > > > > > httpconnection): > > > > > > > > > > > > In SOAPHTTPConnection.send() we call > TransportMessage.save(). > > > > > > Let's look into it (see my comment how I understand it: > > > > > > > > > > > > String rootContentType = null; > > > > > > > > > > > > // Root Part is Not set for Simple Envelope ! > > > > > > > > > > > > if (ctx.isRootPartSet()) { > > > > > > //... Not in use for simple case > > > > > > } > > > > > > > > > > > > if (rootContentType == null) > > > > > > rootContentType = > > > Constants.HEADERVAL_CONTENT_TYPE_UTF8; > > > > > > if (getEnvelope() != null) { > > > > > > > > > > > > // Now really create root part - how important it is if we > > > > > now how to > > > > > write > > > > > > this Envelope without involving Mime !!! > > > > > > > > > > > > ctx.setRootPart(envelope, rootContentType); > > > > > > } else { > > > > > > //... Not in use for simple case > > > > > > } > > > > > > > > > > > > // Print the whole response to a byte array. > > > > > > // Tracing into this code we'll found that all it will do it > add > > > > > > unnecessary header to envelope > > > > > > // The headers include Content-Type - we know which is, > > > > > > // Content-id - do we need it? Even if yes we can > create any > id > > > > > > // Content-Transfer-Encoding - not for HTTp, anyway we > > > force it to > > > 8 > > > > > bit > > > > > > // Content-Lenght - easy to calculate > > > > > > > > > > > > ByteArrayOutputStream payload = > > > > > > new ByteArrayOutputStream(1024); > > > > > > ctx.writeTo(payload); > > > > > > bytes = payload.toByteArray(); > > > > > > > > > > > > // Now strip off the headers. (Grmbl, get rid > > > of JavaMail > > > > > > // for MIME support). Just intercept the > Content-Type > > > > > > // Remove headers which created right now.... > > > > > > > > > > > > .... > > > > > > > > > > > > // TODO: should not send for HTTP response > > > > > > headers.put("Accept-Encoding", "x-gzip"); > > > > > > if (Boolean.TRUE.equals(ctx.getGzip())) { > > > > > > // Deflate > > > > > > ByteArrayOutputStream baos = > > > > > > new > > > > > ByteArrayOutputStream(bytes.length > > > > > * 2); > > > > > > GZIPOutputStream gzos = new > GZIPOutputStream(baos); > > > > > > gzos.write(bytes, offset, bytes.length > - offset); > > > > > > gzos.close(); > > > > > > baos.close(); > > > > > > bytes = baos.toByteArray(); > > > > > > offset = 0; > > > > > > > > > > > > headers.put("Content-Encoding", "x-gzip"); > > > > > > } > > > > > > > > > > > > Seems like we are doing wonderful job of running a lot > > > unnecessary > > > > > > operations, involving a lot of memory allocations... It > > > > > could be most > > > > > > advanced improvement we ever done! > > > > > > > > > > > > Best regards, > > > > > > Pavel > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > To unsubscribe, e-mail: > > > > <mailto:[EMAIL PROTECTED]> > > > > > For additional commands, e-mail: > > > <mailto:[EMAIL PROTECTED]> > > > > > > > > > > > > > > > > > > > > > > -- > > > > To unsubscribe, e-mail: > > > <mailto:[EMAIL PROTECTED]> > > > > For additional commands, e-mail: > > > <mailto:[EMAIL PROTECTED]> > > > > > > > > > > > > > > > > > -------------------------------------------------------------- > > > ---------- > > > -------- > > > > > > > > > > -- > > > > To unsubscribe, e-mail: > > > <mailto:[EMAIL PROTECTED]> > > > > For additional commands, e-mail: > > > <mailto:[EMAIL PROTECTED]> > > > > > > > > > -- > > > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > > > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > > > > > > > > > -------------------------------------------------------------- > ---------- > -------- > > > > -- > > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > > -- > To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: <mailto:[EMAIL PROTECTED]> >
Index: TransportMessage.java =================================================================== RCS file: /home/cvspublic/xml-soap/java/src/org/apache/soap/transport/TransportMessage.java,v retrieving revision 1.19 diff -u -r1.19 TransportMessage.java --- TransportMessage.java 14 Nov 2002 05:00:42 -0000 1.19 +++ TransportMessage.java 20 Nov 2002 10:52:07 -0000 @@ -169,7 +169,7 @@ if (contentLength != 0) { int offset = 0; int bytesRead = 0; - + // We're done reading when we get all the content OR when the stream // returns a -1. while ((contentLength < 0 || offset < contentLength) && (bytesRead >= 0)) { @@ -258,19 +258,19 @@ // Parse and validate content type. ContentType cType = null; if (contentType != null) { - try { - // Hack since WebSphere puts ;; instead of just ; - int pos = contentType.indexOf( ";;" ); - if ( pos != -1 ) - contentType = contentType.substring(0,pos) + - contentType.substring(pos+1) ; - cType = new ContentType(contentType); - } catch(ParseException pe) { - } - } - if (cType == null) + // Hack since WebSphere puts ;; instead of just ; + int pos = contentType.indexOf( ";;" ); + if ( pos != -1 ) + contentType = contentType.substring(0,pos) + + contentType.substring(pos+1) ; + cType = MimeUtils.getContentType(contentType); + } + if (cType == null) { + if (ctx.getOneWay()) + return null; throw new SOAPException(Constants.FAULT_CODE_PROTOCOL, "Missing content type."); + } // Check encoding String encoding = (String) HTTPUtils.getHeaderValue(headers, @@ -315,26 +315,34 @@ // Parse multipart request. ByteArrayDataSource ds = new ByteArrayDataSource(bytes, contentType); - // Parse multipart mime into parts. ctx.readMultipart(ds); // Find root part. rootPart = ctx.getRootPart(); - rootContentType = new ContentType(rootPart.getContentType()); + rootContentType = MimeUtils.getContentType(rootPart.getContentType()); ByteArrayDataSource bads = new ByteArrayDataSource( rootPart.getInputStream(), null); rootBytes = bads.toByteArray(); } else { rootBytes = bytes; rootContentType = cType; - // Set the single part as the root part of SOAPContext. - ByteArrayDataSource ds = new ByteArrayDataSource(bytes, - contentType); - DataHandler dh = new DataHandler(ds); - rootPart = new MimeBodyPart(); - rootPart.setDataHandler(dh); - ctx.addBodyPart(rootPart); + + // If the root part is text, extract it as a String. + // Note that we could use JAF's help to do this (see save()) + // but implementing it ourselves is safer and faster. + if (Constants.CTYPE_TEXT_ALL.match(rootContentType)) { + String charset = rootContentType.getParameter("charset"); + // Hmm, risky, the default charset is transport-specific... + if (charset == null || charset.equals("")) + charset = Constants.HEADERVAL_DEFAULT_CHARSET; + envelope = new String(rootBytes, MimeUtility.javaCharset(charset)); + // Set the envelope as a String. + ctx.setRootPart(envelope, contentType); + } else + // Set the single part as the root part of SOAPContext. + ctx.setRootPart(bytes, contentType); + return envelope; } // If the root part is text, extract it as a String. @@ -392,20 +400,14 @@ * and try to use it as the envelope. */ String rootContentType = null; - if (ctx.isRootPartSet()) { - MimeBodyPart rootPart = ctx.getRootPart(); - if (rootPart != null) - // Note: don't call MimeBodyPart.getContent() because it will - // default to "text/plain" if the Content-Type header isn't - // set. - rootContentType = rootPart.getHeader( - Constants.HEADER_CONTENT_TYPE, null); - } - if (rootContentType == null) - rootContentType = Constants.HEADERVAL_CONTENT_TYPE_UTF8; if (getEnvelope() != null && !rootPartIsEnvelope) { - ctx.setRootPart(envelope, rootContentType); + rootContentType = Constants.HEADERVAL_CONTENT_TYPE_UTF8; + ctx.setRootPart(envelope, Constants.HEADERVAL_CONTENT_TYPE_UTF8); rootPartIsEnvelope = true; + } else { + rootContentType = ctx.getRootPartContentType(); + if (rootContentType == null) + rootContentType = Constants.HEADERVAL_CONTENT_TYPE_UTF8; } // If we only have one part, it is the envelope, so handle @@ -420,7 +422,7 @@ new ByteArrayOutputStream(65536); ctx.writeTo(payload); bytes = payload.toByteArray(); - + // Now strip off the headers. (Grmbl, get rid of JavaMail // for MIME support). Just intercept the Content-Type // header. We don't want any of the MIME headers, and we know the @@ -509,21 +511,8 @@ */ public String getEnvelope() { if (envelope == null) { - // Assign the root part, if any, to the envelope. - try { - MimeBodyPart rootPart = ctx.getRootPart(); - if (rootPart != null) { - String ctype = rootPart.getContentType(); - ContentType type = new ContentType(ctype); - if (type != null && Constants.CTYPE_TEXT_ALL.match(type)) { - ByteArrayDataSource ds = new ByteArrayDataSource( - rootPart.getInputStream(), ctype); - envelope = ds.getText(); - } - rootPartIsEnvelope = true; - } - } catch (Exception e) { - } + // get an enveope from SOAPContext + envelope = ctx.getEnvelope(); } return envelope; } Index: SOAPContext.java =================================================================== RCS file: /home/cvspublic/xml-soap/java/src/org/apache/soap/rpc/SOAPContext.java,v retrieving revision 1.13 diff -u -r1.13 SOAPContext.java --- SOAPContext.java 6 Nov 2002 18:08:04 -0000 1.13 +++ SOAPContext.java 20 Nov 2002 10:53:50 -0000 @@ -83,13 +83,18 @@ protected MimeMultipart parts; protected Hashtable bag = new Hashtable(); protected ClassLoader loader = null ; - protected Vector multiRef = new Vector(); - protected Hashtable deserializedMultiRef = new Hashtable(); + protected List multiRef = null; + protected Map deserializedMultiRef = null; protected String currentId = null; protected boolean docLitSerialization = false; protected Boolean gzip = null; protected Boolean acceptGzip = null; + protected boolean isRootPartEnvelope = false; + protected String rootPartString = null; + protected String rootPartContentType = ""; + protected boolean oneWay = false; + // Constants for checking type for base64 encoding private static MimeType MIME_STREAM; private static MimeType MIME_IMAGE; @@ -159,6 +164,7 @@ * @return the Part, or null if no such part exists. */ public MimeBodyPart getBodyPart(int index) { + fillRootPart(); /* Actually, this method never throws a MessagingException. In case a * future implementation does, catch it and throw an * IndexOutOfBoundsException @@ -188,6 +194,7 @@ * @return the Part, or null if no such part exists. */ public MimeBodyPart getBodyPart(String CID) { + fillRootPart(); if (parts == null) { return null; } @@ -221,6 +228,7 @@ * @return the Part or null if not found */ public MimeBodyPart findBodyPart(String uri) { + fillRootPart(); if (parts == null || uri == null) { return null; } @@ -419,10 +427,12 @@ * @exception MessagingException */ public void setRootPart(MimeBodyPart part) throws MessagingException { + if (rootPartSet && !isRootPartEnvelope) + parts.removeBodyPart(getRootPart()); + isRootPartEnvelope = false; + String rootCid = '<' + MimeUtils.getUniqueValue() + '>'; part.setHeader(Constants.HEADER_CONTENT_ID, rootCid); - if (rootPartSet) - parts.removeBodyPart(getRootPart()); addBodyPart(part, 0); rootPartSet = true; } @@ -437,7 +447,10 @@ */ public void setRootPart(String s, String contentType) throws MessagingException, IOException { - setRootPart(s.getBytes("UTF8"), contentType); + isRootPartEnvelope = true; + rootPartContentType = contentType; + rootPartString = s; + rootPartSet = true; } /** @@ -457,7 +470,6 @@ bp.setDataHandler(dh); bp.setHeader(Constants.HEADER_CONTENT_LENGTH, String.valueOf(ds.getSize())); - // Avoid letting JavaMail determine a transfer-encoding of // quoted-printable or base64... Force 8-bit encoding. bp.setHeader("Content-Transfer-Encoding", "8bit"); @@ -466,6 +478,60 @@ } /** + * In case MimePart is requested converts Envelope to MimePart + */ + private void fillRootPart() { + if (isRootPartEnvelope) { + try { + byte[] rootPartBytes = rootPartString.getBytes( + MimeUtils.getEncoding(rootPartContentType, "UTF8")); + + ByteArrayDataSource ds = + new ByteArrayDataSource(rootPartBytes, rootPartContentType); + DataHandler dh = new DataHandler(ds); + MimeBodyPart bp = new MimeBodyPart(); + bp.setDataHandler(dh); + bp.setHeader(Constants.HEADER_CONTENT_LENGTH, + String.valueOf(ds.getSize())); + + // Avoid letting JavaMail determine a transfer-encoding of + // quoted-printable or base64... Force 8-bit encoding. + bp.setHeader("Content-Transfer-Encoding", "8bit"); + bp.setHeader(Constants.HEADERVAL_CONTENT_TYPE, rootPartContentType); + setRootPart(bp); + } + catch (MessagingException e) {} // Should never happen + catch (UnsupportedEncodingException e) {} // Should never happen + } + } + + /** + * Return Envelope String set up as setRootPart(String) + * or converted from MimeBodyPart + * + * @return Envelope String + */ + public String getEnvelope() { + if (isRootPartEnvelope) { + return rootPartString; + } + // Envelope was set up as MimeBodyPart - let's get a text from it + try { + MimeBodyPart rootPart = getRootPart(); + if (rootPart != null) { + String ctype = rootPart.getContentType(); + ContentType type = MimeUtils.getContentType(ctype); + if (type != null && Constants.CTYPE_TEXT_ALL.match(type)) { + ByteArrayDataSource ds = new ByteArrayDataSource( + rootPart.getInputStream(), ctype); + return ds.getText(); + } + } + } catch (Exception e) {} + return null; + } + + /** * Find the root part. For multipart, search for a "start" Content-Type * header modifier, if not present or invalid, assume the first part is * the root. @@ -474,6 +540,7 @@ * @exception MessagingException */ public MimeBodyPart getRootPart() throws MessagingException { + fillRootPart(); MimeBodyPart rootPart = null; if (getCount() > 1) { String startCid = new ContentType( @@ -487,6 +554,22 @@ } /** + * Returns root part ContentType. + * + * @return root BodyPart ContentType + * @exception MessagingException + */ + public String getRootPartContentType() throws MessagingException { + if (isRootPartEnvelope) + return rootPartContentType; + else { + MimeBodyPart rootPart = getRootPart(); + return rootPart == null ? null : rootPart.getContentType(); + } + } + + + /** * Set the MultiPart Mime subtype. This method should be invoked only on * a new MimeMultipart object created by the client. The default subtype * of such a multipart object is "related".<p> @@ -514,10 +597,11 @@ * @return number of parts */ public int getCount() throws MessagingException { + int countRoot = isRootPartEnvelope ? 1 : 0; if (parts == null) - return 0; + return countRoot; else - return parts.getCount(); + return parts.getCount() + countRoot; } /** @@ -529,7 +613,7 @@ */ public String getContentType() throws MessagingException { if (parts == null) - return null; + return isRootPartEnvelope ? rootPartContentType : null; else if (parts.getCount() == 1) return getRootPart().getContentType(); @@ -553,6 +637,8 @@ getRootPart().writeTo(os); } else { + // fill Root part first + fillRootPart(); Session session = Session.getDefaultInstance(new Properties(), null); MimeMessage msg = new MimeMessage(session); msg.setContent(parts); @@ -690,14 +776,16 @@ * @return The id of the element. */ public int addMultiRef(Object obj, Serializer ser) { + if (multiRef == null) + multiRef = new ArrayList(); // While a Hashtable might seem a better choice than a vector here, // a Vector is easier to work with during serialization, since // multiRefs may be added during the serialization of other multiRefs. for (int i = 0; i < multiRef.size(); i++) { - if (((MultiRefInfo) multiRef.elementAt(i)).obj == obj) + if (((MultiRefInfo) multiRef.get(i)).obj == obj) return i; } - multiRef.addElement(new MultiRefInfo(obj, ser)); + multiRef.add(new MultiRefInfo(obj, ser)); return multiRef.size() - 1; } @@ -708,7 +796,7 @@ * @return The multiRef object for the id. */ public Object getMultiRefObject(int id) { - return ((MultiRefInfo) multiRef.elementAt(id)).obj; + return multiRef != null ? ((MultiRefInfo) multiRef.get(id)).obj : null; } /** @@ -718,7 +806,7 @@ * @return The multiRef serializer for the id. */ public Serializer getMultiRefSerializer(int id) { - return ((MultiRefInfo) multiRef.elementAt(id)).ser; + return multiRef != null ? ((MultiRefInfo) multiRef.get(id)).ser : null; } /** @@ -727,7 +815,7 @@ * @return The number of multiRefs. */ public int getMultiRefCount() { - return multiRef.size(); + return multiRef != null ? multiRef.size() : 0; } /** @@ -737,6 +825,8 @@ * @param o The deserialized object. */ public void addDeserializedMultiRef(String id, Object o) { + if (deserializedMultiRef == null) + deserializedMultiRef = new HashMap(); deserializedMultiRef.put(id, o); } @@ -747,7 +837,21 @@ * @return The multiRef for the id. */ public Object getDeserializedMultiRef(String id) { - return deserializedMultiRef.get(id); + return deserializedMultiRef == null ? null :deserializedMultiRef.get(id); + } + + /** + * Gets whether this is a one-way response. + */ + public boolean getOneWay() { + return oneWay; + } + + /** + * Sets whether this is a one-way response. + */ + public void setOneWay(boolean oneWay) { + this.oneWay = oneWay; } /** @@ -779,7 +883,9 @@ pw.print("}]"); - pw.print(" multiRefs: " + multiRef.size() + " deserializedMultiRefs: " + deserializedMultiRef.size()); + pw.print(" multiRefs: " + (multiRef != null ? multiRef.size() : '0') + + " deserializedMultiRefs: " + + (deserializedMultiRef != null ? deserializedMultiRef.size() : '0')); return sw.toString(); }
-- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>