RE: Interaction between redefine+include breaks augmenting a base schema's element with new attributes

[Sam Harwell]

Yes, we’re talking specifically about the XPath 2.0 test suite. I’m currently 
in the process of identifying specific bugs revealed by the tests, filing 
individual reports for those bugs, and breaking up all my changes into specific 
corrections to them.

Note that all of the bugs have been filed yet, and pull requests are not 
available for some of the bugs that have been filed. The biggest upcoming 
change from an implementation standpoint is dramatic improvements to the way 
“seconds” are stored (xs:decimal instead of xs:double) along with major 
improvements to timezone handling and very old dates (including before 1AD). 
The biggest upcoming change from a “size of diff” standpoint is a correction to 
the eq operator for QName; it needs access to the static context to correctly 
handle namespaces which ends up changing quite a few evaluation methods (an 
argument changes from DynamicContext to EvaluationContext). I believe Rackspace 
is currently relying on Saxon due to bugs within the date/time/timezone 
handling functionality, so this will be an important change for them.

Thank you,
Sam Harwell

From: Jorge Williams [mailto:jorge.willi...@rackspace.com]
Sent: Friday, June 24, 2016 9:16 AM
To: j-users@xerces.apache.org; muk...@apache.org
Cc: Sam Harwell 
Subject: Re: Interaction between redefine+include breaks augmenting a base 
schema's element with new attributes


Hey Mukul,



Sorry I should have been more specific.  I'm sure that Xerces does indeed pass 
the XML Schema 1.1 test suite.  That said, we have run into a number of issues 
that are directly related with errors in XPath  -- which XML Schema 1.1 uses 
for assertions.  So when I say test suite, I'm talking about PsycoPath passing 
the W3C XPath / XQuery test suite. If you look at the github project 
(https://github.com/sharwell/webtools.sourceediting.xpath), you can see that  
Sam has integrated the test suite into the build process ​and you can see 
specifically which tests are failing and which tests have been corrected in the 
Rackspace branch.



-jOrGe W.






From: Mukul Gandhi 
Sent: Friday, June 24, 2016 3:58 AM
To: j-users@xerces.apache.org
Cc: s...@tunnelvisionlabs.com
Subject: Re: Interaction between redefine+include breaks augmenting a base 
schema's element with new attributes

Hi Jorge,
   We (that includes I think me, Jesper & Dave Carver) had done various fixes 
earlier to the originally contributed PsycoPath library that Xerces uses for 
its XML Schema 1.1 implementation. I can say with much responsibility (being a 
previous committer to Eclipse WTP source editing, and also as a PMC member of 
Xerces), that the current jar file for PsycoPath, on Xerces SVN conforms fully 
to the W3C XML Schema 1.1 test suite.

You say, '.. XPath 2 implementation reports over 800 failures in the W3C test 
suite'. I'm curious what W3C XML Schema 1.1 tests, you and Sam found that 
PsycoPath library used by Xerces does not pass? Our XML Schema 1.1 conformance 
reports are located at: 
http://wiki.apache.org/xerces/xmlschema11TestSuiteReports. I believe, this is 
also indirectly linked from the XML Schema 1.1 specification.

On 24 June 2016 at 02:49, Jorge Williams 
mailto:jorge.willi...@rackspace.com>> wrote:
Hey Guys,


If we are planning a release of Xerces, may I suggest that we also update 
version of PsycoPath that Xerces is currently using. The current version of the 
XPath 2 implementation reports over 800 failures in the W3C test suite.  Sam 
Harwell has been able to reduce this number to about 34!

I’m in the process of reviewing these fixes — which you can find here:

 
https://github.com/sharwell/webtools.sourceediting.xpath/pulls

Any help with these code review these PRs would be great and it would be 
awesome to include a new build of PsycoPath with the next Xerces release.

Thanks,

-jOrGe W.



> On Jun 17, 2016, at 8:54 AM, Michael Glavassevich 
> mailto:mrgla...@ca.ibm.com>> wrote:
>
> Thanks David.
>
> Michael Glavassevich
> XML Technologies and WAS Development
> IBM Toronto Lab
> E-mail: mrgla...@ca.ibm.com
> E-mail: mrgla...@apache.org
>
> David Costanzo mailto:david_costa...@yahoo.com>> 
> wrote on 06/17/2016 09:37:56 AM:
>
>> From: David Costanzo
 I started with the JIRA release notes [1] and it doesn't look like I
 have permissions to help with this. [snip]
>>
>> From: Michael Glavassevich
>>> I just add you to the list of contributors in JIRA. Hopefully that
> also
>>> granted you permission to update that field.
>>
>> Yes, that works.  I should be able to build up the release notes,
>> now.  I'll start a new thread in the "xerces-devs" list for any
>> conversation specific to the 2.12.0 release, instead of rambling
>> off-topic on this thread.
>>
>>
>>
>> From: David Costanzo
>>
 As mentioned previously, the fix that my organization wants > >
>> (XERCESJ

Re: Interaction between redefine+include breaks augmenting a base schema's element with new attributes

[Mukul Gandhi]

Hi Jorge,
   I've just re-analyzed the situation you've described in right earnest,
and following are my findings:

Xerces-J is supposed to use following two branches from Eclipse WTP project
located at dev.eclipse.org

R3_2_maintenance (this is where the product code of XPath 2.0
processor, is stored which Xerces-J is using. We use this, because this
branch as agreed works with JDK 1.4)
R3_2_xpath2_14unittests(this is where the unit tests are located, which
tests the product code of R3_2_maintenance branch. To run these unit tests,
we would need JDK 1.5)

These two code branches were created only to facilitate Xerces-J XML Schema
1.1 development. I think these decisions date back to few years ago.

I have all the latest code from these two branches, on my development
workstation. Following are the results of unit tests run, that I get as of
today:
There are total 8276 tests on the R3_2_xpath2_14unittests branch. Except
for 3 tests in the file CatalogTest.class, that give
java.lang.OutOfMemoryError to me, 8273 tests pass. I think these results
are great, and there should not be any doubt about the quality of the XPath
2.0 processor, that Xerces-J is currently using on the SVN.

The other points that I would like to reply are:
You've mentioned a github project, that's forking the XPath 2.0 code base
from Eclipse WTP dev.eclipse.org site and making improvements on it. I
personally have no issues with this and if you wish to use such an improved
PsychoPath XPath 2.0 processor at rackspace.
I think, it is convenient for Xerces-J for the next XML Schema 1.1 release,
to use the PsychoPath jar which is kept at present on the SVN. But I also
think, Xerces-J must have a mechanism to have an improved XPath 2.0
processor anytime in future, than what we currently have on the SVN. I
think, the right way for this will be one of the following options:
1) Any improvements must be mentioned as Eclipse WTP bugs on Eclipse
forums, and ask for fixes there. We can get new XPath 2.0 jars as part of
Eclipse WTP releases.
2) We host a copy of PsychoPath XPath 2.0 code base from R3_2_maintenance
and R3_2_xpath2_14unittests branches, as branches on Xerces-J source code
site. I think I'll be able to do this, by committing from my development
workstation. But for that, I'll need permission from Xerces-J dev members
and ideally from PMC also. Someone else may also, fetch latest code base
from Eclipse WTP R3_2_maintenance and R3_2_xpath2_14unittests branches, and
commit to Xerces-J SVN.
3) We take all new improvements for PsychoPath XPath 2.0 code base, from
the github PsychoPath site that you've created for rackspace.

But the first two options that I've mentioned, look more natural to me when
I see how Xerces-J and Eclipse WTP work together.

I'm copying this mail to Michael if he would like to comment anything on
this.

On 24 June 2016 at 19:46, Jorge Williams 
wrote:

> Hey Mukul,
>
>
> Sorry I should have been more specific.  I'm sure that Xerces does indeed
> pass the XML Schema 1.1 test suite.  That said, we have run into a number
> of issues that are directly related with errors in XPath  -- which XML
> Schema 1.1 uses for assertions.  So when I say test suite, I'm talking
> about PsycoPath passing the W3C XPath / XQuery test suite. If you look at
> the github project (
> https://github.com/sharwell/webtools.sourceediting.xpath), you can see
> that  Sam has integrated the test suite into the build process ​and you
> can see specifically which tests are failing and which tests have been
> corrected in the Rackspace branch.
>
>
> -jOrGe W.
>
>
>
> --
> *From:* Mukul Gandhi 
> *Sent:* Friday, June 24, 2016 3:58 AM
> *To:* j-users@xerces.apache.org
> *Cc:* s...@tunnelvisionlabs.com
> *Subject:* Re: Interaction between redefine+include breaks augmenting a
> base schema's element with new attributes
>
> Hi Jorge,
>We (that includes I think me, Jesper & Dave Carver) had done various
> fixes earlier to the originally contributed PsycoPath library that Xerces
> uses for its XML Schema 1.1 implementation. I can say with much
> responsibility (being a previous committer to Eclipse WTP source editing,
> and also as a PMC member of Xerces), that the current jar file for
> PsycoPath, on Xerces SVN conforms fully to the W3C XML Schema 1.1 test
> suite.
>
> You say, '.. XPath 2 implementation reports over 800 failures in the W3C
> test suite'. I'm curious what W3C XML Schema 1.1 tests, you and Sam found
> that PsycoPath library used by Xerces does not pass? Our XML Schema 1.1
> conformance reports are located at:
> http://wiki.apache.org/xerces/xmlschema11TestSuiteReports. I believe,
> this is also indirectly linked from the XML Schema 1.1 specification.
>
>
> On 24 June 2016 at 02:49, Jorge Williams 
> wrote:
>
>> Hey Guys,
>>
>>
>> If we are planning a release of Xerces, may I suggest that we also update
>> version of PsycoPath that Xerces is currently using. The current version of
>> the XPath 2 i