Re: [swift-corelibs-dev] URLSession test fest!

[Peter Tomaselli via swift-corelibs-dev]

Hi all, another new person here. This sounded like a lot of fun so I figured I 
would try and pitch in! However, I’m unable to get off the ground getting the 
current tests running. I’m sure it’s something I’m doing wrong, since I am 
completely new to the project, but I’d love it if anyone had any suggestions to 
get un-stuck.

I poked around in JIRA and there are at least two other tickets that have been 
filed describing the issue I also am having:

https://bugs.swift.org/browse/SR-3703  
[I commented here with the specifics of my system, if you think that is 
important]
https://bugs.swift.org/browse/SR-3775  

Perhaps I am wrong, but it should be possible to use macOS to contribute tests 
for URLSession as described here, right? If so, does anyone have any 
suggestions for tackling the problem described in those two tickets? Thanks!

Cheers,

Peter

> On Jan 27, 2017, at 6:31 PM, Tony Parker via swift-corelibs-dev 
>  wrote:
> 
> Hi Paul,
> 
>> On Jan 27, 2017, at 12:51 AM, Paul Stringer via swift-corelibs-dev 
>> mailto:swift-corelibs-dev@swift.org>> wrote:
>> 
>> Hi Tony,
>> 
>> First post and excited to join the party.
> 
> Great, welcome!
> 

___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] URLSession test fest!

[Peter Tomaselli via swift-corelibs-dev]

Rad, thanks Tony. I can run all the tests now. Hoping to have some time soon to 
dig in and see if I can make a contribution. Love that this thing is spinning 
up its own HTTP server for this — seems real fun.

Cheers,

Peter

> On Jan 30, 2017, at 5:46 PM, Tony Parker  wrote:
> 
> I see what this problem is. I’ll have a PR up shortly to fix it.
> 
> - Tony

___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] URLSession test fest!

[Peter Tomaselli via swift-corelibs-dev]

Following up on this…

I’m wondering if anyone has any info on what happened with the (rather more 
elaborate) “loopback” server and set of tests from @danieleggert’s PR #299:

https://github.com/apple/swift-corelibs-foundation/pull/299 


Is there some reason we ended up with a somewhat cut-down version of those (I 
understand that perhaps the URLSession implementation itself from this PR was 
not used, but perhaps we could pull some of the tests forward, or the more 
flexible loopback implementation) in master? Sorry if this is a silly question 
— I’m not very well-versed in the more low-level networking stuff involved here 
so there may be some obvious reason these are not suitable that I am missing.

Mailing list user Mike Ferenduros also put together what to my eye are some 
pretty comprehensive tests that have not ever seemed to be PR’d or merged, 
described here:

https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20160912/000943.html
 


Seems like there was some concern over the use of httpbin.org 
, but these could be perhaps pulled in and simply 
redirected to hit the loopback, as the rest seems pretty well thought-out.

Just wondering if anyone has any of the background information here. I started 
writing some tests and loopback enhancements of my own but these other two 
gentlemen are way ahead of me, so maybe we should grab what we can from them, 
if it is possible to do so! Happy to take a crack at that if it seems like a 
sane idea.

Cheers,

Peter

> On Jan 31, 2017, at 8:45 AM, Peter Tomaselli  wrote:
> 
> Rad, thanks Tony. I can run all the tests now. Hoping to have some time soon 
> to dig in and see if I can make a contribution. Love that this thing is 
> spinning up its own HTTP server for this — seems real fun.
> 
> Cheers,
> 
> Peter
> 
>> On Jan 30, 2017, at 5:46 PM, Tony Parker  wrote:
>> 
>> I see what this problem is. I’ll have a PR up shortly to fix it.
>> 
>> - Tony
> 

___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-corelibs-dev] URLSession test fest!

[Peter Tomaselli via swift-corelibs-dev]

Hi Pushkar, thanks so much for the information! That does help to clear
everything up.

I'm afraid I don't have a very specific solution to propose. Only that, at
a high level, I think both of these earlier versions will serve as good
inspiration for new tests. For example, there are some test cases in both
of these earlier versions that I would never have thought of myself.

While it's not exactly my area of expertise, the existing loopback server
is very interesting to me, so I hope to have some time soon to improve it a
bit, adding URLSession tests as I go.

Of course, it is slightly tricky when both the system-under-test and the
mechanism by which it is tested are under construction at the same time! :)

Cheers,

Peter

On Fri, Feb 3, 2017 at 1:44 AM, Pushkar N Kulkarni 
wrote:

> Hi Peter,
>
> I'll try to answer your questions to some extent!
>
>
> *I’m wondering if anyone has any info on what happened with the (rather
> more elaborate) “loopback” server and set of tests from @danieleggert’s PR
> #299:*
>
> *https://github.com/apple/swift-corelibs-foundation/pull/299*
> <https://github.com/apple/swift-corelibs-foundation/pull/299>
>
> *Is there some reason we ended up with a somewhat cut-down version of
> those (I understand that perhaps the URLSession implementation itself from
> this PR was not used, but perhaps we could pull some of the tests forward,
> or the more flexible loopback implementation) in master? Sorry if this is a
> silly question — I’m not very well-versed in the more low-level networking
> stuff involved here so there may be some obvious reason these are not
> suitable that I am missing.*
>
> >> Yes, PR 299 from Daniel Eggert had a much mature loopback server and a
> better test coverage. However, it used functionality from the internal
> classes in the NSURLSession package (HTTPBodySource and friends), for which
> we needed to used a @testable import
> <https://github.com/apple/swift-corelibs-foundation/pull/299/files#diff-aec4665d7562834babd21b66fa84ee45R20>in
> TestFoundation. If I remember our discussions right,  to support a
> `@testable import` you'd need to do build Foundation with the
> `-enable-testing` flag, which may not acceptable for Release builds. Of
> course, there was the possibility of rewriting this functionality (from the
> internal classes) all over again. But we decided to start with a simple
> loopback server and then build on it.
>
>
>
>
> *Mailing list user Mike Ferenduros also put together what to my eye are
> some pretty comprehensive tests that have not ever seemed to be PR’d or
> merged, described here:*
>
> *https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20160912/000943.html*
> <https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20160912/000943.html>
>
> *>> *Yes, I am aware of the extensive tests Mike Ferenduros has written.
> In my opinion, it will be great to have them contributed once we have a
> more mature loopback server infrastructure. Mike has also opened a dozen
> URLSession bugs!
>
>
>
> *Seems like there was some concern over the use of httpbin.org
> <http://httpbin.org>, but these could be perhaps pulled in and simply
> redirected to hit the loopback, as the rest seems pretty well thought-out.*
> >> The problem we have had with external URLs is the delays and
> intermittent failures which keep breaking the CI builds, causing much agony
> to many! Can you tell me more about this solution you propose?
>
>
>
> Thanks!
>
> Pushkar N Kulkarni,
> IBM Runtimes
>
> *Simplicity is prerequisite for reliability - Edsger W. Dijkstra*
>
>
>
> -swift-corelibs-dev-boun...@swift.org wrote: -
> To: swift-corelibs-dev 
> From: Peter Tomaselli via swift-corelibs-dev
> Sent by: swift-corelibs-dev-boun...@swift.org
> Date: 02/03/2017 08:11AM
> Subject: Re: [swift-corelibs-dev] URLSession test fest!
>
>
> Following up on this…
>
> I’m wondering if anyone has any info on what happened with the (rather
> more elaborate) “loopback” server and set of tests from @danieleggert’s PR
> #299:
>
> https://github.com/apple/swift-corelibs-foundation/pull/299
>
> Is there some reason we ended up with a somewhat cut-down version of those
> (I understand that perhaps the URLSession implementation itself from this
> PR was not used, but perhaps we could pull some of the tests forward, or
> the more flexible loopback implementation) in master? Sorry if this is a
> silly question — I’m not very well-versed in the more low-level networking
> stuff involved here so there may be some obvious reason these are not
> suitable that I am missing.
>
> Mailing list user Mike Ferenduros also put together what to

Re: [swift-corelibs-dev] URLSession test fest!

[Peter Tomaselli via swift-corelibs-dev]

Just one outside data point on this: as someone who's been wanting to 
contribute to the test fest, I agree that being able to test against 
httpbin.org (or some other pre-existing server — as opposed to the loopback 
server) would make it much easier to make useful contributions.

Not that the loopback server is bad, but I trashed several attempts at a 
contribution due to lack of confidence about my changes. Specifically, with 
both "ends" of the test apparatus in flux (for example, having to add 
"features" to the loopback server just so that features of URLSession could be 
tested), I found it very hard to know if either end was correct. 

So, just two cents from a lurker here, but this sounds like a great idea to me. 
Cheers,

Peter

> On Mar 9, 2017, at 4:38 PM, Tony Parker via swift-corelibs-dev 
>  wrote:
> 
> Hi Pushkar,
> 
> Another option is to continue to use httpbin.org, but make the tests that use 
> that network service not part of the normal unit tests but runnable on demand 
> by us as we develop. That means we don’t have to reimplement the whole thing 
> but we at least have some tools at our disposal to help debug and diagnose 
> issues. As we make fixes we can improve our local unit test server’s 
> infrastructure to prevent regressions.
> 
> - Tony
> 
>> On Mar 9, 2017, at 7:02 AM, Pushkar N Kulkarni  wrote:
>> 
>> Hi Tony, 
>> 
>> PRs 911 and 913 reminded me of this conversation you started. 
>> 
>> The current unit testing infrastructure for URLSession may prove 
>> insufficient for future contributions. I've been looking at these tests 
>> written by Mike Ferenduros and they appear quite exhaustive! The only issue 
>> is that they connect to an external HTTP test service - http://httpbin.org.  
>> 
>> A quick look at httpbin.org tells me it has sufficient functionality to aid 
>> in URLSession unit testing. Do you think it would make sense to emulate this 
>> functionality within TestFoundation?
>> 
>> 
>> Pushkar N Kulkarni,
>> IBM Runtimes
>> 
>> Simplicity is prerequisite for reliability - Edsger W. Dijkstra
>> 
>> 
>> 
>> -swift-corelibs-dev-boun...@swift.org wrote: -
>> To: swift-corelibs-dev 
>> From: Tony Parker via swift-corelibs-dev 
>> Sent by: swift-corelibs-dev-boun...@swift.org
>> Date: 01/25/2017 01:13AM
>> Subject: [swift-corelibs-dev] URLSession test fest!
>> 
>> Hi everyone,
>> 
>> As we’re wrapping up Swift 3.1, I think it’d be a great opportunity to have 
>> a quality focus area on one of our most important APIs: URLSession. If 
>> you’ve been wondering how to make a meaningful contribution, this is the 
>> perfect time.
>> 
>> It’d be great to greatly expand both unit test and higher level test 
>> coverage of URLSession. For testing that can’t really be part of a unit test 
>> and CI like poorly performing networks or esoteric configurations, it would 
>> also be nice to build a different kind of test tool. That tool would provide 
>> a lot of knobs for controlling a URLSession. We could make this tool part of 
>> the swift-corelibs-foundation project.
>> 
>> Is anyone interested in chipping in here? We don’t have to start big. Even 
>> small contributions could get us moving in the right direction.
>> 
>> Thanks,
>> - Tony
>> 
>> 
>> 
>> ___
>> swift-corelibs-dev mailing list
>> swift-corelibs-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
>> 
> 
> ___
> swift-corelibs-dev mailing list
> swift-corelibs-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev