Re: Map Keywords are functions, why not vector elements?

2017-11-14 Thread Gary Verhaegen
If you're going to call that for multiple elements of the same vector, it's
worth thinking about doing some work upfront so that each look-up is faster:

(defn indices [vect]
  (->> vect
   (map-indexed vector)
   (reduce (fn [acc [idx el]]
 (update acc el (fnil conj []) idx))
   {})))
(indices [1 "is" 1 :same]){1 [0 2], "is" [1], :same [3]}


would walk through the vector only once and create a map of value -> list
of indices.

On 14 November 2017 at 05:49, Sean Corfield  wrote:

> I don’t think anyone addressed your question about finding all the indices
> in a vector where the element matches a given search value?
>
>
>
> There are a number of ways to tackle that (given it’s going to be a linear
> search). Since you want the indices, you are either going to need to track
> them directly or use map-indexed to produce them automatically.
>
>
>
> (defn indices [x l] ; produces a vector
>
>   (loop [i 0 l l r []]
>
> (if (seq l)
>
>   (recur (inc i) (rest l) (if (= x (first l)) (conj r i) r))
>
>   r)))
>
>
>
> (defn indices [x l] ; produces a lazy sequence
>
>   (keep identity (map-indexed (fn [i v] (when (= x v) i)) l)))
>
>
>
> (defn indices [x l] ; produces a vector
>
>   (transduce (comp (map-indexed vector)
>
>(filter (comp (partial = x) second))
>
>(map first))
>
>  conj
>
>  []
>
>  l))
>
>
>
> Hopefully that’ll give you some options to think about…
>
>
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>
> --
> *From:* clojure@googlegroups.com  on behalf of
> Stephen Feyrer 
> *Sent:* Monday, November 13, 2017 5:19:32 PM
> *To:* clojure@googlegroups.com
> *Subject:* Re: Map Keywords are functions, why not vector elements?
>
> Hi Alex, Didier,
>
> Thanks for your patience.
>
> That covers everything which I can think of and a fair bit more :)  I have
> a bit of reading and thinking to do now.
>
> Again, thank you.
>
>
> --
> Rule of thumb simple question, complicated answer
>
> Stephen.
>
> On 13 November 2017 at 22:09, Didier  wrote:
>
>> Yo are looking for indexOf (.indexOf vector value).
>>
>> (.indexOf ["a" "b"] "b")
>> => 1
>>
>> (.indexOf ["a" "b" "b"] "b")
>> => 1
>>
>> Note how indexOf searches for the index of the first value which matches
>> value.
>>
>> To do what you ask, is a query over a vector, which requires a search on
>> each element. This will take O(N) time. For a small list like in your
>> example its probably good enough and not an issue.
>>
>> If you want the name of the horse in a given position, that's a key
>> lookup, which is ~O(1) time. You can use get:
>>
>> (get ["a" "b"] 1)
>> => "b"
>>
>> If you really needed performance, you would need to combine a LinkedList
>> and a map. Some datastructures do it for you under the hood, like Apache
>> LinkedMap, amalloy ordered, java LinkedHashMap, etc.
>>
>> Its possible to also just use sorted-map-by in a closure. But this only
>> works if you're not going to add/update things to the datastructure after
>> first creation.
>>
>> See the example on clojuredocs: https://clojuredocs.org/clojur
>> e.core/sorted-map-by#example-542692d5c026201cdc327094
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Not

Re: [ANN] data.xml 0.2.0-alpha5

2017-11-14 Thread Herwig Hochleitner
2017-11-14 4:03 GMT+01:00 Alan Thompson :

> I seem to still be getting the JDK 1.9 reflection warning:
>
>
> ~/tupelo > grep data.xml project.clj
> [org.clojure/data.xml "0.2.0-alpha5"]
>
> > lein test
> lein test tst.tupelo._bootstrap
>
> -
>Clojure 1.9.0-RC1Java 9.0.1
> -
>
> lein test tst.tupelo.array
>
> lein test tst.tupelo.async
>
> lein test tst.tupelo.dev
>
> lein test tst.tupelo.forest
>
> lein test tst.tupelo.forest-examples
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by clojure.lang.Reflector
> (file:/home/alan/.m2/repository/org/clojure/clojure/1.9.0-RC1/clojure-1.9.0-RC1.jar)
> to method com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.
> parse(org.xml.sax.InputSource,org.xml.sax.helpers.DefaultHandler)
> WARNING: Please consider reporting this to the maintainers of
> clojure.lang.Reflector
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> 
>
>
Hm, that's not showing up in the test suite:
https://build.clojure.org/job/data.xml-test-matrix/344/CLOJURE_VERSION=1.9.0-RC1,jdk=Open%20JDK%209%20with%20Eclipse%20Open%20J9/console

Might this warning only be occurring on oracle jdk?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [ANN] data.xml 0.2.0-alpha5

2017-11-14 Thread Alex Miller
I can replicate that in the tupelo tests, but it's coming from enlive, not 
data.xml. I think enlive needs a type hint on s in this function:

https://github.com/cgrand/enlive/blob/master/src/net/cgrand/xml.clj#L85


  actual: java.lang.IllegalAccessException: class clojure.lang.Reflector 
cannot access class com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl 
(in module java.xml) because module java.xml does not export 
com.sun.org.apache.xerces.internal.jaxp to unnamed module @7b0eff2
 at jdk.internal.reflect.Reflection.throwIllegalAccessException 
(Reflection.java:423)
jdk.internal.reflect.Reflection.throwIllegalAccessException 
(Reflection.java:414)
jdk.internal.reflect.Reflection.ensureMemberAccess (Reflection.java:112)
java.lang.reflect.AccessibleObject.slowCheckMemberAccess 
(AccessibleObject.java:632)
java.lang.reflect.AccessibleObject.checkAccess 
(AccessibleObject.java:624)
java.lang.reflect.Method.invoke (Method.java:539)
clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
*net.cgrand.xml$startparse_sax.invokeStatic (xml.clj:85)*
*net.cgrand.xml$startparse_sax.invoke (xml.clj:76)*
*net.cgrand.xml$parse.invokeStatic (xml.clj:99)*
net.cgrand.xml$parse.invoke (xml.clj:87)
net.cgrand.xml$parse.invokeStatic (xml.clj:94)
net.cgrand.xml$parse.invoke (xml.clj:87)
net.cgrand.enlive_html$xml_parser.invokeStatic (enlive_html.clj:55)
net.cgrand.enlive_html$xml_parser.invoke (enlive_html.clj:51)
net.cgrand.enlive_html$eval34196$fn__34197.invoke (enlive_html.clj:103)



On Monday, November 13, 2017 at 9:04:01 PM UTC-6, Alan Thompson wrote:
>
> I seem to still be getting the JDK 1.9 reflection warning:
>
>
> ~/tupelo > grep data.xml project.clj 
> [org.clojure/data.xml "0.2.0-alpha5"]
>
> > lein test
> lein test tst.tupelo._bootstrap
>
> -
>Clojure 1.9.0-RC1Java 9.0.1
> -
>
> lein test tst.tupelo.array
>
> lein test tst.tupelo.async
>
> lein test tst.tupelo.dev
>
> lein test tst.tupelo.forest
>
> lein test tst.tupelo.forest-examples
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by clojure.lang.Reflector 
> (file:/home/alan/.m2/repository/org/clojure/clojure/1.9.0-RC1/clojure-1.9.0-RC1.jar)
>  
> to method 
> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(org.xml.sax.InputSource,org.xml.sax.helpers.DefaultHandler)
> WARNING: Please consider reporting this to the maintainers of 
> clojure.lang.Reflector
> WARNING: Use --illegal-access=warn to enable warnings of further illegal 
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> 
>
>
>
>
> On Mon, Nov 13, 2017 at 3:47 PM, Herwig Hochleitner <
> hhochleit...@gmail.com> wrote:
>
>> data.xml is a Clojure contrib library that parses and emits XML.
>>
>> Github: https://github.com/clojure/data.xml
>> Changelog: https://github.com/clojure/data.xml/blob/master/CHANGES.md
>>
>> Information on updating the dependency is here 
>> .
>>
>> 0.2.0-alpha5 fixes a bug with emitting attributes in the builtin `xml:` 
>> namespace. In the process, there was a lot of clean-up work, removing 
>> various partial implementations of the recently introduced pu-map.
>>
>> Michael Blume got rid of reflection warnings. This should also remove the 
>> "illegal reflection" warning (CLJ-2066) on jdk >= 1.9
>>
>> Thanks to @MichaelBlume and all data.xml contributors!
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
t

Re: [ANN] data.xml 0.2.0-alpha5

2017-11-14 Thread Alex Miller
Although it looks like that call is intentionally reflective to cover 
several possible types. Probably need to solve that with a cond to check 
for and hint the actual type at the point of the call.

On Tuesday, November 14, 2017 at 8:31:48 AM UTC-6, Alex Miller wrote:
>
> I can replicate that in the tupelo tests, but it's coming from enlive, not 
> data.xml. I think enlive needs a type hint on s in this function:
>
> https://github.com/cgrand/enlive/blob/master/src/net/cgrand/xml.clj#L85
>
>
>   actual: java.lang.IllegalAccessException: class clojure.lang.Reflector 
> cannot access class com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl 
> (in module java.xml) because module java.xml does not export 
> com.sun.org.apache.xerces.internal.jaxp to unnamed module @7b0eff2
>  at jdk.internal.reflect.Reflection.throwIllegalAccessException 
> (Reflection.java:423)
> jdk.internal.reflect.Reflection.throwIllegalAccessException 
> (Reflection.java:414)
> jdk.internal.reflect.Reflection.ensureMemberAccess 
> (Reflection.java:112)
> java.lang.reflect.AccessibleObject.slowCheckMemberAccess 
> (AccessibleObject.java:632)
> java.lang.reflect.AccessibleObject.checkAccess 
> (AccessibleObject.java:624)
> java.lang.reflect.Method.invoke (Method.java:539)
> clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
> clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
> *net.cgrand.xml$startparse_sax.invokeStatic (xml.clj:85)*
> *net.cgrand.xml$startparse_sax.invoke (xml.clj:76)*
> *net.cgrand.xml$parse.invokeStatic (xml.clj:99)*
> net.cgrand.xml$parse.invoke (xml.clj:87)
> net.cgrand.xml$parse.invokeStatic (xml.clj:94)
> net.cgrand.xml$parse.invoke (xml.clj:87)
> net.cgrand.enlive_html$xml_parser.invokeStatic (enlive_html.clj:55)
> net.cgrand.enlive_html$xml_parser.invoke (enlive_html.clj:51)
> net.cgrand.enlive_html$eval34196$fn__34197.invoke (enlive_html.clj:103)
>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [ANN] data.xml 0.2.0-alpha5

2017-11-14 Thread Herwig Hochleitner
2017-11-14 15:33 GMT+01:00 Alex Miller :

> Although it looks like that call is intentionally reflective to cover
> several possible types. Probably need to solve that with a cond to check
> for and hint the actual type at the point of the call.
>

Yep, see the corresponding change in data.xml:
https://github.com/clojure/data.xml/commit/ef8d0c816fa8d9c02d759de990bdb2e0042c0d2a#diff-2a9e7af9410568bc9667e92c2a4227c4R126

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Eric Normand
Hey everybody!

I'm chiming in after seeing this linked to in The Repl 
(https://therepl.net/).

On Alex's suggestion, I rewatched Spec-ulation last night. The parts about 
negation and evolution are towards the end. I was struck (once again) by 
how clearly he picked apart changes. Relaxing a requirement is growth. And 
adding requirements is breakage. But it left me with a question:

Isn't disallowing a key and then allowing it (as optional) growth (instead 
of breakage)? All of the old clients are still fine, and new clients can 
use the key if they choose. You're relaxing the requirements. Taking the 
opposite approach, I require some keys plus allow anything else. Some 
clients will inevitably send me something with extra keys, which is okay, 
they pass my specs. Later, I add in an optional key with a defined spec. So 
I'm now restricting what used to be completely open. Isn't that breakage? I 
feel like I'm seeing it exactly opposite as Rich Hickey. He says if you 
disallow things, it's forever, because if you need to allow it later, 
that's breakage. But there's not enough explanation for me to understand. 
It seems like relaxing requirements. I feel like I'm missing something. In 
short: why is it forever?

He does mention is that logic engines don't have negation. Does this hint 
that we will want to be using logic engines to reason over our specs?

Thanks
Eric

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Seth Verrinder
I took part of the goal to be that specs themselves would remain 
compatible, so an old set of specs wouldn't start failing on data that 
conforms to a new but compatible set of specs. That sort of compatibility 
isn't possible when you go from disallowing something to allowing it.

On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:
>
> Hey everybody!
>
> I'm chiming in after seeing this linked to in The Repl (
> https://therepl.net/).
>
> On Alex's suggestion, I rewatched Spec-ulation last night. The parts about 
> negation and evolution are towards the end. I was struck (once again) by 
> how clearly he picked apart changes. Relaxing a requirement is growth. And 
> adding requirements is breakage. But it left me with a question:
>
> Isn't disallowing a key and then allowing it (as optional) growth (instead 
> of breakage)? All of the old clients are still fine, and new clients can 
> use the key if they choose. You're relaxing the requirements. Taking the 
> opposite approach, I require some keys plus allow anything else. Some 
> clients will inevitably send me something with extra keys, which is okay, 
> they pass my specs. Later, I add in an optional key with a defined spec. So 
> I'm now restricting what used to be completely open. Isn't that breakage? I 
> feel like I'm seeing it exactly opposite as Rich Hickey. He says if you 
> disallow things, it's forever, because if you need to allow it later, 
> that's breakage. But there's not enough explanation for me to understand. 
> It seems like relaxing requirements. I feel like I'm missing something. In 
> short: why is it forever?
>
> He does mention is that logic engines don't have negation. Does this hint 
> that we will want to be using logic engines to reason over our specs?
>
> Thanks
> Eric
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Eric Normand
Oh, I see! That makes sense. Thanks!

On Tuesday, November 14, 2017 at 10:45:30 AM UTC-6, Seth Verrinder wrote:
>
> I took part of the goal to be that specs themselves would remain 
> compatible, so an old set of specs wouldn't start failing on data that 
> conforms to a new but compatible set of specs. That sort of compatibility 
> isn't possible when you go from disallowing something to allowing it.
>
> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:
>>
>> Hey everybody!
>>
>> I'm chiming in after seeing this linked to in The Repl (
>> https://therepl.net/).
>>
>> On Alex's suggestion, I rewatched Spec-ulation last night. The parts 
>> about negation and evolution are towards the end. I was struck (once again) 
>> by how clearly he picked apart changes. Relaxing a requirement is growth. 
>> And adding requirements is breakage. But it left me with a question:
>>
>> Isn't disallowing a key and then allowing it (as optional) growth 
>> (instead of breakage)? All of the old clients are still fine, and new 
>> clients can use the key if they choose. You're relaxing the requirements. 
>> Taking the opposite approach, I require some keys plus allow anything else. 
>> Some clients will inevitably send me something with extra keys, which is 
>> okay, they pass my specs. Later, I add in an optional key with a defined 
>> spec. So I'm now restricting what used to be completely open. Isn't that 
>> breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He 
>> says if you disallow things, it's forever, because if you need to allow it 
>> later, that's breakage. But there's not enough explanation for me to 
>> understand. It seems like relaxing requirements. I feel like I'm missing 
>> something. In short: why is it forever?
>>
>> He does mention is that logic engines don't have negation. Does this hint 
>> that we will want to be using logic engines to reason over our specs?
>>
>> Thanks
>> Eric
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


RE: [core.spec] Stricter map validations?

2017-11-14 Thread Sean Corfield
Eric does raise an interesting question tho’:

If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later specify 
that ‘d’ is optional and should be an ‘int?’, does that qualify as breakage or 
growth? If clients were sending ‘d’ as a string before but you ignored it, it 
will break those clients. Clients that were not sending ‘d’ will not be 
affected by the change. The old spec – allowing ‘d’ to be ‘any?’ essentially – 
won’t fail on any data that omits ‘d’ or passes it as ‘int?’ so it passes your 
compatibility test.

(we actually ran into this at work because a client app was passing a field we 
didn’t care about and we later decided that was an optional field but couldn’t 
be an empty string and it broke that client)

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


From: clojure@googlegroups.com  on behalf of Seth 
Verrinder 
Sent: Tuesday, November 14, 2017 8:45:30 AM
To: Clojure
Subject: Re: [core.spec] Stricter map validations?

I took part of the goal to be that specs themselves would remain compatible, so 
an old set of specs wouldn't start failing on data that conforms to a new but 
compatible set of specs. That sort of compatibility isn't possible when you go 
from disallowing something to allowing it.

On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:
Hey everybody!

I'm chiming in after seeing this linked to in The Repl (https://therepl.net/).

On Alex's suggestion, I rewatched Spec-ulation last night. The parts about 
negation and evolution are towards the end. I was struck (once again) by how 
clearly he picked apart changes. Relaxing a requirement is growth. And adding 
requirements is breakage. But it left me with a question:

Isn't disallowing a key and then allowing it (as optional) growth (instead of 
breakage)? All of the old clients are still fine, and new clients can use the 
key if they choose. You're relaxing the requirements. Taking the opposite 
approach, I require some keys plus allow anything else. Some clients will 
inevitably send me something with extra keys, which is okay, they pass my 
specs. Later, I add in an optional key with a defined spec. So I'm now 
restricting what used to be completely open. Isn't that breakage? I feel like 
I'm seeing it exactly opposite as Rich Hickey. He says if you disallow things, 
it's forever, because if you need to allow it later, that's breakage. But 
there's not enough explanation for me to understand. It seems like relaxing 
requirements. I feel like I'm missing something. In short: why is it forever?

He does mention is that logic engines don't have negation. Does this hint that 
we will want to be using logic engines to reason over our specs?

Thanks
Eric

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Eric Normand
Yeah, it seems like if s/keys specs add any keys (required or not) that
have specs defined, you're breaking things. Removing the spec definition
and removing the key is growth. This explains why people would want things
to be stricter at first because it would allow them to relax later.

The idea that specs have to be future-compatible as well as backwards
compatible is interesting but I haven't ever made that a goal of my
systems. Old specs have to work on new data and new specs have to work on
old data. Has this been explored somewhere? I'd love to read up on it.

I'm curious: are there any resources laying out these design principles of
spec and how it means we should be designing systems that use it? There are
a ton of things on the mechanics of it and how to do X, Y and Z with it.
But I haven't seen anything that talks about evolution on long timescales.
Can someone point me to some resources?



On Tue, Nov 14, 2017 at 12:35 PM, Sean Corfield  wrote:

> Eric does raise an interesting question tho’:
>
>
>
> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later
> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as
> breakage or growth? If clients were sending ‘d’ as a string before but you
> ignored it, it will break those clients. Clients that were not sending ‘d’
> will not be affected by the change. The old spec – allowing ‘d’ to be
> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as
> ‘int?’ so it passes your compatibility test.
>
>
>
> (we actually ran into this at work because a client app was passing a
> field we didn’t care about and we later decided that was an optional field
> but couldn’t be an empty string and it broke that client)
>
>
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>
> --
> *From:* clojure@googlegroups.com  on behalf of
> Seth Verrinder 
> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
> *To:* Clojure
> *Subject:* Re: [core.spec] Stricter map validations?
>
> I took part of the goal to be that specs themselves would remain
> compatible, so an old set of specs wouldn't start failing on data that
> conforms to a new but compatible set of specs. That sort of compatibility
> isn't possible when you go from disallowing something to allowing it.
>
> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:
>>
>> Hey everybody!
>>
>> I'm chiming in after seeing this linked to in The Repl (
>> https://therepl.net/).
>>
>> On Alex's suggestion, I rewatched Spec-ulation last night. The parts
>> about negation and evolution are towards the end. I was struck (once again)
>> by how clearly he picked apart changes. Relaxing a requirement is growth.
>> And adding requirements is breakage. But it left me with a question:
>>
>> Isn't disallowing a key and then allowing it (as optional) growth
>> (instead of breakage)? All of the old clients are still fine, and new
>> clients can use the key if they choose. You're relaxing the requirements.
>> Taking the opposite approach, I require some keys plus allow anything else.
>> Some clients will inevitably send me something with extra keys, which is
>> okay, they pass my specs. Later, I add in an optional key with a defined
>> spec. So I'm now restricting what used to be completely open. Isn't that
>> breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He
>> says if you disallow things, it's forever, because if you need to allow it
>> later, that's breakage. But there's not enough explanation for me to
>> understand. It seems like relaxing requirements. I feel like I'm missing
>> something. In short: why is it forever?
>>
>> He does mention is that logic engines don't have negation. Does this hint
>> that we will want to be using logic engines to reason over our specs?
>>
>> Thanks
>> Eric
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send ema

Re: [core.spec] Stricter map validations?

2017-11-14 Thread Robin Heggelund Hansen
Isn't this precisely why you should use namespaced keywords?

tirsdag 14. november 2017 19.43.55 UTC+1 skrev Sean Corfield følgende:
>
> Eric does raise an interesting question tho’:
>
>  
>
> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later 
> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as 
> breakage or growth? If clients were sending ‘d’ as a string before but you 
> ignored it, it will break those clients. Clients that were not sending ‘d’ 
> will not be affected by the change. The old spec – allowing ‘d’ to be 
> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as 
> ‘int?’ so it passes your compatibility test.
>
>  
>
> (we actually ran into this at work because a client app was passing a 
> field we didn’t care about and we later decided that was an optional field 
> but couldn’t be an empty string and it broke that client)
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
> --
> *From:* clo...@googlegroups.com   > on behalf of Seth Verrinder 
> >
> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
> *To:* Clojure
> *Subject:* Re: [core.spec] Stricter map validations? 
>  
> I took part of the goal to be that specs themselves would remain 
> compatible, so an old set of specs wouldn't start failing on data that 
> conforms to a new but compatible set of specs. That sort of compatibility 
> isn't possible when you go from disallowing something to allowing it.
>
> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote: 
>>
>> Hey everybody! 
>>
>> I'm chiming in after seeing this linked to in The Repl (
>> https://therepl.net/).
>>
>> On Alex's suggestion, I rewatched Spec-ulation last night. The parts 
>> about negation and evolution are towards the end. I was struck (once again) 
>> by how clearly he picked apart changes. Relaxing a requirement is growth. 
>> And adding requirements is breakage. But it left me with a question:
>>
>> Isn't disallowing a key and then allowing it (as optional) growth 
>> (instead of breakage)? All of the old clients are still fine, and new 
>> clients can use the key if they choose. You're relaxing the requirements. 
>> Taking the opposite approach, I require some keys plus allow anything else. 
>> Some clients will inevitably send me something with extra keys, which is 
>> okay, they pass my specs. Later, I add in an optional key with a defined 
>> spec. So I'm now restricting what used to be completely open. Isn't that 
>> breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He 
>> says if you disallow things, it's forever, because if you need to allow it 
>> later, that's breakage. But there's not enough explanation for me to 
>> understand. It seems like relaxing requirements. I feel like I'm missing 
>> something. In short: why is it forever?
>>
>> He does mention is that logic engines don't have negation. Does this hint 
>> that we will want to be using logic engines to reason over our specs?
>>
>> Thanks
>> Eric
>>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Didier

>
> Eric does raise an interesting question tho 


I think so too. I'm still finding it hard to come up with a single example 
of why allowing extra keys to validate would ever be useful even for 
non-breakage. I can't see how it would break anything.

I believe the only advantage of allowing extra keys, is to allow partial 
specs. So say you never had specs, and your API takes a ton of keys, and 
you start by only speccing half the keys it takes. That's the only use case 
I could think of.

Personally I also wouldn't consider a client who sends a key that is not 
used, to be broken if I now validate that he is not supposed to send it. I 
did not break the API, I'm only making him realize he might have been wrong 
on how the API worked since the beginning, if he thought somehow that this 
key was a valid input and maybe expected it to alter the behavior. But in 
terms of semantics and behavior, nothing has changes or been broken on my 
API.

On Tuesday, 14 November 2017 11:20:14 UTC-8, Robin Heggelund Hansen wrote:
>
> Isn't this precisely why you should use namespaced keywords?
>
> tirsdag 14. november 2017 19.43.55 UTC+1 skrev Sean Corfield følgende:
>>
>> Eric does raise an interesting question tho’:
>>
>>  
>>
>> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later 
>> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as 
>> breakage or growth? If clients were sending ‘d’ as a string before but you 
>> ignored it, it will break those clients. Clients that were not sending ‘d’ 
>> will not be affected by the change. The old spec – allowing ‘d’ to be 
>> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as 
>> ‘int?’ so it passes your compatibility test.
>>
>>  
>>
>> (we actually ran into this at work because a client app was passing a 
>> field we didn’t care about and we later decided that was an optional field 
>> but couldn’t be an empty string and it broke that client)
>>
>>  
>>
>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
>> An Architect's View -- http://corfield.org/
>>
>> "If you're not annoying somebody, you're not really alive."
>> -- Margaret Atwood
>>
>>  
>> --
>> *From:* clo...@googlegroups.com  on behalf of 
>> Seth Verrinder 
>> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
>> *To:* Clojure
>> *Subject:* Re: [core.spec] Stricter map validations? 
>>  
>> I took part of the goal to be that specs themselves would remain 
>> compatible, so an old set of specs wouldn't start failing on data that 
>> conforms to a new but compatible set of specs. That sort of compatibility 
>> isn't possible when you go from disallowing something to allowing it.
>>
>> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote: 
>>>
>>> Hey everybody! 
>>>
>>> I'm chiming in after seeing this linked to in The Repl (
>>> https://therepl.net/).
>>>
>>> On Alex's suggestion, I rewatched Spec-ulation last night. The parts 
>>> about negation and evolution are towards the end. I was struck (once again) 
>>> by how clearly he picked apart changes. Relaxing a requirement is growth. 
>>> And adding requirements is breakage. But it left me with a question:
>>>
>>> Isn't disallowing a key and then allowing it (as optional) growth 
>>> (instead of breakage)? All of the old clients are still fine, and new 
>>> clients can use the key if they choose. You're relaxing the requirements. 
>>> Taking the opposite approach, I require some keys plus allow anything else. 
>>> Some clients will inevitably send me something with extra keys, which is 
>>> okay, they pass my specs. Later, I add in an optional key with a defined 
>>> spec. So I'm now restricting what used to be completely open. Isn't that 
>>> breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He 
>>> says if you disallow things, it's forever, because if you need to allow it 
>>> later, that's breakage. But there's not enough explanation for me to 
>>> understand. It seems like relaxing requirements. I feel like I'm missing 
>>> something. In short: why is it forever?
>>>
>>> He does mention is that logic engines don't have negation. Does this 
>>> hint that we will want to be using logic engines to reason over our specs?
>>>
>>> Thanks
>>> Eric
>>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com.
>> For more options, visit https://groups.goog

Re: [core.spec] Stricter map validations?

2017-11-14 Thread Eric Normand
>
> I think so too. I'm still finding it hard to come up with a single example
> of why allowing extra keys to validate would ever be useful even for
> non-breakage. I can't see how it would break anything.
>

I think you're assuming you're validating API endpoints where client and
server are tightly coupled. I can imagine a microservices system where some
services put maps on a queue and others consume them. The consumers should
allow keys they don't understand. They and also need to validate that the
keys they need are there and of the right shape.




>
> On Tuesday, 14 November 2017 11:20:14 UTC-8, Robin Heggelund Hansen wrote:
>>
>> Isn't this precisely why you should use namespaced keywords?
>>
>> tirsdag 14. november 2017 19.43.55 UTC+1 skrev Sean Corfield følgende:
>>>
>>> Eric does raise an interesting question tho’:
>>>
>>>
>>>
>>> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later
>>> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as
>>> breakage or growth? If clients were sending ‘d’ as a string before but you
>>> ignored it, it will break those clients. Clients that were not sending ‘d’
>>> will not be affected by the change. The old spec – allowing ‘d’ to be
>>> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as
>>> ‘int?’ so it passes your compatibility test.
>>>
>>>
>>>
>>> (we actually ran into this at work because a client app was passing a
>>> field we didn’t care about and we later decided that was an optional field
>>> but couldn’t be an empty string and it broke that client)
>>>
>>>
>>>
>>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
>>> An Architect's View -- http://corfield.org/
>>>
>>> "If you're not annoying somebody, you're not really alive."
>>> -- Margaret Atwood
>>>
>>>
>>> --
>>> *From:* clo...@googlegroups.com  on behalf of
>>> Seth Verrinder 
>>> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
>>> *To:* Clojure
>>> *Subject:* Re: [core.spec] Stricter map validations?
>>>
>>> I took part of the goal to be that specs themselves would remain
>>> compatible, so an old set of specs wouldn't start failing on data that
>>> conforms to a new but compatible set of specs. That sort of compatibility
>>> isn't possible when you go from disallowing something to allowing it.
>>>
>>> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:

 Hey everybody!

 I'm chiming in after seeing this linked to in The Repl (
 https://therepl.net/).

 On Alex's suggestion, I rewatched Spec-ulation last night. The parts
 about negation and evolution are towards the end. I was struck (once again)
 by how clearly he picked apart changes. Relaxing a requirement is growth.
 And adding requirements is breakage. But it left me with a question:

 Isn't disallowing a key and then allowing it (as optional) growth
 (instead of breakage)? All of the old clients are still fine, and new
 clients can use the key if they choose. You're relaxing the requirements.
 Taking the opposite approach, I require some keys plus allow anything else.
 Some clients will inevitably send me something with extra keys, which is
 okay, they pass my specs. Later, I add in an optional key with a defined
 spec. So I'm now restricting what used to be completely open. Isn't that
 breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He
 says if you disallow things, it's forever, because if you need to allow it
 later, that's breakage. But there's not enough explanation for me to
 understand. It seems like relaxing requirements. I feel like I'm missing
 something. In short: why is it forever?

 He does mention is that logic engines don't have negation. Does this
 hint that we will want to be using logic engines to reason over our specs?

 Thanks
 Eric

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clo...@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+u...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr.

Re: [ANN] tools.deps.alpha 0.2.167 and cli updates

2017-11-14 Thread Eric Normand
Oh this looks cool!

On Tuesday, October 31, 2017 at 8:21:54 AM UTC-5, Alex Miller wrote:
>
> tools.deps.alpha 0.2.167 was released on Oct 26th. I have not announced 
> all interim releases but some of the recent changes include:
>
>- Overhaul of the provider extension points
>- Initial support for understanding different kinds of "manifests" in 
>local projects. Currently, only deps.edn projects are understood. Others 
>will be added.
>- Added new -Spom option to `clj` - this option will generate (or 
>update!) a pom.xml in the same project based on the contents of the 
>deps.edn file. When updating, only the dependencies section is updated and 
>other parts of the pom file will be left as is.
>- Added built-in s3 private Maven repo support (docs 
>
> ).
>  
>Many thanks to the creators of the s3-wagon-private project (which wraps 
>the Spring aws-maven project) for making this easy! 
>
> The brew installer has been updated to include these changes (versions 
> 1.8.0.179 for stable, 1.9.0-beta3.240 for --devel). If you installed using 
> the brew installer, use "brew upgrade clojure" or "brew upgrade clojure 
> --devel" to get the latest stable or development version. 
>
> Links:
>
>- Deps and CLI guide  - 
>updated for latest release
>- tools.deps.alpha project 
>
>- Change log 
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Eric Normand
>
> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later
> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as
> breakage or growth? If clients were sending ‘d’ as a string before but you
> ignored it, it will break those clients. Clients that were not sending ‘d’
> will not be affected by the change. The old spec – allowing ‘d’ to be
> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as
> ‘int?’ so it passes your compatibility test.



Isn't this precisely why you should use namespaced keywords?
>


Yes! Great point.

That sounds like a great thing to document in your API. Something like:
"all keys in com.xyz namespace are reserved and could be restricted at any
time. Please don't use them." You could send a warning back in your API if
they use any that don't validate.

Are there any resources that document these best practices? It seems like
Clojure is trying to push us down this road but there's no map yet at the
moment.



>
>
> tirsdag 14. november 2017 19.43.55 UTC+1 skrev Sean Corfield følgende:
>>
>> Eric does raise an interesting question tho’:
>>
>>
>>
>>
>>
>> (we actually ran into this at work because a client app was passing a
>> field we didn’t care about and we later decided that was an optional field
>> but couldn’t be an empty string and it broke that client)
>>
>>
>>
>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
>> An Architect's View -- http://corfield.org/
>>
>> "If you're not annoying somebody, you're not really alive."
>> -- Margaret Atwood
>>
>>
>> --
>> *From:* clo...@googlegroups.com  on behalf of
>> Seth Verrinder 
>> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
>> *To:* Clojure
>> *Subject:* Re: [core.spec] Stricter map validations?
>>
>> I took part of the goal to be that specs themselves would remain
>> compatible, so an old set of specs wouldn't start failing on data that
>> conforms to a new but compatible set of specs. That sort of compatibility
>> isn't possible when you go from disallowing something to allowing it.
>>
>> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:
>>>
>>> Hey everybody!
>>>
>>> I'm chiming in after seeing this linked to in The Repl (
>>> https://therepl.net/).
>>>
>>> On Alex's suggestion, I rewatched Spec-ulation last night. The parts
>>> about negation and evolution are towards the end. I was struck (once again)
>>> by how clearly he picked apart changes. Relaxing a requirement is growth.
>>> And adding requirements is breakage. But it left me with a question:
>>>
>>> Isn't disallowing a key and then allowing it (as optional) growth
>>> (instead of breakage)? All of the old clients are still fine, and new
>>> clients can use the key if they choose. You're relaxing the requirements.
>>> Taking the opposite approach, I require some keys plus allow anything else.
>>> Some clients will inevitably send me something with extra keys, which is
>>> okay, they pass my specs. Later, I add in an optional key with a defined
>>> spec. So I'm now restricting what used to be completely open. Isn't that
>>> breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He
>>> says if you disallow things, it's forever, because if you need to allow it
>>> later, that's breakage. But there's not enough explanation for me to
>>> understand. It seems like relaxing requirements. I feel like I'm missing
>>> something. In short: why is it forever?
>>>
>>> He does mention is that logic engines don't have negation. Does this
>>> hint that we will want to be using logic engines to reason over our specs?
>>>
>>> Thanks
>>> Eric
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+u...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/cloju

Re: [core.spec] Stricter map validations?

2017-11-14 Thread Daniel Compton
> I think so too. I'm still finding it hard to come up with a single
example of why allowing extra keys to validate would ever be useful even
for non-breakage. I can't see how it would break anything.
>
> I believe the only advantage of allowing extra keys, is to allow partial
specs. So say you never had specs, and your API takes a ton of keys, and
you start by only speccing half the keys it takes. That's the only use case
I could think of.

This is my core concern as well. It seems like it is going to lead to bugs
from typos or people expecting specs to have been defined for keywords that
weren't defined. It also is incongruous with other parts of spec like
s/cat, which does require specs to be registered at check time. I
understand there is a technical/implementation detail for the reasons here,
but it strikes me as a rough edge that people are likely to continually run
into. I've hit it myself already in projects where I spec'ed a map, but
didn't have some of the keys spec'ed. I discovered later my mistake, but
like Didier, I can't see any time that I would actually want this feature.

Because spec's information model is all about namespaced keywords that have
a unique unambiguous meaning (at least I think that's the idea?), allowing
specing of a map with keywords that don't exist because they may be defined
by some other part of the system doesn't make a lot of sense to me. Even if
this is a valid use case, it seems like a niche one that would be better
served by defining a new function that allows it (perhaps in another
library), rather than it being part of the behaviour of the more general
purpose s/keys.

On Wed, Nov 15, 2017 at 8:53 AM Eric Normand  wrote:

> If you have an API that cares about ‘a’, ‘b’, and ‘c’ and you later
>> specify that ‘d’ is optional and should be an ‘int?’, does that qualify as
>> breakage or growth? If clients were sending ‘d’ as a string before but you
>> ignored it, it will break those clients. Clients that were not sending ‘d’
>> will not be affected by the change. The old spec – allowing ‘d’ to be
>> ‘any?’ essentially – won’t fail on any data that omits ‘d’ or passes it as
>> ‘int?’ so it passes your compatibility test.
>
>
> Isn't this precisely why you should use namespaced keywords?
>>
>
>
> Yes! Great point.
>
> That sounds like a great thing to document in your API. Something like:
> "all keys in com.xyz namespace are reserved and could be restricted at
> any time. Please don't use them." You could send a warning back in your API
> if they use any that don't validate.
>
> Are there any resources that document these best practices? It seems like
> Clojure is trying to push us down this road but there's no map yet at the
> moment.
>
>
>
>>
>>
>> tirsdag 14. november 2017 19.43.55 UTC+1 skrev Sean Corfield følgende:
>>
> Eric does raise an interesting question tho’:
>>>
>>>
>>>
>>>
>>>
>> (we actually ran into this at work because a client app was passing a
>>> field we didn’t care about and we later decided that was an optional field
>>> but couldn’t be an empty string and it broke that client)
>>>
>>
>>>
>>> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
>>> An Architect's View -- http://corfield.org/
>>>
>>> "If you're not annoying somebody, you're not really alive."
>>> -- Margaret Atwood
>>>
>>>
>>>
>> *From:* clo...@googlegroups.com  on behalf of
>>> Seth Verrinder 
>>> *Sent:* Tuesday, November 14, 2017 8:45:30 AM
>>> *To:* Clojure
>>> *Subject:* Re: [core.spec] Stricter map validations?
>>>
>>> I took part of the goal to be that specs themselves would remain
>>> compatible, so an old set of specs wouldn't start failing on data that
>>> conforms to a new but compatible set of specs. That sort of compatibility
>>> isn't possible when you go from disallowing something to allowing it.
>>>
>>> On Tuesday, November 14, 2017 at 10:15:23 AM UTC-6, Eric Normand wrote:

 Hey everybody!

 I'm chiming in after seeing this linked to in The Repl (
 https://therepl.net/).

 On Alex's suggestion, I rewatched Spec-ulation last night. The parts
 about negation and evolution are towards the end. I was struck (once again)
 by how clearly he picked apart changes. Relaxing a requirement is growth.
 And adding requirements is breakage. But it left me with a question:

 Isn't disallowing a key and then allowing it (as optional) growth
 (instead of breakage)? All of the old clients are still fine, and new
 clients can use the key if they choose. You're relaxing the requirements.
 Taking the opposite approach, I require some keys plus allow anything else.
 Some clients will inevitably send me something with extra keys, which is
 okay, they pass my specs. Later, I add in an optional key with a defined
 spec. So I'm now restricting what used to be completely open. Isn't that
 breakage? I feel like I'm seeing it exactly opposite as Rich Hickey. He
 says if you disallow things, it's forever, because if yo

Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Russell Mull
(apologies for the resend; this didn't reach the web interface the first
time around)

When I have a function whose arguments I have spec'ed with an fdef, and
that function is used in the construction of a lazy seq, it's easy to
stumble into situations where the arguments are not actually checked even
when the function is instrumented.

Here's a contrived example: (using [org.clojure/clojure "1.9.0-RC1"]
[org.clojure/spec.alpha "0.1.143"])

(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as st])

(s/fdef add1 :args (s/cat :x int?))
(defn add1 [x]
  (+ x 1))

(st/instrument [`add1])
(add1 1) ;; => 2
(add1 "1") ;; => spec error for args to add1


(s/fdef print-coll-with-banner
:args (s/cat :banner string?
 :xs (s/coll-of any?)))

(defn print-coll-with-banner [banner c]
  (println banner)
  (println c))


(print-coll-with-banner "hello adding world" (map add1 [1 2 3]))
;; => prints some stuff

(print-coll-with-banner "hello adding world" (map add1 [1 2 "3"]))
;; => spec error for args to add1

(st/instrument [`print-coll-with-banner])
(print-coll-with-banner "hello adding world" (map add1 [1 2 "3"]))
;; => ClassCastException inside of add1

This appears to be happening because the lazy sequence is actually realized
*by* spec when instrumentation is on, but at that point checking is
actually disabled with the st/with-instrument-disabled macro.

A few questions:

* Is this the intended behavior?
* The code in question has been there as far back as I can trace it in git.
What is this check trying to guard against?
* Does the fact that I got into this situation indicate that I'm doing
something wrong? (The above code is clearly contrived, but I did run into
this in a real program)

- Russell

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Didier
 | I think you're assuming you're validating API endpoints where client and 
server are tightly coupled. I can imagine a microservices system where some 
services put maps on a queue and others consume them. The consumers should 
allow keys they don't understand. They and also need to validate that the keys 
they need are there and of the right shape.

That use case sounds more like the exception then the rule to me. I still think 
its valid, and for that I recognise there should be a way to have a spec which 
allows any key and requires specific ones.

Now that I've listened to everyone, I see that there might be many things 
needed by different people, and its not clear if all should be provided by spec 
directly.

Mostly I can see wanting:
  A. To spec a map in which you expect the keys to have a registered spec.
  B. To spec a map in which you don't expect the keys to have a registered spec.
  1. To spec a map in which the keys should be closed to only the req and opt 
keys that are defined. 
  2. To spec a map in which the keys should be open to what is defined, but 
where certain keys are required and others reserved as optional.

Where we want to be able to mix and match A,B with 1,2.

I see a safe default being A,1. Though I'd be okay with A,2. I think B is the 
bigger issue not to have as default, and is less intuitive. It's prone to typos 
and forgetting to add the spec or require the proper namespace, etc.

My conclusion: I think s/keys is a funny part of spec, because compared to all 
other provided speccing tool, this one is really just syntax sugar for 
convenience. Yet it makes sense, because it is just so damn convenient. But for 
a lot of people like me, its convenient in a surprising way, which leans 
towards compatibility instead of correctness and security. And most baffling, 
trying to spec a map in a correct, secure and convenient way is not possible, 
at least not with the same convenience. So I'd say if there was just also 
convenient s/closed-keys and s/specified-keys and s/specified-closed-keys 
everyone would be happy, and it would also be more explicit what distinction 
exist between all 4. I'd probably always reach first for 
s/specified-closed-keys, and relax if I need too only, but all least everyone 
would have what they want and could chose for themselves.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Didier
That's weird, not sure why you get ClassCastException and not the spec error of 
add1.

That said, you can not validate a lazy-seq without realizing it. You can use 
s/every? on big collections, that will sample x element and validate only those.

So its normal for coll-of to realize the lazy-seq, as it is looping on all its 
elements and checking that they all conform.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Alex Miller


On Tuesday, November 14, 2017 at 4:16:00 PM UTC-6, Russell Mull wrote:
>
>
> This appears to be happening because the lazy sequence is actually 
> realized *by* spec when instrumentation is on, but at that point checking 
> is actually disabled with the st/with-instrument-disabled macro.
>
> A few questions:
>
> * Is this the intended behavior?
> * The code in question has been there as far back as I can trace it in 
> git. What is this check trying to guard against?
> * Does the fact that I got into this situation indicate that I'm doing 
> something wrong? (The above code is clearly contrived, but I did run into 
> this in a real program)
>

Hey Russell, this is a good repro and a good question. The intent here is 
to turn off instrumentation while doing the args conform, then re-enable it 
when the function is actually invoked so that you don't end up in a 
scenario where you are recursing through endless instrumentation frames 
(for cases where a spec uses functions that are also instrumented).

Certainly, the laziness of the input coll is the confounding factor here. 
One option is to use the non-conforming s/every instead of the conforming 
s/coll-of:

(s/fdef print-coll-with-banner
   :args (s/cat :banner string? :xs (s/every any?)))

You'll find with this change that things work as you expect. 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Didier
 | One option is to use the non-conforming s/every instead of the conforming 
s/coll-of

Why isn't instrument just validating for all spec? Doesn't conform serve no 
purpose here?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Alex Miller

On Tuesday, November 14, 2017 at 11:11:27 PM UTC-6, Didier wrote:
>
> | One option is to use the non-conforming s/every instead of the 
> conforming s/coll-of
>
> Why isn't instrument just validating for all spec? Doesn't conform serve 
> no purpose here?
>

Indeed, this could probably be changed now. Originally, instrumentation 
would validate :ret and :fn specs too and the fn spec took the conformed 
args and ret values.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Surprising st/instrument behavior with lazy seqs

2017-11-14 Thread Alex Miller
Logged this at https://dev.clojure.org/jira/browse/CLJ-2266 with a patch.

On Tuesday, November 14, 2017 at 11:26:45 PM UTC-6, Alex Miller wrote:
>
>
> On Tuesday, November 14, 2017 at 11:11:27 PM UTC-6, Didier wrote:
>>
>> | One option is to use the non-conforming s/every instead of the 
>> conforming s/coll-of
>>
>> Why isn't instrument just validating for all spec? Doesn't conform serve 
>> no purpose here?
>>
>
> Indeed, this could probably be changed now. Originally, instrumentation 
> would validate :ret and :fn specs too and the fn spec took the conformed 
> args and ret values.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [core.spec] Stricter map validations?

2017-11-14 Thread Greg Mitchell
Saw this thread on the REPL, interesting discussion.

On Tuesday, October 3, 2017 at 10:57:34 AM UTC-7, Alex Miller wrote:
>
>
> It's not about easier, it's about possible. Open grows, closed breaks.
>   
>

I agree with the broad brush, but it's important to listen to this 
usability feedback before releasing an important feature like Spec. As an 
aside, seriously great work with Spec; it's the biggest thing Clojure was 
missing for institutional support I think. 

At Google, we use Protocol Buffers 
 heavily to achieve some 
similar goals to Spec, and I think there are good lessons to look at. Like 
you said, the spec (whether proto or capital-s Spec) needs to be forwards 
compatible across distributed clients if it's going to describe data in an 
useful way as the project evolves. Protos achieve that by convention - best 
practice recommends fields are always optional and the type can never 
change - and technically by always accepting unknown fields.

Where they differ highlights the pain points that people in this thread 
mentioned. 
- Protos cannot be created with extra fields, so there's nothing like the 
sanitizing issue puzzler mentioned. The generalization is to strictly 
specify the output of a function/system, but gracefully handle input. This 
makes me think there should be something like a s/conform-strict function 
that coerces data to only the Spec'd keys, and ensures keys are defined.
- Protos specify the type of a field along with its name, and both are 
checked at compile time, so there aren't the late-caught errors like the OP 
pointed out. Rich has good reasoning for wanting to re-use keys, but 
typically the solution is to create a named composite type (a message in 
proto terms) that is trivial to re-use as a field type. The idea of 
specifying keys separately is different from how literally every type 
system does things, so it's inherently not "easy" in Rich's definition of 
being mentally at hand. That's going to create pain for people learning the 
tool. You should strongly think about ways to reduce that friction.

Protos describe a strictly specified open message, which doesn't neatly fit 
in your categories or open or closed. It's an important technical precedent 
to think about though.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.