I like your thinking. Replacing UTs with asserts sounds pretty interesting to me. Let me give a good thought about it. There's clearly enormous potential, but there's also potentially many unforeseen implications. But we should continue this discussion offline or in a separate thread. I have duly noted your suggestion. :-)
On Mon, Nov 26, 2018 at 12:48 PM robert engels <reng...@ix.netcom.com> wrote: > If you re-read what you wrote you will see the problem - you weren’t > setting a prev/next link correctly - the proper way test test these > invariants IMO is to use asserts within the implementation itself - that > way the implementation change change without breaking all of the tests. > Otherwise code is impossible to refactor - as you need to “determine the > design” by reading very low-level unit tests - when I review your unit > tests they tell me me nothing explicitly about the design - I have to > “figure that out”… not good. > > In many ways, your unit tests present more complex/obscure logic to > understand than the code itself… > > I’ll repeat, I can’t find any “unit tests” of that type in the Go stdlib, > and reviewing the ones I do find (not in a different _test package) it > seems it was a mistake, as they still only exercise the public API - > that’s a hint to me that you are doing something wrong. > > > > On Nov 26, 2018, at 2:34 PM, Christian Petrin <christianpet...@gmail.com> > wrote: > > It's funny because I'm not a big fan of unit testing for all the same > reasons you mentioned. Impl7 > <https://github.com/christianrpetrin/queue-tests/blob/master/queueimpl7/queueimpl7_test.go>, > for instance, I didn't implement any unit tests as the implementation was > very simple and integration tests is sufficient. > > With deque, I initially implemented just integration tests. They were all > passing and deque was behaving, according to the tests, correctly, always > returning the expected results. When I deemed to have finalized the > implementation (checking its correctness via integration tests), the deque > was not performing as expected in a number of test scenarios. That caused > me to scratch my head pretty hard for a good deal of time until I realized > there was nothing wrong with the design. There should have some issue with > the implementation. I tried my best to find out where the problem was, but > failed. So I realized I should implement unit tests to make sure the > implementation was not only correct, but follows the design perfectly. > > With the UTs, I was able to find the bug. A very sneaky one where I was > not setting a link (prev/next) correctly, and that caused the deque to not > use the spare links when the design dictate the deque should. Even with the > bug, the queue was performing correctly, but not efficiently. > > In any case, my point is a well validated implementation should have tests > at all levels (unit, integration, API, e2e) as some bugs can only be > discovered with some types of tests. > > On Mon, Nov 26, 2018 at 10:09 AM robert engels <reng...@ix.netcom.com> > wrote: > >> I am just offering, and many would disagree, that unit tests that like >> are brittle and not worthwhile. I don’t see them in the Go stdlib for major >> packages, but I could be wrong. >> >> My thought on this is that if you are testing the nitty details of the >> implementation, you need to get that “correct” twice - and if it is >> failing, you don’t really know which one is wrong… This is especially >> problematic as things are refactored - all of those types of tests break. >> >> I’m of the opinion - test the behavior comprehensively, and expose the >> expected usage (and corresponding design) there. >> >> On Nov 26, 2018, at 11:58 AM, Christian Petrin <christianpet...@gmail.com> >> wrote: >> >> Hi Robert, >> >> the deque has unit, integration and API tests. The unit tests, I agree, >> are hard to follow as they have to check all internal properties to make >> sure the code is doing what it is supposed to do (keep a well formed ring >> linked list during all steps). Given their nature (unit tests), the only >> way to access the internal deque variables is to have them in the same >> package. >> >> Regarding the other, API and integration tests, I agree 100%. They should >> be in the dequeue_test package as they are not supposed to access (or know >> of) any of the deque internal variables. I'll move these tests to a >> different file in the dequeue_test package. >> >> Really appreciate the suggestion (correction, really). >> >> Thanks, >> Christian >> >> >> >> >> On Mon, Nov 26, 2018 at 9:42 AM robert engels <reng...@ix.netcom.com> >> wrote: >> >>> Just an FYI, IMO your testing is incorrect. >>> >>> Your tests should be in a package dequeue_test so that they cannot >>> access the internal details. This makes them brittle and hard to follow. >>> >>> For reference, look at the Go stdlib sync/map.go and sync/map_test.go. >>> All of the tests are in sync_test for all of the structs/methods. >>> >>> >>> On Nov 26, 2018, at 10:59 AM, Christian Petrin < >>> christianpet...@gmail.com> wrote: >>> >>> Hello, >>> >>> I'm working on a logging system called CloudLogger >>> <https://github.com/cloud-logger/docs> and to cut to the chase, >>> CloudLogger needs an unbounded in-memory queue. The idea is to use the >>> queue as a sequential data store. As there's no telling beforehand how much >>> data will need to be stored, the queue has to be able to scale to support >>> large amounts of data, so CloudLogger needs an unbounded queue, a very fast >>> and efficient one. >>> >>> Noticing Go doesn't offer an unbounded queue implementation, I came up >>> with an idea to build a queue based on linked slices. >>> >>> I was so impressed with the results that I decided to spend some time >>> researching about other queue designs and ways to improve it. I'm finally >>> done with the research and I feel like the new queue is worth being >>> considered to be part of the standard library. >>> >>> So please take a look at the issue. All suggestions, questions, >>> comments, etc are most welcome! >>> >>> Due to many suggestions, I have deployed the deque as a proper external >>> package. In order to help validate the deque and get it into the standard >>> library, I need to somehow get people to start using it. If you are using a >>> queue/stack/deque, please consider switching to the new deque. Also if you >>> know someone who is using a queue/stack/deque, please let them know about >>> the deque. >>> >>> I'm extremely interested in improving the queue and I love challenges. >>> If you suspect there's a possibly better design or a better, more >>> performant and balanced deque implementation out there, please let me know >>> and I'm very glad to benchmark it against deque. >>> >>> Proposal: https://github.com/golang/go/issues/27935 >>> Deque package: https://github.com/ef-ds/deque >>> >>> Thanks, >>> Christian >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >> >> -- >> Thanks, >> Christian >> >> >> > > -- > Thanks, > Christian > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > -- Thanks, Christian -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
