Interesting. Part of the "rationality community" that I find troubling is their tendency to *game* everything, including things like charity (Effective Altruism). But the good news is that your attempt to game this test is what the test *wants* you to do, i.e. play around with your tacit bindings between belief and knowledge. Given that I doubt everything, every single test I've ever taken has been filled with "trick questions" precisely because they don't have various spectra associated. Each T/F question should look more like those Strongly Disagree ... Strongly Agree questions. Every question needs a "confidence selector". Etc. Such is the curse of the agnostic.
Anyway, it wouldn't be very difficult to replicate this test's measures, but on a large, pseudo-randomized database of questions. I haven't done the search. But my guess is it already exists somewhere. On 9/29/21 2:39 PM, Steve Smith wrote: > > I retook the test back-to-back and was a little surprised by the results: > > 1. I did increase my high confidence correct answers marginally > (unsurprising) > 1. This means I still got a few dead wrong. > 2. I did lower my overall confidence. > 1. no-brainer after seeing how overconfident I was first time around > 3. I lowered my overall correct answers (this is the surprise). > 1. not sure what this is about, trying too hard to "flip" my guesses and > getting them wrong? > 2. both times, I got 100% of my 50% confidence answers correct (3 or 4 > of them?) > > I would have probably been more better at improving my results if I'd paid > more attention the first time to how I answered the low-confidence > questions... even though I was "guessing" I quoted a higher-than-50% > confidence.. Sounds dumb huh? > > >>> Do you know what you know? >>> A Confidence Calibration Exercise >>> http://confidence.success-equation.com/ >> >> I share Glen's interest in retaking such a test under different personal >> contexts. I found some of the questions seemingly a little disingenous and >> was surprised by the modest number that were easy to answer with high >> confidence. A randomly selected set from a larger group might give me a >> slightly different mix of these. >> >> Unsurprisingly (to me if not everyone), my Percent Correct was lower than >> Glen's while my Confidence was higher. >> >> The only thing I feel a little proud of was that most of my high confidence >> answers were in fact correct. >> >> I think I might have gotten better scores if I'd followed an intuition that >> the questions were worded to yield an equal distribution of true/false >> questions... I definitely allowed my own optimistic nature to bias toward >> answering "yes" rather than "no" when I had low confidence. A second pass >> through the questions with that in mind would probably have had me flipping >> some of my low-confidence "true"s to low confidence "false"s. Maybe this is >> an incorrect assumption about the design of the test. >> >> I may take it again to see if that improves my hit rate... I think my >> performance *would* be skewed by having seen the evaluation... knowing that >> a few of my high confidence answers were *wrong* will surely yield a few >> more "hedged bets" there... if I study the results with an eye to >> improving my scores, I can probably recognize a few other systematic areas >> for improvement. >> >>> "After answering each of the true/false questions below, indicate how >>> confident you are in your answer using the corresponding slider. A value of >>> 50% means you have no idea what the right answer is (the same probability >>> as a random guess between the two choices); a value of 100% means you are >>> completely confident in your answer." >>> >>> It seems to present the same questions each time, which is a shame. I'd >>> love to try it fully alert. But my attempt at 4am, with an irritating >>> headache, turned out this way: >>> >>> Mean confidence: 61.60% >>> Actual percent correct: 78.00% >>> You want your mean confidence and actual score to be as close as possible. >>> Mean confidence on correct answers: 63.59% >>> Mean confidence on incorrect answers: 54.55% >>> You want your mean confidence to be low for incorrect answers and high for >>> correct answers. >>> >>> Quiz score >>> 39 correct out of 50 questions answered (78.00%) >>> 27 correct out of 38 questions answered with low (50 or 60%) confidence >>> (71.05%) >>> 5 correct out of 5 questions answered with medium (70% or 80%) confidence >>> (100.00%) >>> 7 correct out of 7 questions answered with high (90 or 100%) confidence >>> (100.00%) -- "Better to be slapped with the truth than kissed with a lie." ☤>$ uǝlƃ .-- .- -. - / .- -.-. - .. --- -. ..--.. / -.-. --- -. .--- ..- --. .- - . FRIAM Applied Complexity Group listserv Zoom Fridays 9:30a-12p Mtn UTC-6 bit.ly/virtualfriam un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
