> 852 questions over the past month Wow, that's much more than I expected!
Very cool, thanks again for your work on this Nic! On Thu, Jun 12, 2025 at 3:29 PM Nic Crane <thisis...@gmail.com> wrote: > An update for folks on this - I've extracted a few statistics below from > the main dashboard. We said it would be worth testing this out for 1 > release to see what we thought. I haven't had time to do a deep dive, but > overall, I do think it is providing value for users, so we should keep it > there. Happy to discuss though. > > 852 questions over the past month, averaging around 191 per week. > > People aren't using the feedback mechanisms much, but there were 19 thumbs > up and 3 thumbs down. > > In 90% of cases, the bot was able to give a confident response. > > Languages: > * English - 88% > * Chinese - 8% > * Russian - 3% > * Japanese - 1% > > Topics: > * Unsupported features - 14% > * Troubleshooting - 9% > * Comparison with similar technologies - 2% > * Off topic questions - 2% > > If there are any other PMC members or committers who'd like access to the > dashboard, please do let me know. > > On Mon, 12 May 2025 at 15:02, Matt Topol <zotthewiz...@gmail.com> wrote: > > > Thanks! > > > > On Mon, May 12, 2025, 9:47 AM Nic Crane <thisis...@gmail.com> wrote: > > > > > Absolutely Matt - I'm adding them now and I'll add you as an admin so > > > you'll be able to review Go questions in the UI and make any changes > > you'd > > > like to what we include. > > > > > > On Wed, 7 May 2025 at 12:59, Matt Topol <zotthewiz...@gmail.com> > wrote: > > > > > > > Would we be able to feed it the arrow-go documentation too? > > > > > > > > On Wed, May 7, 2025, 12:52 PM Nic Crane <thisis...@gmail.com> wrote: > > > > > > > > > A brief update on this from me - I've now added the Python, R, C++, > > and > > > > > Java cookbooks as a source for the kapa.ai question bot, so code > > > > examples > > > > > from those should be on there now. > > > > > > > > > > At a later point, I'll take a look at potentially adding GitHub PRs > > and > > > > > issues, but I'd like to test what we have so far before adding more > > in > > > > > there. > > > > > > > > > > > > > > > On Mon, 28 Apr 2025 at 04:56, Nic Crane <thisis...@gmail.com> > wrote: > > > > > > > > > > > We've current had 64 questions, though many of these are from > folks > > > > > > testing it out. I've personally found it useful for some of my > dev > > > > work. > > > > > > > > > > > > It's hard to tell yet if it's useful for users but we'll have > more > > > data > > > > > > once it's live on the latest docs rather than the dev docs, so my > > > plan > > > > is > > > > > > to keeping checking in on the questions and answers between the > > > > upcoming > > > > > > release and the next one so we can decide whether it's a useful > > tool > > > > for > > > > > us > > > > > > to have. > > > > > > > > > > > > If anyone would like a login for kapa.ai so you can see the > > > > > conversations > > > > > > and answers (and add in any feedback you'd like to add yourself > for > > > > > > questions), then please email me and I'll add you as a user. > > > > > > > > > > > > > > > > > > On Wed, 23 Apr 2025 at 06:11, Nic Crane <thisis...@gmail.com> > > wrote: > > > > > > > > > > > >> > nit: any chance the popup can be made to respect the > light/dark > > > mode > > > > > >> toggle > > > > > >> in the docs? It seems to always use a light color scheme. > > > > > >> > > > > > >> From the looks of it, the colour can be configured via the > > > > > >> 'data-project-color' parameter[1] so I imagine this could be > done > > > via > > > > > >> updating the widget script with some custom JS on our side. > > > > > >> > > > > > >> [1] > > https://docs.kapa.ai/integrations/website-widget/configuration > > > > > >> > > > > > >> On Mon, 31 Mar 2025 at 17:15, Brian Hulette < > bhule...@apache.org> > > > > > wrote: > > > > > >> > > > > > >>> Very cool, I tried a few queries and it provided good answers. > > > > > >>> > > > > > >>> nit: any chance the popup can be made to respect the light/dark > > > mode > > > > > >>> toggle > > > > > >>> in the docs? It seems to always use a light color scheme. > > > > > >>> > > > > > >>> On Fri, Mar 28, 2025 at 5:05 PM Nic Crane <thisis...@gmail.com > > > > > > wrote: > > > > > >>> > > > > > >>> > Cookbooks aren't part of the sources but we can certainly add > > > them > > > > > >>> later. > > > > > >>> > > > > > > >>> > The kapa folk recommend trying with an initial sunset of > > sources > > > > and > > > > > >>> adding > > > > > >>> > more in gradually, so I'll check back in on this next week > and > > > > think > > > > > >>> about > > > > > >>> > what we might want to add next. > > > > > >>> > > > > > > >>> > On Fri, 28 Mar 2025, 21:45 Aldrin, > <octalene....@pm.me.invalid > > > > > > > > wrote: > > > > > >>> > > > > > > >>> > > Ooh, yeah it's looking fairly effective. I asked some > > questions > > > > > and I > > > > > >>> > like > > > > > >>> > > that the answers address differences in language > > > implementations > > > > > >>> (e.g. > > > > > >>> > > python bindings vs cpp) and that there are relatively good > > code > > > > > >>> > suggestions. > > > > > >>> > > > > > > > >>> > > I assume this means the tests and benchmarks are helping > the > > > RAG > > > > a > > > > > >>> lot, > > > > > >>> > so > > > > > >>> > > general well done to all contributors. Out of curiosity, > are > > > the > > > > > >>> > cookbooks > > > > > >>> > > part of the learning/retrieval sources? > > > > > >>> > > > > > > > >>> > > Sent from Proton Mail <https://proton.me/mail/home> for > iOS > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > On Fri, Mar 28, 2025 at 13:58, Neal Richardson < > > > > > >>> > > neal.p.richard...@gmail.com > > > > > >>> > > > > <On+Fri,+Mar+28,+2025+at+13:58,+Neal+Richardson+%3C%3Ca+href=>> > > > > > >>> wrote: > > > > > >>> > > > > > > > >>> > > Awesome, thanks for doing this! I tried out a few queries > and > > > was > > > > > >>> > > impressed > > > > > >>> > > with what I got back. > > > > > >>> > > > > > > > >>> > > Neal > > > > > >>> > > > > > > > >>> > > On Fri, Mar 28, 2025 at 2:01 PM Nic Crane < > > thisis...@gmail.com > > > > > > > > > >>> wrote: > > > > > >>> > > > > > > > >>> > > > Hi folks, > > > > > >>> > > > > > > > > >>> > > > The kapa.ai bot is now live on the dev docs! We're using > > it > > > to > > > > > >>> test > > > > > >>> > > > whether > > > > > >>> > > > we can enhance user experience of our documentation via > > > > retrieval > > > > > >>> > > augmented > > > > > >>> > > > generation which generates answers to user questions by > > > > combining > > > > > >>> > > > similarity searches of the docs and LLM output. > > > > > >>> > > > > > > > > >>> > > > You can use it by clicking the "Ask AI" button in the > > bottom > > > > > right > > > > > >>> > > corner > > > > > >>> > > > of https://arrow.apache.org/docs/dev/index.html > > > > > >>> > > > > > > > > >>> > > > Currently we've just ingested the docs, but as we > > experiment > > > > > more, > > > > > >>> we > > > > > >>> > > can > > > > > >>> > > > add other sources like Zulip and the mailing lists if we > > > think > > > > > >>> these > > > > > >>> > > will > > > > > >>> > > > provide useful additional source material. > > > > > >>> > > > > > > > > >>> > > > Please give it a go and use the "good answer" and "bad > > > answer" > > > > > >>> buttons > > > > > >>> > > so > > > > > >>> > > > we can assess how useful this is likely to be for our end > > > > users; > > > > > >>> bonus > > > > > >>> > > > points for adding a better answer when you're not happy > > with > > > > the > > > > > >>> one > > > > > >>> > > given. > > > > > >>> > > > > > > > > >>> > > > I'll be checking in on it every so often and will report > > back > > > > > here > > > > > >>> with > > > > > >>> > > the > > > > > >>> > > > results. > > > > > >>> > > > > > > > > >>> > > > Thanks, > > > > > >>> > > > > > > > > >>> > > > Nic > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >> > > > > > > > > > > > > > > >
