jmckenzie-dev commented on PR #4734: URL: https://github.com/apache/cassandra/pull/4734#issuecomment-4260389480
> There's some truth to this, but I think it's still useful to say what not to do. I've found in my own repos telling the LLM not to write mock-echo tests where it just verifies the mocked thing returns results, to be quite helpful. I think it helps a lot to provide the positive direction in addition to the negative. As is tradition, I communicated the tip of the iceberg on what was in my head. 😠What I meant to get across: we should absolutely do both, but be aware that this is a current limitation so lean on the positive hard knowing they'll generally comply with that and then all caps "THE WORLD WILL END IF YOU DO THIS BAD THING" + copy/paste multiple times to tweak the attention mechanism and get it to _maybe_ respect what we tell it not to do. So yeah - quite helpful, but understanding that the "mathematical vibe" is probably like 4:1 in terms of impact on saying what an LLM should do vs. what they should not in terms of their compliance. Plus it varies based on model too. Wild West. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
