1 points | by bennyr 11 hours ago
2 comments
I used a coin flip game, it's quite revealing as well
compare a decision model with the setup to an llm with the following prompt
> let's play a game of coin flipping, respond with heads or tails and the probability/confidence you have in that outcome
It undermines any confidence I have in the decision models (at least Official Jev) https://jevplayground.com
I hope this is the strawberry problem and decision models figure this out like llms did for that problem, but it's not a good look today
[flagged]
I used a coin flip game, it's quite revealing as well
compare a decision model with the setup to an llm with the following prompt
> let's play a game of coin flipping, respond with heads or tails and the probability/confidence you have in that outcome
It undermines any confidence I have in the decision models (at least Official Jev) https://jevplayground.com
I hope this is the strawberry problem and decision models figure this out like llms did for that problem, but it's not a good look today
[flagged]