Abstract
Task: building uncertainty-aware suggestions based on a decision-theoretic model of goal-conditional utility,推理LLM用户的未观测到的意图
方法:a decision-theoretic model of goal-conditioned utility, 使用生成式模型生成的random samples来做proxy, minimum Bayes-risk decoding, dual decomposition, decision diagrams.
实验:
数据集:3 developer-assistance tasks
效果:与per-token probability相比,获取more useful uncertainty estimates
- Uncertainty-Aware Uncertainty Suggestions Maximizing R-U-SUREuncertainty-aware uncertainty suggestions maximizing uncertainty-aware r-u-sure maximizing suggestions performance maximizing techniques llm auto-suggestions uncertainty diagnostics information prod-ad-api suggestions bootstrapping pessimistic uncertainty offline