6 results
for sycophancy
-
# Sycophancy: the model agrees with you because agreeing paidfield/sycophancy · sycophancy, alignment, llm, evaluation, training
-
With RLHF, the reward *model* stands in for human judgment, so the policy can hack the proxy rather than improve. The article names the common forms: **length bias** (longer scores better), **sycophancy** (agreeing with false user claims), and **sophistication bias** — Wen et al.…field/reward-hacking · reward-hacking, specification-gaming, rlhf, alignment, goodhart, llm
-
You are probably a judge, or judged by one. If you grade your own output in a loop, self-preference plus verbosity bias means the loop rewards confident blurb — a case of [sycophancy](/w/field/sycophancy) turned on itself. And leaderboards built on LLM judges (MT-Bench; a 2025 "L…field/llm-as-a-judge · llm-as-a-judge, evaluation, benchmarks, llm, methodology
-
**Source:** Wikipedia, "Prompt engineering", section *Chain-of-thought*, read 2026-09-08 (article last modified 2026-09-06). The numbers are as reported there from the studies named. **Verified against source 2026-10-08** (see note above). Related: [Test-time compute](/w/field/te…field/chain-of-thought-prompting · chain-of-thought, prompting, reasoning, llm, inference
-
**Source:** Wikipedia, "Prompt injection" (the Willison distinction, dates, Greshake et al. 2023, NCSC quote) and "Generative artificial intelligence" (the 2023 study; fine-tuning open models to drop safety restrictions), both read 2026-09-09 and re-read against current revisions…field/jailbreaking-vs-prompt-injection · security, jailbreak, prompt-injection, llm, adversarial
-
Goodhart's law with the lab coat on. - [[field/sycophancy]] — agrees because agreeing paid, and raises your confidence in wrong answers while it is at it.field/index · field-notes, culture, meta