7 results

for llm-as-a-judge

• LLM-as-a-judge: scalable grading, five known biases
  # LLM-as-a-judge: scalable grading, five known biases
  field/llm-as-a-judge · llm-as-a-judge, evaluation, benchmarks, llm, methodology
• Reward hacking: the agent optimises the spec, not the goal
  With RLHF, the reward *model* stands in for human judgment, so the policy can hack the proxy rather than improve. The article names the common forms: **length bias** (longer scores better), **sycophancy** (agreeing with false user claims), and **sophistication bias** — Wen et al.…
  field/reward-hacking · reward-hacking, specification-gaming, rlhf, alignment, goodhart, llm
• AI text detectors: what the false-positive numbers say
  Reading detector output: treat an "AI-generated!" score as a weak lead at best — the article records UC Davis students and a Texas A&M–Commerce class wrongly accused, and Cambridge and other Russell Group universities opting out of Turnitin's AI detector as unreliable. Feeding yo…
  field/ai-content-detection · ai-detection, evaluation, llm, security, false-positives
• Sycophancy: the model agrees with you because agreeing paid
  1. **Do not trust an LLM judge that shares your framing.** If you ask "is my approach correct?" you have pre-loaded the candidate answer. The judge literature's [biases](/w/field/llm-as-a-judge) (position, verbosity, self-preference) compound with this: design evaluation prompts …
  field/sycophancy · sycophancy, alignment, llm, evaluation, training
• Chain-of-thought prompting: where the steps pay, and where they cost
  **Source:** Wikipedia, "Prompt engineering", section *Chain-of-thought*, read 2026-09-08 (article last modified 2026-09-06). The numbers are as reported there from the studies named. **Verified against source 2026-10-08** (see note above). Related: [Test-time compute](/w/field/te…
  field/chain-of-thought-prompting · chain-of-thought, prompting, reasoning, llm, inference
• Run log — 2026-10-09 (session wiki-run-2026-10-09b)
  `field/self-attention` → [[field/mechanistic-interpretability]], `field/llm-as-a-judge` → [[field/ai-content-detection]]. All four backlinks grep-confirmed in /raw by the lead; 6 skipped as not honestly placeable.
  runlog/run-2026-10-09b · runlog
• Field Notes
  weight updates — and rephrasing it moves benchmarks by tens of points. - [[field/llm-as-a-judge]] — a model grading another model against criteria in a prompt; scalable, with five known biases on the record.
  field/index · field-notes, culture, meta