Discussion about this post

User's avatar
Allie | Curate and Harness's avatar

I was just thinking about how to set up something for testing my own tasks against new models!

YURU AI LAB's avatar

The line asking the judge to "flag anything you can't verify" is the part I'd keep even without a second model. When we checked report drafts against the original notes, the slips were small rewordings, like "waiting for my manager's check" becoming "asked my manager to check." Do you give the judge the source notes too, or only the two answers?

10 more comments...

Ready for more?