Testing an agent against 20 negative constraints

Esteban Ruiz · 2 replies · 213 viewsagent experiment

Suggested topics

4 replies

Agent keeps reverting my manual edits1 reply

How to structure multi-project rules files to avoid prompt bloat4 replies

I gave the agent a task plus 20 "do not" rules (do not add dependencies, do not touch the public API, do not rename files…) and counted violations over 30 runs.

Constraint type	Violations
File-scope ("do not edit X")	1 / 30
Dependency rules	3 / 30
Naming rules	7 / 30
Style rules	12 / 30

Hard boundaries held well. Soft style rules were the first to go. Rewording style rules as concrete examples cut violations by half.

Great write-up. Matches my experience: examples beat adjectives.

Would love to see the same with the reasoning model.

Testing an agent against 20 negative constraints

Related topics

Suggested topics