Vercel reduced known failure modes by 57% when coding agents used a new design guidance file to generate web pages, according to a case study published Monday by the frontend platform company. The company conducted more than 200 agent runs to create design.md, a public prompt file that teaches agents how to build pages matching Vercel's brand and visual standards even when they can't access the company's internal code. The experiment offers a concrete example of how developers can turn human design judgment into reusable instructions for AI agents, though the results also highlight the technology's persistent reliability challenges.
In testing across three desktop scenarios, Vercel ran Codex with GPT-5.5 to generate six pages twice—once with design.md loaded and once without. Deterministic checks tallied 39 instances of known failure modes when agents used design.md, compared to 91 failures without the file, representing the 57% drop. Despite the improvement, every single page contained at least one flaw severe enough to block shipping, even after more than 200 refinement runs. The company noted that the sample size was admittedly small, covering just six pages, but the data pattern suggests that failures explicitly identified and written into guidance are less likely to happen again.
According to the report, the challenge arose because Vercel's existing internal tool, called product design, proved valuable for agents working inside the codebase but couldn't help external tools produce the same on-brand output. The company's initial attempt to simply convert product design into a public prompt failed because subjective design language was interpreted differently by each model, and critical design and implementation details remained locked in code that external agents couldn't reach. Vercel ultimately built design.md from scratch, testing each version against seven repeatable evaluation prompts to measure both what changes the file triggered and how different agents interpreted the instructions.
The final system divides guidance into three layers: a prompt file instructing agents on design decisions, copy, hierarchy, typography, color, and publishing while explicitly listing forbidden design patterns; a public stylesheet defining reusable implementation details like spacing and layout; and an evaluation loop converting human feedback into updated guidance and automated checks for mechanical failures. To keep the file current, Vercel created design-agent, which can be summoned in Slack to build a requested page using the published stylesheet and post a screenshot and URL back to the thread. Weekly, all feedback from Slack, GitHub reviews, and Figma gets consolidated, with repeated complaints automatically flagged as proposed changes for human approval. Vercel tracks how often each complaint recurs over time—if a fix doesn't reduce the count, it signals the correction needs more work.
The company frames the effort as evidence that agent instructions should be managed like software with a full development lifecycle, complete with version control, testing, and iteration. While better guidance doesn't guarantee agent reliability, the decline in recurring failures indicates that naming specific problems and encoding fixes into reusable files can make a measurable difference. Teams attempting similar approaches will need to balance the upfront investment of building and maintaining guidance files against the ongoing cost of manual corrections—a trade-off that becomes more favorable as the volume of agent-generated work scales. The broader lesson is that agents remain far from autonomous, but structured guardrails can shift the failure rate enough to make the technology more practical for production use.