Rare Tomato lets you review reported activity from multiple AI assistants, turn what they got wrong into rules shared across those assistants, and track how each follows your guidance over time.
My role
Product direction, testing, and iteration; AI-assisted implementation and design
Context
Independent, open-source project
Delivery
Built and deployed
Supplied interface designs from the project write-up. Example activity shown is illustrative. Click the image to enlarge.
01 / The problem
Make corrections useful beyond one conversation
I used several AI agents for different tasks. Preferences drifted between them, and correcting one agent did not give me a shared rule I could apply across the others. I wanted a way to teach agents how I work over time: turn a correction into a rule I approve once, make it available to connected agents, and review their reported activity against it.
The product’s value is the full feedback loop. Shared preferences give agents a common starting point; corrections and activity review let the user keep refining that guidance.
02 / My decisions
The choices behind the product
01
Make feedback actionable
An agent retrieves the rules it is allowed to see and reports its activity. A user reviews how that activity fits their expectations and turns a correction into a rule they approve. That rule becomes available across connected agents, so the feedback can inform future work. Activity review helps the user track adherence; it does not guarantee that an agent will obey every rule.
02
Test retrieval before assuming it works
I ran six prompts per agent and counted whether each agent called Rare Tomato without being asked. Rewording the server instructions did not improve Claude or ChatGPT retrieval. Adding one line in each agent’s own settings did.
03
Treat setup as part of the product
An outside technical tester understood the task but abandoned the attempt to connect one agent. Too many steps and too much movement between products made setup cumbersome. That observation moved onboarding simplification ahead of new features.
04
Keep responsibility visible
Claude helped with planning and review, Claude Code with implementation and deployment, and Claude Design with the interface. I remained responsible for the product decisions, testing, and what changed next.
Small experiment / Six prompts per agent
Where the instruction lived changed the result.
I counted whether agents called Rare Tomato on their own. Changing the server instructions did not help. Adding a line to the agents’ settings changed retrieval on the tested prompts.
Self-reported agent-call results from my project experiment
Agent
Without custom instruction
With instruction in agent settings
Claude
0 / 6
6 / 6
ChatGPT
0 / 6
6 / 6
Grok Bot
Called on its own; count not reported
Not reported
Muse
Connection errors; inconclusive
Not reported
This measured tool retrieval on a small prompt set. It did not establish rule compliance, reliability across tasks, or long-term behavior.
03 / Delivery
What I delivered
I built and deployed a working app with an MCP endpoint, scoped agent access, field-level encryption for sensitive text, and an access audit trail. I tested live integrations, permissions, authentication, and the setup flow. The automated suite reached nearly 1,000 tests across 69 files. These are implementation milestones, not proof that the experience works well for users.
04 / The open question
What I’d test next
The next iteration starts with simplifying setup. I want to observe whether people can connect an agent and reach their first useful correction without assistance, then broaden the behavior evaluations.
About the evidence
This case is based on my project write-up. The agent-call experiment was a small directional test, not a benchmark. Repository implementation has not been independently audited for this portfolio.