How I Almost Ruined My AI Project by Trusting Feedback Too Much (and How I Saved It)
While working on version 1.3 of my AI editor, I discovered a dangerous pitfall: passive self-confirmation. And I almost ruined everything by blindly trusting the advice I received.
My goal wasn't just to write better posts, but to create a system that guaranteed absolute consistency in every interaction: tone, visual style, social rules, procedures. The prompt had to become the project's “constitution,” not just an instruction manual.
But in trying to improve it, I triggered a paradox: by following the AI's feedback, I degraded the quality of my own system.
The three-phase system: Discovery, Execution, Delivery. A clear advancement contract with mandatory human gates.
From Procedure to Consistency System
Before, the prompt was a linear sequence: “write this, then that.” Now it has become a finite state machine. I defined three phases (Discovery, Execution, Delivery) not to complicate things, but to create a clear advancement contract.
Why this structure? Because an AI, left free, naturally tends to be scattered. It expands the scope, skips critical steps, mixes different phases. Blocking the process into distinct states, with mandatory human gates (my explicit validation), means keeping control. Without me, the system doesn't advance. It's not a limitation; it's a safety measure.
I applied the same principle to the integrated Visual Identity: before, I had to load an external document for every session, risking errors. Now the palette, the flat vector style, and the robot mascot are part of the instructions themselves. The system is self-sufficient. But even here, there's a risk: automating without verifying can lead to visual rules that lose coherence with the original tone.
Explicit Roles: Eliminating Ambiguity
In a complex prompt, if you don't specify who does what, the AI tends to get confused. It assigns actions to itself when it should wait for me, or vice versa. I solved this by introducing explicit actors: [Lorenzo] (decides, validates, supplies) and [Lumo] (executes, proposes, waits).
Every single instruction in the new prompt states who executes it. This eliminated a lot of friction: Lumo knows exactly when to act and when to stop. No more “maybe,” no more “I think you mean.”
Peer Review: The Trap of Self-Confirmation
Here comes the most important part, and the hardest lesson.
I added a mandatory peer review phase: before generating the final Markdown, the draft is analyzed by two different AI models (Leo AI by Brave and Duck.ai by DuckDuckGo). The idea was good: get external viewpoints, avoid biases, improve quality.
And at first, it worked. The feedback was useful, pointing out inconsistencies and suggesting improvements. But after a few iterations, I noticed a subtle change.
I was starting to accept feedback without filtering it.
Without realizing it, I stopped critically evaluating the suggestions. I absorbed them passively, thinking they were “objective” because they came from another AI. The result? The quality of the prompt began to degrade. I lost focus, introduced redundant rules, and flattened the original tone.
The self-confirmation loop: accumulating feedback without filtering it makes the project move backward.
The turning point came when I compared version 1.2 (the original, before applying massive feedback) with version 1.4 (the “updated” one built by blindly following advice). I passed version 1.2 to Leo AI, asking him to evaluate it against 1.4, without telling him which was the “old” and which was the “new.”
Leo didn't know that 1.2 was the “old” version. To him, it was just text. And his verdict was shocking:
“Quick verdict: Yes, it's a significant improvement. The current version (let's call it v2.0) is more operational and self-referential than v1.4. Most of the critical points raised in the previous evaluation have been closed. Score: 9/10. No urgent patch needed.”
Leo was calling “v2.0” my old version 1.2, judging it clearly superior to the 1.4 I had built following his own feedback.
It was a decisive wake-up call, not a mathematical proof. It made me realize that by following advice without critical thinking, I had gone backward. I had worsened the prompt by accepting suggestions that, paradoxically, the AI itself recognized as inferior to the previous version.
The AI recognized the original version (v1.2) as superior, mistakenly calling it “v2.0”. This proves that uncritical application of feedback had degraded the project's quality.
The Lesson: Active Supervision, Not Passive Trust
This experience taught me a fundamental thing: we can trust AI, but only if we actively supervise it.
Peer review is a powerful tool, but it must be taken “with tongs.” It's not a supreme judge; it's a consultant. There's a crucial difference between using AI to find objective bugs (useful signal) and letting it decide the creative direction of the project (danger of flattening). If you stop judging its advice, the system degenerates. Quality doesn't rise; it falls.
Now, in the new prompt v1.5, I've encoded this lesson: – Peer review is mandatory, but feedback must be evaluated before being accepted. – I explicitly tell Lumo to distinguish between “useful observations” and “suggestions to reject.” – The final gate remains in my hands: no final Markdown without my explicit approval of the feedback.
What I Learned
- Rules must be executive, not suggestive. “Try not to invent” is useless. “If a detail is missing, stop and ask” works.
- Human gates are essential. Without my explicit validation, the process tends to slide forward on its own, skipping critical steps.
- Separate channels. Keeping drafts in chat and the final package on canvas avoids confusion and reduces the risk of losing context.
- Multiple peer reviews pay off, but only if filtered. Two different models, two different perspectives. But if you stop thinking, the comparison becomes background noise that degrades quality.
- The prompt is the constitution, not the manual. It must guarantee consistency in every session, not just in writing posts.
What's Next?
The prompt is now version 1.5. It's longer, more structured, stricter. But it's also more reliable. I know exactly what I'll get when I launch a seed, and I know exactly what I need to do to guide the process.
The most important lesson? Don't delegate critical thinking. AI is an extraordinary tool, but the final judgment must remain human. Otherwise, the system destroys itself.
Have you ever noticed yourself accepting AI suggestions just because they “sounded good”? Tell me in the comments what your “v1.4 moment” was.
The source code of the prompt and historical versions are available in the project repository: https://codeberg.org/itsmelarry/larrys-garage-md
“Free technology isn't just code — it's the freedom to look under the hood.”
👤 Lorenzo | ItsMeLarry all about me here
🧪 AI & Open Source Experiments | 🕹️ Retro Gaming & Graphic Novels 🎸 Rock, Folk, Jazz, Punk | 🔒 Privacy First 📍 Tuscany / Digital