Build a fair test for any new AI model on your own work
› I want to decide whether a newly released AI model is worth switching to for my work. Its maker has published no benchmark scores I can use, so I need my own test.
My work: [describe what you use AI for, e.g. reviewing contracts, writing support replies, building a web app]
The model I use now: [current model]
Please:
1. Propose 5 test tasks drawn from my real work, from routine to hard, including at least one multi-step task where the model has to recover from its own mistake.
2. For each task, write the exact prompt I should paste into both models, word for word, so the comparison is fair.
3. Give me a scoring sheet with 3 to 5 criteria per task (accuracy, follow-through, how many corrections I had to make, time to a usable result), each scored 1 to 5, with what a 1 and a 5 look like.
4. List what I should not paste into a new model before its safety report and data terms are public, given my kind of work.
5. Tell me how to decide: what score gap would justify switching, and what would make me stay.
Ask me any questions you need first, then build the test.
Works in any AI chat. In Magai, run it against several models at once.