Responding to the Quora conversation we covered last week, Gartner Research Director Lydia Leong published a blog post detailing the process of creating a Magic Quadrant report. " We lay this process out formally in the initiation letters sent to vendors when we start MQ research, so I'm not giving away any secrets here, just exposing a process that's probably not well known to people outside of analyst relations roles," she wrote.
Here's a summary of the process:
- Step 1: Define a market and inclusion criteria.
- Step 2: Get approval from chief analysts.
- Step 3: Decide evaluation criteria and weights.
- Step 4: Send the evaluation criteria and weights to vendors.
- Step 5: Do hour long briefings with vendors.
- Step 6: Contact three to five reference customers provided by the vendor (however, Gartner mostly relies on the experiences of its clients).
- Step 7: Enter the numeric scores into the tool that generates the Magic Quadrant graph.
- Step 8: Write-up all the text.
- Step 9: Peer review.
- Step 10: Alter text and numeric scores accordingly.
- Step 11: Send the vendors a copy of the graphic and text written about them for fact-checking.
- Step 12: Finally, it goes on to editing and review.
The whole process generally takes four months, and Leong went into quite a bit more detail in her post.
And here's a particularly relevant bit:
Client status and whatnot doesn't make any difference whatsoever on the MQ. (Gartner gets 80% of its revenue from IT buyers who rely on us to be neutral evaluators. Nothing a vendor could pay us would ever be worth risking that revenue stream.)
Hopefully this will help make the Magic Quadrants less mysterious and more understandable.