It is common that competitions have judges to watch over the fairness of competitions. A big issue with non-SCG competitions is that players design their programs for the current benchmarks. In SCG the benchmarks are dynamically generated by avatars and not known before the competition.
In SCG, each playground needs a judge to assist the SCG Playground Designer. The playground designer is interested in getting a computational problem solved. Instead of assigning employees to the task, an SCG playground is designed and a competition (a series of tournaments) is organized. The winning avatars will contain software that solves the computational problem. In SCG, the winner is determined by the game rules but there is a possibility for ties. In this case the SCG Playground Judge will choose the winner for the first price.
The SCG Playground Judge (judge for short) works in cooperation with the SCG Playground Designer. First the SCG Playground Designer provides the judge with a description of the computational problem to be solved. Second, after the playground has been configured and tested with baby avatars, the judge receives the playground definition. The judge checks the playground definition and compares it with the definition of the computational problem. Usually there are many ways to design a playground around a computational problem. The judge checks for a reasonable playground design. Third, the judge is informed about the results of the tournaments (smart histories, reputations of avatars and their ranking, knowledge base created by the game, i.e. the welfare set of claims) and checks them. The judge is looking for signs of cheating and for lack of fairness. Fourth, in case of ties for the first place, the judge will select the winner.
There is a separate Guide for SCG Playground Designers: Spring 2011 User Guide.