“Without getting into too much detail, I can tell you some of what Sun did to create their various Java certification exams. So, in the old Sun days…
The whole process to create a new exam takes a long time. A first team of experts is assembled and for several months the team develops a new set of objectives. Once created, the new set of objectives go through all sorts of vetting / verification processes.
Once the objectives are set, a second set of experts is assembled to create the actual questions. During this process, the experts all cross-check each other’s questions. In addition, all the questions are checked by psychometricians and experts in localization to make sure that they will be understandable on a world-wide basis.
Once all that has happened, the questions go to the “beta” phase that some of you guys have participated in. As you beta takers might recall, Sun asked you your expertise level as part of taking the beta. Once the beta tests have been given, the results are tabulated, and a lot of questions get thrown out - too easy, too hard, or too confusing or misleading being the most common reasons that a question will get thrown out. Sometimes the beta takers will also find an error in a question, and those get thrown out too. On interesting tidbit is that sometimes a question will be harder for “experts” than for “beginners”. This usually means the question is poorly worded, and, it gets thrown out.
After all that, a big batch o’ fun questions are left and when you go to a test center, you get to bang your head over some subset of them
As far as mock questions go, my guess is that the mock writers take the test at least once, study the objectives really hard, read existing study guides, take existing mocks, and go from there. "
Source: http://www.coderanch.com/t/522719/java-programmer-SCJP/certification/practice-exam-questions-created