Content Manager Guidelines for WOK question Creation Handbook

This is an extended guideline for the art of creating great questions in WOK (see a shorter version: Guidelines for question creation). It´s a general guideline to be used for teachers, administrators, and wokers who voluntarily like to contribute to creating the Worlds largest database of multiple choice questions on top of Wikipedia articles.

It´s also a guide to Wikimedians who feel they like to add value to the Wikipedia project by enhancing the academic value to all Wikipedia articles by questions for the students of the World using Wikipedia as the main tool for learning. 
It addresses more in detail of to make multiple choice questions (MCQ) more interesting and entertaining. And the validation process for the WOK administrators. 





WOKcraft or shortly WOK is aiming to be the worlds greatest, largest and most useful database of multiple choice questions. The project WOK aims to make knowledge an learning fun by turning it into games. Or quizzes.
To turn the gamification of learning into an educational revolution. 

As a member of WOKcraft (WOKer) you can create any question to any subject. To keep the quality of the questions at a high standard, the collaboration of WOKers is essential for the maintaining and developing of WOK.
As WOKer: You are a valuable part of contributing to the database. We all can enjoy the process of creating quality questions and as a member of the WOK community: The Joy of taking good questions and enhance the learning process.

First of all questions shall be

Be spelt correctly
Obey the rules of English grammar or if in any other language, this language grammar.
Be written according to the guidelines to WOK. The Short Guidelines and the more Granular below. 

HOWEVER, obeying these regulations alone is not sufficient to create a quality question. 
Each question must also be as easy to understand AND be as interesting and entertaining as possible.

If you consistently produce more questions than is required on a mission, or if you demonstrate consistent improvement relative to your previous performance, it will be noticed.
This is why feedback is so important.



Questions are entered into the database using 'Drama', a browser-based program that allows questions to be entered into specific categories and subjects, then verified.
The team is assigned a subject, which the Team Leader will then break down into smaller parts for each Questioner to research.  For a film subject, it may be broken down alphabetically, with one questioner researching A to E, the next F to J, and so on.  Alternatively, a history subject may be broken down into decades or centuries.

The team leader will suggest useful websites to start with, but each team member's research will produce more.  Once the Questioner has found a good-quality website that can be used to generate entertaining and accurate questions, they are entered into the database as follows:
The question is entered into the Question field.

All nouns and other 'misspellable' words should be copied and pasted from source.

  • The correct answer is copied into the Correct Answer field.
  • The URL, or web address, of the source website, is copied into the first URL field.
If the website is official (see Official Sites) move to step 6.

  • The Questioneer uses Google (or other sources) to find a second website to confirm that both the question and correct answer are accurate.
(see Web Research)

  • The URL of the second source website is copied into the second URL field.
  • Using the research, the SIX incorrect answers are selected and entered.  They do not need to be sourced but should be copied and pasted where possible.
(see Incorrect Answers)

  • The difficulty setting of the question is set.
(see Question Difficulty)

  • The Market(s) of the question is set.
(see Market)

  • The whole question, including sources and answers, is checked for spelling, grammatical or factual errors.
  • Save the question.

The question is sent to validation (see The Validation Process)

The validator will check the following:

·       the accuracy of the question by reviewing the source
·       the grammar
·       the spelling
·       the difficulty level
·       the general quality of the question
·       the Market
If there are any mistakes, the validator will correct, if possible. If it is not possible to correct a question, for any reason, the question will be discarded. The validator will record the creator of any discarded question.
The validator will provide the Team Leader with a complete list of all discarded questions with the names of the questioner responsible for creating the question.
All questions that pass the validation process will be upgraded to classes. 

Official Sites other than Wikipedia for input and fact checking
Usually, the input process requires a second source to confirm the accuracy of a website.  However, if the first source is 'official', then only one is required.
In most cases, it is obvious that a website is 'official'-

Regarding TV or Movies, often there are several 'official' sites, including internal TV network sites-

Though these sites do not follow the www.'title'.com style, they may still be official.

Although the above examples may not follow the www.'title'.com

If there is ANY doubt regarding the 'official' status of a website, you MUST use a second source.  In many cases a website may be professionally designed, but cannot be considered an official source-

Regarding Wikipedia, etc, the information on these websites is almost certainly accurate, but they are NOT 'official'.  They can still be used as a source, often it is very useful to do so, but due to the third-party nature of such sites, they cannot be official.  Also, because both Wikipedia and often retrieve their information from the same sources, they cannot BOTH be used as the two required sources.

There are many sites that may contain useful information or confirm a source but are either too unreliable to use as a source-

Or are simply too amateurish, and though this is a matter of discretion, it is also fairly obvious whether a website is the demented scrawl of a single madman or not-

We require questions of mixed levels of difficulty. Assessing and adjusting the difficulty of a question is mostly a matter of intuition.  There is no easy method to determine the inherent difficulty of any given question; it is a matter of phrasing, of how much information is given in the question and the variety of the answers.

The following factors are some of the methods by which difficulty can be established and adjusted.


Questions should NEVER be written in a more complicated way in order to make them more difficult.  They should always be as SIMPLE as possible.

However, the question can be made easier by including more information in the question.  Adding a phrase like 'In 1977' to a film question can give the player a clue about the era, who were the famous film stars of the time etc, so that even if the player hasn't seen the film, they have a better chance of answering it correctly.

Similarly, use of adjectives like 'sci-fi' or 'comedy' for films, or more precise terms like 'striker' instead of 'player' for sports, can clarify the question, and therefore make it easier.  Also, more factual information about the question's context can be added, such as the author of a novel.

e.g. In which 19th Century Russian novel does 'Konstantin Levin' appear
In which novel by Leo Tolstoy does 'Konstantin Levin' appear


In which novel does the character 'Konstantin Levin' appear

Incorrect Answers
Good  incorrect answers are as important as any other component of question creation.  They are also often the most difficult stage.  In addition to determining the quality of a question, the answers can help determine the difficulty.

A question with numerical answers can often be made easier by spacing the answers further apart -
e.g. 1985, 1785, 1945, 1845
20th Century, 19th Century, 18th Century, 17th Century

1985, 1986, 1987, 1988

A question with text-based answers can be made easier by either selecting correct answers from a broader pool, such as using all 19th novel titles as a pool instead of just Dickens' titles -
e.g. Great Expectations, Middlemarch, Vanity Fair, Jude the Obscure

Great Expectations, Little Dorrit, The Pickwick Papers, A Christmas Carol

Or using variations on the correct answer -
e.g. Great Expectations, Great Celebrations, Great Experiences, Great Catastrophes

HOWEVER, using variations can very often make the question too easy, which can be more damaging to the quality of the play-experience than if questions were too hard.  Easy questions are often harder to write than hard questions for this reason; the difficulty must be easy, but NOT too easy.

e.g. Johnny Depp, Jamie Depp, Tommy Depp, Gary Depp
Johnny Depp, Jahnny Dopp, Jihnny Dipp, Juhnny Dupp

Simply, these tags determine the suitable 'markets' or geographical/cultural areas a question is suitable for.  Most questions are suitable for all markets, BUT there are exceptions, and assigning the correct markets to these questions is as VITAL to ensuring question quality as any other stage of the process.

e.g. A Music question about the Billboard charts is ONLY suitable for USA.

Likewise, a Music question about the UK charts is ONLY suitable for UK.

The correct market(s) for any exceptions should be obvious.  If you are in ANY doubt, ask your Team Leader and only assign the specific markets you are certain are correct.

3.1 WHO or WHOM?
Who should always be used in preference to Whom. This is NOT because Who is always right, but  because our questions should be phrased to avoid having to use Whom.
e.g. In 1793, who invented the cotton gin?
In 1793, the cotton gin was invented by whom?

Even though the latter example is also grammatically correct, Whom can be seen as pretentious.  There is no reason to alienate players when the question can be easily rewritten.  The following is incorrect on both counts -
e.g. NEVER In 1793, the cotton gin was invented by who

3.2 WHAT or WHICH?
What should usually only be used when the pool of possible answers is either infinite -

e.g. What is the number of loaves in a baker's dozen

unknown -
e.g. What was name of President Reagan's dog

or at least very large -
e.g. What colour is the ugli fruit

Which should usually be used when the pool of possible answers is limited.  As we offer 4 choices for each question, Which is almost always the right choice in a “what or which” debate.

e.g. Which character does actor Zach Braff play in 'Scrubs'

In these cases, Which can often be followed by 'of the following'.  It can unnecessarily lengthen a question, but it may also add clarity.
e.g. Which of the following characters, played by Zach Braff, appear in 'Scrubs'

In Which
For items than 'contain' the correct answer, such as centuries, years, TV shows, In which is almost always better than What.

e.g. In which century was explorer Christopher Columbus born
What century was Christopher Columbus born

In which TV sitcom did the character 'Frasier Crane' appear
Which TV sitcom featured the character 'Frasier Crane'
What TV show featured the character 'Frasier Crane'

The disadvantage of using In which is that it forces the verb associated with the preposition In to the end of the question; in the above example, 'appear'.  Using Which usually means the object of the sentence, i.e. the distinctive part of the question (in the above example 'Frasier Crane'), is at the very end of the question.  This focuses the player on the relevant part and should make the question slightly easier to understand.
3.3 IN or ON?
The distinction between these two prepositions is usually fairly obvious.  In indicates that one element is enclosed within another -
e.g. The city of Seattle is in which country
The city of Seattle is on which country

On, unsurprisingly enough, indicates that one element is positioned above another -
e.g. On which island did Captain James Cook die
In which island did Captain Cook die

Regarding question input, the 'in or on' debate usually depends either on the literal relations of two tangible elements -
e.g. On which street would you find 'Big Bird' and 'Elmo'
or on the divisibility of the object -
e.g. In Britain, On which day of the week does May Day lie

This is an inaccurate example, but if the object is small, or not usually broken into smaller parts (in the context of the question) then 'On' should be used -

e.g. On which island, On which day

However, if the object is large, or is considered a group name for a collection of elements usually separate, In should be used -
e.g. In which year was Christopher Columbus born 

In which continent is the city of Seattle located

Capital letters should always be used for the first letter of each question.  The first letter of a quote should usually also be capitalised.

The titles of books, films, TV shows etc, should have a capital letter for each noun, as should the first article, and only if it is the first word of the title, 'the', 'a' etc -

e.g. The Dirty Dozen, Bridge Over the River Kwai

Note that in the second example, 'the' is NOT capitalized.
The pronoun 'I' should always be capitalized.
Proper nouns

The name(s) of a person, TV show, city etc should always be capitalized -
e.g. David, The Good Life, New York

The names of months, days are also capitalized, as are national-group words -
e.g. March, Friday, Irish, American

Deities are capitalized.  However, 'god' referring to ancient religions, such as Greek or Roman, is NOT capitalized, nor is the plural 'gods' -
e.g. God, Allah, Lord

Common nouns
The names of objects, animals etc int the question text only are not capitalized -
e.g. hammerhead shark NOT Hammerhead Shark

If the adjective is based on a proper noun, it may have a capital letter.  This varies from word to word -
e.g. 'Shakespearean' is capitalized, 'quixotic' is not.

Usually, these are all capitalized -
e.g. BBC

Occasionally an acronym may have become a bonafide word, and will only have the first letter capitalized like a proper noun.  In these cases, either style is acceptable -
e.g. Unicef or UNICEF

An individual's title, such as Mr, Sir, King, President will always be capitalized if it appears before their name, or if it used to address them instead of a name -
e.g. Mother Teresa, Sir Walter Raleigh, Who sang 'Happy Birthday Mr President'

However, if the title appears elsewhere, it is not capitalized -
e.g. Who was the 42nd president of the United States of America

Brands are usually capitalized, such as Coca-Cola, but occasionally they are not -
e.g. eBay, iPod

The question input process rarely, if ever, requires a period or full stop at the end of a sentence.

Where appropriate, a period should be used at the end of a quote, inside the quotation marks- e.g. Who famously said 'God does not play dice with the universe?'

Periods or full stops should not be used in acronyms -
e.g. BBC not B.B.C.

In most cases, periods do not need to be used for titles -
e.g. Mr NOT Mr.

There will almost certainly be exceptions to this.

The comma should be used to separate the clauses of a question -
e.g. The character 'Frasier Crane', played by Kelsey Grammer, appeared in which sitcom
Often the clause that the commas surround, 'played by Kelsey Grammer', can be removed and the question will still make sense.
e.g. The character 'Frasier Crane' appeared in which sitcom?

In the above example, using 'played by' improves the quality of the question, but perhaps makes it slightly easier.
The comma should also be used to create a pause, particularly in a two-part question, or to separate a statement that clarifies the question, such as defining the release year of a film -

e.g. In 1977, who directed the movie 'Star Wars?
In 1977 who directed the movie 'Star Wars?

It can be helpful to read the question slowly and sense where a pause may be.
It is unlikely that you would use more than one comma if you were not including a clarification as above, 'played by Kelsey Grammer', but the situation may arise.
e.g.  In the Summer of 1969, if you were listening to The Who, Janis Joplin and Jimi Hendrix live, where would you be

As in the above example, commas should also be used to record lists.  This, however, will rarely be relevant for question input.

Always use 'text' NOT “text”.  The single ' is less messy and obtrusive than “ and requires less effort to type.
Quotation marks should usually only be used in the question, NOT in the answers.
Use quotation marks for titles of TV shows, films, art etc, and nicknames or character names -
e.g. 'The Good Life', 'John Dorian'
Never use them for real names of people, places etc -
e.g. Ted Danson NOT 'Ted Danson'
Ambiguous Usage
The name of a company does not need quotation marks, such as Ford, but their products may -
e.g. either 'Mondeo' or Mondeo

Which of these characters, played by Ted Danson, appeared in 'Cheers'
Which of these characters, played by 'Ted Danson', appeared in Cheers

On the TV show 'Scrubs', who plays 'John Dorian'
On the TV show Scrubs, who plays John Dorian

Quotes should always be recorded as they are found in a webpage, book, etc even if the spelling, grammar etc, contradicts either English rules or these guidelines -
e.g. 'We wuz robbed' should NOT be corrected to 'We were robbed'
As discussed elsewhere, quotes should be indicated by 'text' NOT “text”.

5.1 Who created/Who made
When writing questions about art, TV shows, movies, 'who created' NOT 'who made' should usually be used.
e.g. Who created the artwork titled 'The Birth of Venus'
e.g. Who made the artwork titled 'The Birth of Venus'

When writing questions about manufactured items, such as cars, 'who made' can be used, but 'who manufactured' or 'who built' would usually be better.
e.g. Who manufactured the 'Mondeo' model of car
e.g. Who made the 'Mondeo' model of car

5.2 Starred in
Starred in should only be used in reference to actors and NEVER characters -
e.g. Mark Hamill starred in which classic sci-fi film
Luke Skywalker starred in which classic sci-fi film
Regarding characters, 'appeared in' or 'featured in' should be used instead.

5.3 TV
In all cases, when referring to television, use capitalized 'TV' NOT lower-case 'tv'.

5.4 Art
Questions on this subject can be difficult to phrase.  As stated elsewhere, 'who created' should be used NOT 'who made' -
Who created the sculpture titled 'The Thinker'
Using 'the sculpture', 'the painting' etc can make the question easier and/or more interesting, but often using simply 'the artwork' or 'the work' will suffice -
e.g. Who created the artwork titled 'Starry Night'
When writing these art questions, use 'the artwork titled' rather than 'the artwork called' or simply 'the artwork'.  It seems a less awkward construction.
Use of terms like 'the portrait', 'the impressionist work' etc can make the question even more interesting, which is our goal, but they should only be used if the sources confirm that they are accurate.
e.g. Who painted the impressionist work titled 'La Femme à la Robe Verte'

5.5 Days, Months and Years
Months and days of the week should always be capitalized.
A precise date should be written in the form '25th May 1977'.
e.g. Which famous sci-fi film premièred on 25th May 1977?

In most cases, the year should be written in the full four-digit form. If the year follows a film, TV show etc, the year should be bracketed -
e.g. Which of these actors starred in 'The Rocketeer' (1991)
Which of these actors starred in 'The Rocketeer' (91)

If the two-digit form of a year is required, an apostrophe should precede the number -
e.g. 'Italia '90' was the title of which major sports event
Centuries should be written with 'Century' capitalized -
e.g. Which of these 20th Century conflicts was fought between China and Japan?

For the sake of argument, questions will usually only be written in either present simple/continuous or past historic/continuous tense.
The appropriate tense of a question is dependent on its subject.  In most cases, the right tense to use will be obvious.  It may even be irrelevant, as either past or present tense can be used.  One is usually better than the other -
e.g. 'The Thinker' was created by which artist
'The Thinker' is by which artist
For questions regarding -

- 'permanent' elements, such as locations, names, rules
- something that, once created, has a permanent existence, such as a painting or sculpture
- something that is fictional and exists 'outside' of normal time, such as a character in a TV show
the present tense should usually be used -

Geography -  What is the capital of Belgium?
Sport - How many substitutions may an NBA basketball team make during a game?
Art  - 'Campbell's Soup Can' is a work by which artist?
TV -  Which of the following characters appears in the sitcom 'Cheers'
Movies - The character 'Ellen Ripley' appears in which sci-fi film   

For questions regarding -
- past events, such as wars, births, deaths etc
- the act of creation of something permanent, such as 'was created by', 'was founded in'
- the cast of a film or tv show, such as 'who starred in'

the past tense should usually be used -

History - When was India granted independence from the British Empire?
Sport - Who scored the first goal in the 2014 World Cup Final?
Art  'Campbell's Soup Can' was created by which artist?
TV - Which of the following actors starred in the sitcom 'Cheers'
Movies -In the sci-fi film 'Alien' (1979), which famous actress played 'Ellen Ripley'

Note the difference in tense between a present tense character question, 'who appears in', and a past tense cast question, 'who starred in'.

These are only guidelines. 
Determining tense can be simply a matter of question construction, but as stated in the examples above, some subjects have a better, or more appropriate, tense to use.  Provided the grammar and typography is correct, tense is less important.  Above all, the question must easy to understand.

Log version
Version 1.1 Updated Nov 2, 2017/ EB
Version 1.0 Updated Dec 20, 2016/ EB

Tags: guideline, Create Questions