Guide and tips for how to create good and quality multiple choice questions in WOK
To create questions in WOKcraft is essential for the collaborative social platform WOK. And it's great fun!
Just like Wikipedia is made by the users of Wikipedia, The World of Knowledge; WOK is made by the users of WOK, the wokers.
In the ecosystem of the Knowledge Network, wokers play quiz games in mobile apps like WikiMaster, WikiFlip or Quiz King. (some countries does not have all)
5 easy technical steps to create a multiple choice question in WOK.
All questions in WOK is made in 5 steps to ensure quality in the context and framework:
1. Create a question with 1 correct and 3 incorrect alternatives
2. Confirm the question so everything looks good
3. Categorize for the main category with the implication of WOKbits overview, Quiz King round choices and WikiFlip Home Settings.
4. Add tags, i.e. keywords that are correct Wikipedia articles to make sure the question ends up in quizzes in relevant articles.
5. Add image to make the visual wrapping of the question to be great to view in all WOK apps.
To technically know how to create is knowledge without the art of quality in content is one thing. That was the 5 steps above.
To make GOOD questions you also need to understand what differ quality from its opposite and follow the practice of the following rules and guidelines (see also the Extended version of Content Manager Guidelines):
As a community of knowledge fans: We depend on all jointly efforts. We together are the wokers. Not we and them= Us. All wokers will benefit from the quality input of the questions made by the WOKers. Any topic to learn will be beneficial if there is good question to validate and certify that the knowledge has stayed in the head. We wokers see it like an article on Wikipedia is a textbook without control questions in the end of the chapter. It's so obvious it makes no sense not to have them. After download WikiMaster app: It is like Wikipedia without control questions is missing a core function in the learning process.
We as a WOK community try to change this glitch by providing the WOK API for Wikipedia as a plugin for all Wikipedia users, (still to be accepted and implemented by the Wikipedia community and Wikipedia Commons) so Wikipedia fans can enjoy the benefit of the WOK efforts around the globe without the apps from WOK directly on the Wikipedia. Until then: WOKers will have to download WikiMaster and enjoy this app as a great Wikipedia app as such. Or with all the questions related on top to take a quiz or play challenges.
For us at WOK Wikipedia is equal to a mobile phone and WOK equal to a smartphone. WOK take the Wikipedia to a new dimension of learning, fun, social media and visual interaction without leaving any of Wikipedia strengths behind. All Wikipedia is still in the WOK: From all apps. Everything in WOK connects to our 15 years older cousin Wikipedia. When Wikipedia turned 16, WOK turned 1.
Creating a question in WOK can anyone do. Just click on a Wiki article in WikiMaster and click Create Question. Or click on Create Question in Quiz King menu (Create is not yet a function in WikiFlip). After this is done: It is the appointed SuperWOKers who approve questions made by the WOKers in WikiMaster to be visible in the Quiz King rounds and in WikiFlip Flips. Until this is done; the question remains in WikiMaster as the input was sent.
This page is a guideline with tips on having to get good quality in the process of making Questions in WOK in general.
The advice is also great when you have alerted a badly written question and will have to correct it after 72 hours and ownership has turned to you.
Only one person : the owner can correct a question apart from the SuperWOKer.
Enjoy the art of question creation!
You find Create Question as an option in
a. any of the millions of Wikiarticles in WikiMaster in the lower right green button.
b. Footer menu when unfold for More in WikiMaster
c. The main menu in Quiz King
1. Check facts for the correct answer
When you create a question: Use fact control. Dont just think you know the correct answer: Check it. We recommend to use Wikipedia inside the creation process. In Quiz King and WikiMaster app you can click as in Wikipedia for WOK to get access to Wikipedia while you create a question to be able to write and check at the same time, even at the smallest screen in your smartphone device. Use a "really" correct answer and ensure that the wokers can review the question after they have answerered and check the facts.
2. Use truly incorrect alternatives and one truly correct answer
In WOK you create a multiple chioce question with four (4) alternatives of which only one answer (1) in the question is correct. This means three (3) alternatives shall be incorrect. Not incompletely correct, but truly incorrect with no doubt. To make a question interesting: Make sure the possibility for people who does not know the correct answer believe its correct. Which is an art that need some practice before you master it. But better try and learn than hesitate. Creating good questions is great for learning a subject yourself. Use the possible answer one would guess to be right, but is wrong so the question become interesting.
Spell the names of the alternative correct and use if possible real names. Even if they are wrong, the alternatives for incorrect shall be treated by the creator as the correct alternatives. Example 2:1
All the above is "most correct", but all "Incorrect alternatives" is in this example correct. This is confusing and SHALL be avoided.
All of the above is an alternative that is "most correct in this case to. In WOK it makes a confusing question. Also if you leave a question "hanging", use "_____" as a "to be filled space". In this case question mark comes strictly after who.
3. Dont use NOT in the questions
Questions with the correct alternativ as being "excluded" is not to prefer and will not be upgraded to be used in Quiz King app (class A). However: Questions of this type have some academic value and we have allowed them in WikiMaster : They are classified as E-types. Reason is many: NOT question focus on the negative. Its like playing golf. Think of "the flag" and "not water". A brain is unable to think in not-terms and interpret the command as "water" and you hit the ball out of bounds subconsiusly since the brain has been instructed to "water" instead of "flag".
Another example of class E iexample is "Put the following in order. Start with oldest/first/ etc. This gives one clear answer and there incorrect, but since the question in WOK is "incomplete" in sense: "Put the next 3 in order after put the first", this questions shall also be avoided. We have left some of these questions in the class E category since they have some educational value and many times are of good quality as such.
In Quiz King, "NOT" questions will not be found. In WikiMaster app, the NOT questions will be allowed, but to be avoided. They will have a class E for being visible in WikiMaster but not upgraded to class A (accepted to be a part of the Quiz King rounds). (see also 2. Only correct alternatives)
4. Real names and correct brands
Use real names and real brands carefully, so the question will not look sloppy. Even if you speak and understand, the written language has to be more accurate and avoid nicknames.
Example 4:1. Who created photoshop ?
- > Keep to the real names and not nicknames: Who created Adobe Photoshop ?
Galileo played a pivotal role in which revolution? - Keep to the real names in full - > Change to: Galileo Galilei played a pivotal role in which revolution?
5. Be clear
In combination of using the real brands and no nicknames: Use the full name to make sure a question can be free from any association
Example 1. Who scored the winning goal in World Cup 2014?
Which World Cup do you refer to? There where many World Cup in many sports taking place in 2014
- > Who scored the winning goal in FIFA World Cup 2014?
Example 2: In what suburb was Washington born ? Which person is referred to? I In a quiz called "George Washington Biography" it makes sense to refer to only Washington , but all questions in WOk shall be independent its context so it can be picked randomly for Quiz King in a random quiz question can referred to many famous Washington. Is it singer Diana Washington or the first president? The questions have to be clear. And work over time. Suggested solution:
In what suburb (of city X) was the first president of United States George Washington born in the 18th century?
6. Be consistent
Throughout the question, be consistent.
Example 6:1. Who scored the winning goal in FIFA World Cup 2014?
Mario Götze, Klose, Messi, Luiz
Make it full names or only Surname to be consistent:
Who scored the winning goal in FIFA World Cup 2014?
Götze, Klose, Messi, Luiz
Who scored the winning goal in FIFA World Cup 2014?
Mario Götze, Miroslav Klose, Leonard Messi, David Luiz
Who created Adobe Photoshop?
Brothers Tomas Knoll and Johan Knoll, Brothers Wright, Brothers Lumière, Brothers Goldberg
Keep constant: And remove the brothers First name or add all brothers in the incorrect alternative names - >
Who created Adobe Photoshop?
Brothers Knoll, Brothers Wright, Brothers Lumière, Brothers Goldberg
So keep it together within the question.
7. Be Short and Get straight to the point and keep as short as possible
Short is better. All questions in WOK shall aim to be as clear as possible and use as few characters as possible. It applies for the questions and especially for Quiz King where the limit is set for 105 characters for the question and 50 for the answers (each answer, correct and incorrect have 50 characters limit each. This does not mean we have set a limit to create longer questions. Some complicated issues and academic accuracy is demanding longer questions. This means that characters maximum and alternatives will be limited up to 50 and maximum around 60 characters for the question to be an easy read and easily understood and therefore used in Quiz King.
Longer questions is allowed. Longer questions still valid and appropriate will stay and be used in WikiMaster. The aim to be short applies in general for World of Knowledge. So some questions are good but long and can therefore not be in Quiz King but in WikiMaster. And some is short and good and can be in both WikiMaster and Quiz King. Long questions with images are allowed in WikiFlip. Class A, B and E)
Two sentence question is to be avoided. But at certain times and at academic levels they are in use to the better. To be avoided if possible. (See 7a. Short is better)
Edith Berg was (possibly) the first woman in a plane. Therese Peltier was (possibly) the first woman to fly one. When?
1928, 1908, 1958, 1888
Edith Berg has nothing to do with the year Therese Peltier was flying as a first time passenger 1908.
So an improved quality would be: - > Therese Peltier is popularly believed to have been the first ever woman passenger. In what year?
Felix Baumgartner, whose record-breaking leap from space has been viewed over 30 million times on YouTube, is from which country?
Switzerland, Belgium, Austria, Germany
The 30 million view part of the question is not necessary for the question so it could be skipped to shorten, but it is a finetuning art to create an interesting question. Many would argue that it could stay.
This movie is considered a landmark film, breaking many taboos when it came to portraying sex and violence. After amassing 10 Oscar nominations in 1967, this film ended up winning just two, for Best Cinematography and Best Supporting Actress.
Bonnie and Clyde, Butch Cassidy and the Sundance Kid, Easy Rider, The Graduate
The info about the 10 nominations is somewhat interesting and this info is best put into the field: "Did you know", which will be displayed as extra info for wokWOKer after he has take the question and can review it and enjoy the extra information. Point is: The question is far to long for WOK and Quiz King format if extra info is put into the question. To edit the question and improve it: Get straight to the point and remake it / shorten it like this:
In 1967, which film got Oscar for Best Cinematography and Best Supporting Actress?
Keep it to one sentence. Albeit it might be interesting ; avoid two sentence with a question in one sentence. In this case, WOKer could have created two questions and avoided the problem like this: a) In which country is the Arenal Volcano located ?
b) When did Arenal Volcano erupted for the first time in Centuries ?
Correct: 1968 Incorrect: 1962, 1961, 1974
The Zika question is good and relevant but to long for Quiz King since it pass 105 character: It has 113 which show on the right. It will stay and be used in WikiMaster but not appear in Quiz King unless its shortened.
For SuperWOKers who upgrade questions: the limits with 105 caracters for Quiz King can be bend with a few characters, but not many. Depending on quality. (A really good question with 107 characters can be overruled and upgraded to Quiz King, but 105 character is set as a standard rule)
8. Correct language
Goes without saying. Bad spelling is bad spelling. Bad grammar is bad grammar. What looks good is good. Keep it to simple and plain language. Type it in Google and see if the hits are correct. Bad spelling does not belong anywere except in trash cans. Today we have red dots under the incorrect words and right click suggestion everywhere; what's your excuse?
Example. Who created Adobe Photoshop?
The Knoll Brothers, The Wright Brothers, The Lumière Brothers, The Goldberg Brothers
No "who created photoshop? " "knoll brothers" etc. No "Who won world cup 2016?": What World Cup are we talking about? Sloppy. Sloppiness. Not part of a quality system
9. Boring questions
Avoid boring and dull questions. Keep the questions as intresting as you can. And writing multiple choice questions in WOK can be a huge advantage to your own learning. So if you study something for a test or exam in school; You can use WOK as a tool for learning. By Flip questions in WikiFlip or by going to a Wikipedia article in WikiMaster and write questions based on this topic is the easiest way.
Obvious to easy question is also boring. See 13
10. Write questions that is formulated to be correct for all time
Questions that are written with words like "recent", "last" "coming soon" , "current" "new" "latest" "currently" will be obsolete or incorrect over time. Sooner than you think! Write the questions in a manner that they will be used 10-20 years from now. And still be a question that is correct. Use correct year or time.
Which of these Windows versions did Microsoft release most recently?
XP, 98, Vista, 7
If this queston is written in time for relaese of Windows 7 , which was in 2009, the question is not a good question a couple of years later. Obviusly.
Which of these Windows versions did Microsoft release in 2009?
XP, 98, Vista, 7
What is the latest album released by artist XX ? What album was released in "month, year" by artist XX ? - > "Latest" will here be defined by a timestamp. Always true. Even after next album is released.
How many times did Rafael Nadal won a Grand Slam? - > Up to July 2014, How many Grand Slams have Rafael Nadal won ? (If Rafa won more after the question was written, the answer will still be correct over time)
Who is Hollywood's highest paid actor?
Will be a bad question over time: Improve this by clearify with year and if possible a source; : "Who is Hollywood's highest paid actor 2014 according to Forbes? "
And if possible; Back up with a learning link like: http://www.forbes.com/pictures/mfl45ekhem/robert-downey-jr-20/
11. Keep the question to the facts and not to opinions
11a. Source and Verifiability
A question in WOK is based on facts that can not be questioned. Undisputable facts.
This question needs a reference like "According to XX in "time and publication" or the question "What have women said is their number one electrical appliance?" is a bad question in need for an Alert. Apart from opinion might change over time, it can very well be different in different countries etc.
What have women said is their number one electrical appliance?
Microwave, Kettle, Straighteners, Hairdryer
11b. Neutral point of view
Opinions expressed in a question like "What is the true son of God?" or "What is the best dress to wear in the fall? " is not an undisputable fact and hence bad questions in WOK. It can be very different depending on opinions. But "What is the best dress to wear in the fall according to Karl Lagerfeld interview in New York Times 7 Sept 2014?" is based on a fact that Karl Lagerfeld had an open opinion of it and this had some relevance and can be checked had it been such an interview with him. "According to Christians, what is the true son of God?" can be stated as a fact in the context of beliefs among Christians if Jesus is put into the correct answer.
11c. Verifiability and Reliability
The answer to a question does not need to be in a Wikipedia article. But it shall have some reference that can be checked from a reliable source. Even if most likely at some Wikipedia article, the correct answer to most questions in WOK can be found. A question is put in a context by being added to many articles. Some grand. Some granular. It is obvious that "Who won the Purple Cup in IPL 2017?" has a relevance in India article, but the answer is not to be found in the India article. But in the article List of Indian Premier League Awards, the correct answer "Bhuvneshwar Kumar" is to be found
12. Fill in questions
You can make questions without a questionmark. Like fill in questions. Example: "The Sims is a______. "
Leave _____ and a dot for the sentence so its easy to fill in the blank by one of the alternatives. It does not need the right amount of underscore (_) but a few to mark is good.
Example 12:1. Fill-in question.
The Sims is a_____.
strategic life simulation game, match three puzzle game, racing video game, first person shooter game
13. To easy alternatives
A good question when it comes to multiple choice is one that is not to obvious. Of course sometimes its good to have easy questions, but to stimulate the brain, you have to have some challenge. If the three incorrect alternatives all are obviously wrong, you would not need to know the correct answer on a question. You could just eliminate the incorrect to get the right answer even if you had no clue on the real answer if given no alternative to chose from. Spend some time in making the incorrect answers "as if they could be right" .
Example 13:1. Easy wrong answers
The correct answer is very easy to find out since the alternative are so obvious wrong
Ex. In what country is Paris capital?
A. Gnopedia, B. Groucho. C. France, D. Uzbekistan
Who is Hollywood's highest paid actor 2014 according to Forbesr?
Robert Downey Jr, Robert Davis Jr, Robert Dawson Jr, Robert Daniels Jr.
It would be highly unlikely if you did not know the name of the most well paid actor but if the answers would be George Clooney, Brad Pitt and Angelie Jolie, the question would be much better since it would not be obvious.
Example 13:2. Very close answers / spelling.
Ex. What city hosted Kumbah Mela festival in 2013?
A. Allahabad, B. Allah Badh, C. Alahbad, D. Ellahbad
Even not know about the festival that host every 12 years (main focus), all you need to know is the correct spelling of the city Allahabad, India
Example 13:3. Two close/ Two distant Two alternatives makes a pair of difficulty
Ex. Which part of Cairo do you find the Pyramids?
A. Giza, B. Mohandeseen, C. Leblon, D. Ibapunena
The two first is parts of Cairo, the later two is part of Rio de Janeiro. So this would be a choice between A and B for most people.
Example 13:4. three close, one odd
Neil Armstrong was the first man on the moon. Who was the 2nd?
A. Patric O'Connor, B. Patrick Connor, C. Patric Connolly, D. Buzz Aldrin
We dont recommend make 3 close answers in belief one is correct and one odd in WOK albeit the odd is correct. But this is up to the creator to judge.
Wrong at first sight, correct if you take a closer look. It is no "best possible" answer in WOK. Either it's incorrect or it's correct. Half-correct is not an option. Lots of info on the alternatives that are correct but not completely correct making the question not perfect in sense of wrong and wright. So this is what the Swedes called a "slamkrypare". An error is hidden in the murk.
Which countries are neighbors to Lithuania?
A. Latvia, Belarus, B. Latvia, Belarus, Russia, C. Latvia, Belarus, Russia, Poland, D. Latvia, Belarus, Poland, Sweden
Definition of neighbor is to be cleared. In D Sweden is not a neighbor if you exclude water in between so D would be incorrect although some would alert the question for this reason since it is a neighbor over the Baltic Sea. A is still correct even though Poland is missing. Russia is not a neighbor although a territory of Kaliningrad is on Russian Rule. So even C is not the most correct for some users, this type of question will for WOKcraft users, the WOKers be avoided (!) and will go into limbo. Our suggestion is to make the answer correct "no questions about this fact" - correct and the wrong alternatives "no question it's wrong for any fact reason"- wrong. This would make it easier . A "E" alternative would state Lettland, Latvia, Poland and perhaps Kaliningrad and the other alternatives would include countries like German, Chech Republic and Estland, countries that for sure has no border to Lithuania.
"Which of these major Treaties occurred first?"
The intention from creator is right and there is unfortunaltly two treaties of Utrecht. Which makes the correct answer wrong since the first was in 1474, before the Treaty of Westphalia in 1648. The other Treaty of Utrecht was in 1713.
15. Images to questions
From mid-2015, World of Knowledge has included the possibility to add pictures related to the question. This process is made in WikiMaster as the 5th step in the Create Question step by step process or in Review Mode. And in Quiz King in Review mode. See a special article about adding images.
This pictures added to the question database in WOK are connected to Wikipedia or to be precise, they are from Wikimedia Commons on behalf of the Creative Commons license. Only these pictures can be added as part of the question making the copyright issue much simpler. That is: You can only add a picture that is related to Wikimedia Commons. (not upload any of your own).
For a creator perspective, this gives the woker an enhancement of making the question more fun and graphics. Up to late-November 2018, wokers have added over 178000 images to the growing database of questions in the WOK project of the total 445000 questions (!). Questions with images are more fun and very useful in WikiFlip app also.
It also gives the opportunity to make questions better or make it weaker if not being careful.
General speaking: The pic shall not reveal “the right answer”.
If the question is about “Where is the 2020 Summer Olympics held? , the pic chosen should not be the Japan Flag, but rather the Olympic Rings or the 2020 Olympic logo or the pet associated with the 2020 Olympic Games. Even a pic from the venue could provide some added value. But not the flag itself. In other words; It's okay to give a slight hint, but not write the answer in the face.
A question with images shall add some illustrative value to the question.
However; The question shall make sense without a picture for any reason.
Which flag is this?
Example of questions where the question does not make sense without the images is "Which flag is this? ".
In WikiMaster, the image will be shown, but in other apps, the picture might be lost due to bandwidth, the picture is deleted from Wikipedia at a future point etc.
So, The good question must be "Which flag is having a blue cross on a white background?" and in this case, the question is still valid even if the image does not exist.
Questions that are dependent on the existence of an image to be understood is classified as class E and SHALL be avoided. Cant be used in Quiz King app and in many cases not in WikiMaster either depending on context can differ in different Wikipedia articles.
More tips on how to Create Questions in WOK:
See more in
How to tag and make good keywords to questions in WOK. Tips and tricks for WOKers.
Adding images to WOK questions
How to make good multiple choice questions. Tips and tricks for WOKers.
Extended version on How to Create Questions in WOK with a guide for administrators and validation
How WOK works
FAQ for Quiz King app
FAQ for WOK
2019-01-09 last update by WOK Super admin