I need some help with statistics. I want to give my students a
geography test where they will match the names of countries in list
with a map where the countries are labeled by numbers. I will test
them on 50 countries in Africa.

My question for you: What is the average score a student would obtain
through guessing alone? Let's assume that he/she matches one country
on the map to one name in the list with no duplicates.

Please show your work.  = )

Thanks. (PS I tip well for great answers.)

---

Request for Question Clarification by mvguy-ga on 31 Jan 2006 08:33 PST

The answer is 1.

I can't explain this in a mathematically rigorous way, but I hope I
can make it so you can see how I arrived at the answer.

Let's say there are just two countries on your map. I've made a chart
below where each letter represents a country to show the various
choices that the students can make. In the second column is the number
of correct answers.

A1
B2     2

A2
B1     0

(In these examples, a correct answer is A1, B2, C3, etc.)

As you can see, the number of possible answers the students can choose
is 2. The total number of correct answers among all possibilities is
2, making the average score of 1.

Now let's do the same thing with three countries:

A1
B2      3
C3

A1
C2      1
B3

B1
C2      0
A3

B1
A2      1
C3

C1
A2       0
B3

C1
B2       1
A3

Here we have 6 possible combinations, or 3!, and the number of correct
answers is also 6. Divide the number of combinations by the total of
correct answers, and we end up with 6.

Now let's try it with four countries:

A1      4
B2
C3
D4

A1      2   
B2
D3
C4

A1      2
C2
B3
D4

A1     1    
C2
D3
B4

A1     1
D2
B3
C4

A1     2
D2
C3
B4

B1     2   
A2
C3
D4

B1
A2     0
D3
C4

B1     1
C2
A3
D4

B1     0
C2
D3
A4

B1     0
D2
A3
C4

B1
D2     1
C3
A4

C1     1
A2
B3
D4

C1     0
A2
D3
B4

C1     2
B2   
A3
D4

C1     1
B2
D3
A4

C1     0
D2
A3
C4

C1     1
D2
C3
A4

D1     0
A2     
B3
C4

D1     1
A2
C3
B4

D1     1
B2
A3
C4

D1     1
B2
C3
A4

D1     0
C2
A3
B4

D1     0
C2
B3
A4


The number increases rapidly! Here we have 24 combinations with 24
correct answers, so the average score will be 1.

I hope you can see the pattern here. It will always be 1.

Another way of looking at it, is the point out that with each of the
50 choices, the student has a 1/50 chance of getting the correct
answer. Multiply 50 times 1/50 and you get a total average score of 1.

I don't have the skills to provide a mathematical proof beyond what I
have provided here. If this is sufficient for you, please let me know,
and I will post this as an official answer.

If it isn't sufficient, just say so, and hopefully some other
Researcher (maybe a math teacher?) can come along and flesh out my
answer to your satisfiction.

Thanks!

mvguy-ga

---

Request for Question Clarification by sublime1-ga on 31 Jan 2006 10:47 PST

alwayscurious...

In reading my colleague mvguy-ga's well-reasoned response,
it struck me that you had asked what would be the average
"score" for your students, and "1" does not translate into
a percentile score, as such.

If we extend what's been posted, however, it's clear that
the average score, in percent of correct answers, will vary
with the number of countries.

In the case of 2 countries, only 2 answers are possible.
One would score 100%, and the other 0%, for an average of
50%.

In the case of 3 countries, only 6 answers are possible.
One would score 100%, three would score 33.3333%, and 
two would score 0%, giving a ratio 200%/6 answers, for
an average of 33.3333%.

It becomes clear that the pattern is that the average
score will be 100% divided by the number of countries,
so, with 4 countries, the average score would be 25%.

In other words, mvguy's "1" out of however many countries
there are.

Therefore, with 50 countries, the average score, based
on guessing, should be a mere 2%, or 1 out of 50.

sublime1-ga

---

Request for Question Clarification by czh-ga on 31 Jan 2006 11:08 PST

Hello alwayscurious-ga,

I'm wondering if this is an abstract problem or a real question about
an academic testing situation. If you're really asking about a
classroom test then I think the realities of the context will
influence the likely average score. Are you teaching a geography class
or something else? Are you located in an African country or somewhere
else? Are you teaching elementary students or prospective geography
Ph.D.s? Unless this is an abstract probability problem, knowledge
about the context of the question will have a lot to do with figuring
out a probable average score.

So, what is the real problem we're trying to solve?

~ czh ~

---

Clarification of Question by alwayscurious-ga on 31 Jan 2006 12:16 PST

Wow. I'm impressed with your interest in this question. Thanks.

Let me clarify some points. First, this is a -real- testing situation.
I will be giving a real test as described. My expectation is that many
students will not -know- any of the countries in Africa, but they'll
complete the test anyway. They'll be guessing on every one of them,
though. How many the guessing student like that likely to get right?
The answer is some mean score and there will be some variation because
some students be luckier guessers than others.  = )

I'm looking for an answer that uses more mathematics/statistics than
pure, reasoned logic. I was expecting someone to talk about "sampling
without replacement"* and and then show some formula that for any size
of matching test, the mean correct responses and standard deviation
can be calculated according to the formula.

*Note: I'm not sure this is the actual statistical
assumption/approach. I just use it as an example.

So, MVGUY, I like your reasoning, but I'm not sure if you're right. I
guess I am looking for a mathematical proof of sorts.

SUBLIME is on the right track by noticing the variance.

CZH: I'm dealing with 5th and 6th grade students in the US. I want to
be able to say--if it's true of course--"The average score of students
on the pretest was 4, which on a matching test with 50 items is within
the range of what you would expect to see if every student was
guessing."

Thanks for all of your efforts.

---

Request for Question Clarification by sublime1-ga on 31 Jan 2006 12:56 PST

alwayscurious...

Given your choice of phraseology:

"The average score of students on the pretest was 4, which on
 a matching test with 50 items is within the range of what you
 would expect to see if every student was guessing."

...it seems to me that mvguy's original choice of wording was
correct, and you can expect to see an -average- score of 1 out
of 50 correctly answered. That's based on the assumption that
his reasoning is correct, even if it's not mathematically 
formulized, and it does seem correct to me.

---

Request for Question Clarification by mvguy-ga on 31 Jan 2006 18:19 PST

The answer of 1 just doesn't seem very intuitive, does it? It seems
like the average would be higher. Here's another way to look at it:

For the first question, there's a 1/50 chance the student will get the
correct answer. So the average number of points the student will pick
up from the first question is 1/50.

For the second question, the chance the student will get the correct
answer is 1/49 (49 is the number of answers the student can choose
from) times the chance that the right answer is still remaining, or
49/50 (because there's a 1/50 chance that the correct answer has
already been taken).

In other words the chance the student gets question 2 right is 1/49
times 49/50, which is 1/50. So the average number of points the
student gets from the second question is 1/50.

For the third question, the chance the student will get the correct
answer is 1/48 times 48/50 (since there's a 2/50 chance the right
answer has been taken). Again, then, the chance of getting the right
answer is 1/50.

And on it goes. Since the average number of points an average student
will get from each question is 1/50, the overall average is 50/50 or 1
correct answer.

The interesting thing about this, as one of the commenters has already
suggested, is that the average will be 1 correct answer regardless of
how many questions are on the test.

Does this explain things?

---

Clarification of Question by alwayscurious-ga on 01 Feb 2006 05:44 PST

It looks like the answer is in... Thank you mvguy, sublime, czh,
rracecarr, and ansel001 for your efforts--rracecarr's answer is what I
was looking for.

By the way, I found a presentation online that addresses my question,
and provides the correct answer, but doesn't offer the mathematical
support--pay particular note to the graph on page 15.

www.ipmaac.org/conf05/salyards.pdf

As an experiment, I created an Excel spreadsheet where I listed the
numbers 1 through 50 in column one and in column two. Then I generated
50 random numbers to fill the third column using Excels RAND function.
I sorted the second column and third by the third column--essentially
to randomize the second column. Finally I compared the first and
second columns to see how many matches there were. Sure enough, the
distribution accurately characterized the number of matches.

---

Request for Question Clarification by mvguy-ga on 01 Feb 2006 06:13 PST

Rracecarr-ga isn't a Google Researcher, so this one's a freebie for you.

I have no proof or rigorous analysis to offer, but...

For a sufficiently large number (M) of questions, the probablility of
N correct responses (where N<<M) is

exp(-1)/N!

exp(-1) is 1/e = 0.367879441171...

So, the probabilities are as follows

# correct    Probability
0            0.3678794
1            0.3678794
2            0.1839397
3            0.0613132
4            0.0153283
5            0.0030657
6            0.0005109
7            0.0000730
8            0.0000091

In a class of 30 students, all guessing randomly, you'd expect a
couple to get 3 right, 5 or so to get 2 right, and the rest of the
class split between 1 right and none right.

The fascinating thing is that these probablilities don't depend on the
number of questions, N (as long as it's large enough).  All the
decimal places shown here (and many more) are correct whether N is 25,
50, or 100.

Rracecarr is correct.  This is a matching without replacement problem.
 As an example, if n married couples are at a party and the men and
women are randomly paired for a dance, how many couples would be
expected to be matched with their partners?  As the number of
questions/couples increases, the distribution approximates a Poisson
distribution with mean of one.  What Rracecarr didn't say is that the
expected number of correct answers implied by his probablities is one.

Mvguy, your answer of one is correct but I would differ a little on
your reasoning.  Because of the matching, for every question that is
answered wrong, there is also another question that cannot be answered
correctly.  Similarly, if one question is answered right, the number
of choices for the remaining questions decreases by one.

I might add one more thing.  The convergence to the Poisson
approximation is rapid.  By the time you get to about ten questions,
the probabilities rracecarr shows won't differ significantly from what
they would be for a million questions.

rracecarr:

Please submit your comment as an answer.

First of all, my first language is French, so I apologize for any confused term.  

This problem is quit simple and can be resume in those words:  the
student have 50 tasks to do.  In every step, he (she) has a certain
chance of success.  The student is successful at every single step? by
guessing!
 
In the first step, chances are 1/50 and the student is right!
In the second step, chances are 1/49, the student in right? and so on.

Applying the multiplicative rule of probabilities we (simply) find:

Probability of success for all countries (P) = 1/50 x 1/49 x 1/48?

P = 1/(N) x 1/(N-1) x 1/(N-2) ? where N = number of items (50)

P = 1/N!

The average score a student would obtain through guessing alone in
this kind of test with 50 items is equal to one chance on the number
of permutations possible between 50 elements, say 50 factorial (!).

50! = 50 x 49 x 48 ? = 3.04E64

The answer is:  3.29E-65

In other words, I would recommend revising before the exam.

The answers (with a "s") provide by mvguy-ga was TOTALLY wrong (what
does this guy doing around here?).  Unfortunately, the answer accept
by alwayscurious-ga was approximative, but also wrong on a
mathematical point of view.

Regards

I ran a Monte Carlo simulation with 100000 samples and 50 questions and got:
Number   Percentage
Correct
 0       0.3705
 1       0.3652
 2       0.1832
 3       0.0624
 4       0.0150
 5       0.0030
 6       0.0006
 7       0.0001

which compares well with rracecarr.

The mean number correct was 1.00 with a standard deviation of 1.00 
This is reliable to at least 2 significant digits.

Now, the reason that this is so is because for n=50 questions, Each
question has a 1/n chance of being correct, and after n guesses it is
on average 1 correct answer.

I also considered what would happen if a student was guessing on only
40 or 30 questions. This was run with only 10000 samples(its a little
noisier).

Number of     Number    Standard
Questions     Correct   Deviation
Guessed
    1.0000    1.0000         0
    2.0000    1.0008    1.0000
    3.0000    0.9839    0.9894
    4.0000    1.0098    1.0103
    5.0000    0.9691    0.9807
    6.0000    0.9965    1.0086
    7.0000    0.9878    0.9917
    8.0000    1.0171    1.0008
    9.0000    0.9855    0.9757
   10.0000    1.0083    1.0049
   11.0000    1.0150    1.0085
   12.0000    1.0060    0.9908
   13.0000    1.0030    1.0043
   14.0000    0.9965    0.9970
   15.0000    1.0147    1.0138
   16.0000    1.0049    1.0007
   17.0000    1.0064    1.0051
   18.0000    1.0018    0.9963
   19.0000    0.9963    1.0035
   20.0000    1.0026    1.0032
   21.0000    0.9959    0.9914
   22.0000    0.9843    0.9955
   23.0000    1.0136    1.0264
   24.0000    0.9944    0.9946
   25.0000    1.0052    0.9970
   26.0000    0.9933    0.9947
   27.0000    1.0122    1.0172
   28.0000    0.9969    1.0066
   29.0000    1.0043    1.0065
   30.0000    0.9916    0.9963
   31.0000    0.9865    0.9989
   32.0000    1.0122    1.0007
   33.0000    0.9949    1.0021
   34.0000    0.9952    0.9950
   35.0000    0.9934    0.9899
   36.0000    0.9994    0.9991
   37.0000    1.0047    1.0055
   38.0000    1.0113    1.0151
   39.0000    0.9852    0.9877
   40.0000    1.0130    1.0039
   41.0000    0.9892    0.9919
   42.0000    0.9916    0.9888
   43.0000    1.0067    1.0058
   44.0000    1.0091    1.0120
   45.0000    0.9859    0.9898
   46.0000    0.9974    0.9937
   47.0000    1.0196    1.0076
   48.0000    1.0133    1.0090
   49.0000    0.9913    0.9935
   50.0000    0.9888    1.0031

The uninteresting thing is that the number right is 1.
The interesting thing is that the standard deviation is also 1. ?!?!?!?
I checked it by hand up to n=3.

alwayscurious--

Glad you liked the comment.  As mvguy says, I'm not a researcher.

Not sure if this is helpful in your teaching or not, but for what it's
worth, I *highly* recommend a rather addictive game called "10 Days in
Africa." If you are inclined at all to help reinforce what you teach
using games, this one is a winner. In just a few rounds, you already
start learning/memorizing African geography quite well. And it's a lot
of fun, too!

The game is made by a great game company called "Out of the Box
Publishing" (the creators of the wildly-popular "Apples to Apples"
game). They also have "10 Days in the USA," "10 Days in Europe," and
from what I understand, they're in the works with "10 Days in Asia."

http://www.otb-games.com/africa/index.html

Sincerely,
Boquinha-ga

As ansel001 pointed out, in the limit of an infinite number of
questions, the distribution in question converges to the Poisson
distribution with a mean of 1:  The probability of obtaining C correct
matches is given by P(C) = exp(-1)/C!

For a finite number of questions, N, the formula for the distribution
is given exactly by:

P(C) = 1/(C!)* SUM{from j = 0 to (N-C)} of [(-1)^j/j!]

See the page at http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html for
hints on how to get this result, and also for a nice online java
applet that lets you calculate and display the distribution for
various choices of N.