Are the questions really randomized

[BassPlayingSugarplum]

I noticed when I was doing my ATA analysis that the questions actually seemed to be weighted toward the easier ones upfront, and not randomized liked Cedric (or Meredith) said.

So I checked. I looked at the 59 most recent shows where an unrandomized topic tree was shown (for a total of 463 questions).

Average difficulty level by question #:

1. 3.05
2. 3.92
3. 5.44
4. 5.67
5. 6.34
6. 5.15
7. 6.19
8. 6.97
9. 6.37
10. 6.73


Anyone know whether this is just not enough questions for a random distribution or conscious editing by the producers?

Or are the questions not really random? And no, I haven't checked dollar values yet to see if they follow an unexpected random distribution. This stuff takes time.

My blog on the topic, if you're interested, is at http://jackiefoxdotnet.wordpress.com/20 ... questions/.

[MarleysGh0st]

Anyone know whether this is just not enough questions for a random distribution or conscious editing by the producers?

Or are the questions not really random? And no, I haven't checked dollar values yet to see if they follow an unexpected random distribution. This stuff takes time.

I don't know enough about statistics to tell if this is a large enough sample to be significant. I'm sure other BBs can weigh in here.

However, the Bored has discovered flaws in the game before. A few years ago, one of our BBs noticed that the correct answer letter would never be the same in sequential questions (i.e. if the last right answer was C, the next one wouldn't be). That information was enough to sometimes turn a 50/50 into a sure thing. He brought that to TPTB's attention, after which the correct answer did start repeating.

So, when they say that a part of the game is random, they don't really have any outside expert verifying that whatever algorithms they use actually produce what they promise.

Welcome to the Bored! Please keep crunching those numbers. We love reading about this stuff here. (Or at least we used to.)

[Bob78164]

Welcome aBored! I think we met briefly in Las Vegas. You're from Los Angeles, right?

I don't know off hand, but I think the numbers you've come up with are at least close to statistical significance. --Bob

[BassPlayingSugarplum]

Yes, I auditioned in Vegas, in the voluminous orange/red shirt and jeans. Dan's friend.

[Estonut]

Based on observation alone, it seems to me that 2 of the top 3 unrandomized questions rarely leave the top 4 after "randomization."

[Bob Juch]

Based on observation alone, it seems to me that 2 of the top 3 unrandomized questions rarely leave the top 4 after "randomization."

That's been my impression as well and BPSP's stats tend to prove it.

I wonder if saying they're randomizing the questions but not actually doing so is a violation of the consent agreement after the quiz show scandals?

[Bob78164]

Yes, I auditioned in Vegas, in the voluminous orange/red shirt and jeans. Dan's friend.

I remembered that you were Dan's friend, but a bunch of Dan's friends auditioned in Las Vegas so I wanted to be sure I had the right one. I think I said hello to you while we were waiting in line. --Bob

[ulysses5019]

Yes, I auditioned in Vegas, in the voluminous orange/red shirt and jeans. Dan's friend.

Who the heck is this "Dan" character?...but he does look a lot like uly...

[MarleysGh0st]

Who the heck is this "Dan" character?...but he does look a lot like uly...

Sounds like a troublemaker...

[Estonut]

Who the heck is this "Dan" character?...but he does look a lot like uly...

Sounds like a troublemaker...

I'd bet he's a bug-eater...

[SportsFan68]

Yes, I auditioned in Vegas, in the voluminous orange/red shirt and jeans. Dan's friend.

Who the heck is this "Dan" character?...but he does look a lot like uly...

Found it!

[goongas]

59 games is not statistically significant to draw any conclusions IMO.

[ulysses5019]

Who the heck is this "Dan" character?...but he does look a lot like uly...

Sounds like a troublemaker...

I'd bet he's a bug-eater...

You're tempting me...

[kayrharris]

Sounds like a troublemaker...

I'd bet he's a bug-eater...

You're tempting me...

Don't do it! Please..

kay

[ulysses5019]

I'd bet he's a bug-eater...

You're tempting me...

Don't do it! Please..

kay

YUMMMMMMM!!!!!

[ulysses5019]

I noticed when I was doing my ATA analysis that the questions actually seemed to be weighted toward the easier ones upfront, and not randomized liked Cedric (or Meredith) said.

So I checked. I looked at the 59 most recent shows where an unrandomized topic tree was shown (for a total of 463 questions).

Average difficulty level by question #:

1. 3.05
2. 3.92
3. 5.44
4. 5.67
5. 6.34
6. 5.15
7. 6.19
8. 6.97
9. 6.37
10. 6.73


Anyone know whether this is just not enough questions for a random distribution or conscious editing by the producers?

Or are the questions not really random? And no, I haven't checked dollar values yet to see if they follow an unexpected random distribution. This stuff takes time.

My blog on the topic, if you're interested, is at http://jackiefoxdotnet.wordpress.com/20 ... questions/.

I think that nerdy fox should be wearing a Star Trek uniform instead of the little sign...just sayin'

[smilergrogan]

I noticed when I was doing my ATA analysis that the questions actually seemed to be weighted toward the easier ones upfront, and not randomized liked Cedric (or Meredith) said.

So I checked. I looked at the 59 most recent shows where an unrandomized topic tree was shown (for a total of 463 questions).

Average difficulty level by question #:

1. 3.05
2. 3.92
3. 5.44
4. 5.67
5. 6.34
6. 5.15
7. 6.19
8. 6.97
9. 6.37
10. 6.73


Anyone know whether this is just not enough questions for a random distribution or conscious editing by the producers?

Or are the questions not really random? And no, I haven't checked dollar values yet to see if they follow an unexpected random distribution. This stuff takes time.

My blog on the topic, if you're interested, is at http://jackiefoxdotnet.wordpress.com/20 ... questions/.

It's almost certainly not random, to greater than 95% confidence level. The correlation coefficient (Pearson's r) for your 10 averaged data points above is 0.85, and for a 10 data point set, that has a less than 5% chance of happening if the data is actually uncorrelated. See the following chart:
http://en.wikipedia.org/wiki/File:Corre ... icance.svg
For the full 463 point data set, I'm guessing it would be similar if not higher (for the confidence level that the data is not random), but it would depend on the range of difficulty values for each question #.

Here's a link you might be interested in from last year regarding whether the $ values are random (based on $ values of jumped questions), and within that thread is a re-posted (much older) look I did at whether the 50/50 was random (and a weird glitch in the 50/50 answers that nobody seemed to discover until the syndicated show removed it after the first two years):
http://wwtbambored.com/viewtopic.php?f=1&t=42534

[jarnon]

Also, TPTB have stated repeatedly that the unrandomized topic tree, shown briefly to some contestants, is not meant to be in order of difficulty. A few counterexamples, where very easy questions appeared high in the unrandomized topic tree, were noted in last season's transcripts.

[littlebeast13]

You're tempting me...

Don't do it! Please..

kay

YUMMMMMMM!!!!!

Still the best ulytar ever....

lb13

[geoffil]

I agree. There seems to be a pattern to the randomness. Just like the 50/50. The most unlikely answer was always eliminated IMHO. I think they work within the lawyers definition of random.

Just like the order of how the contestants would play during taping. I found out after my taping that Geoffil husband knew I was not going to tape on the first day because the companions were sitting in order of when the loved one would tape. He was second to last. I was never told that. I sat on deck backstage the entire afternoon and never taped. I kept drinking water, checking hair and makeup and going to the bathroom for nuthin'.

[littlebeast13]

I noticed when I was doing my ATA analysis that the questions actually seemed to be weighted toward the easier ones upfront, and not randomized liked Cedric (or Meredith) said.

So I checked. I looked at the 59 most recent shows where an unrandomized topic tree was shown (for a total of 463 questions).

Average difficulty level by question #:

1. 3.05
2. 3.92
3. 5.44
4. 5.67
5. 6.34
6. 5.15
7. 6.19
8. 6.97
9. 6.37
10. 6.73


Anyone know whether this is just not enough questions for a random distribution or conscious editing by the producers?

Or are the questions not really random? And no, I haven't checked dollar values yet to see if they follow an unexpected random distribution. This stuff takes time.

My blog on the topic, if you're interested, is at http://jackiefoxdotnet.wordpress.com/20 ... questions/.

I think that nerdy fox should be wearing a Star Trek uniform instead of the little sign...just sayin'

Like this?

lb13

[jarnon]

I think that nerdy fox should be wearing a Star Trek uniform instead of the little sign...just sayin'

Like this?

lb13

Found it, Jim.

[jarnon]

I agree. There seems to be a pattern to the randomness. Just like the 50/50. The most unlikely answer was always eliminated IMHO.

In the early days of primetime Millionaire, it was obviously not random. It got more random in later seasons.

I think they work within the lawyers definition of random.

At the end of my audition interview last month, Samantha told me: "The selection process from this point on is totally random." I knew she wasn't using the mathematical definition of random. She meant that it was arbitrary or subjective, unlike the multiple-choice test.

[SportsFan68]

You're tempting me...

Don't do it! Please..

kay

YUMMMMMMM!!!!!

Found it!

[littlebeast13]

Don't do it! Please..

kay

YUMMMMMMM!!!!!

Found it!

Ahem.... I thought someone was a fan of Star Trek....

lb13