Skip to main content

Randomization & Participation Rates

Somebody asked in another thread why the panels weren't randomized such that we got an equal number of responses, but I figured I'd start a new thread on the subject in case people had any helpful suggestions re. the way this was done.

Regarding randomization... my randomization method was to randomly assign the seven panels, BUT to ensure that all panels were assigned equally at the same time. So the very first e-mail I received had a random 1/7 panel, the second e-mail a 1/6 randomization, and so on down to the seventh, where they were guaranteed a certain panel to make sure everything was even... but the randomness of receiving the e-mail at that particular time compensated for that, as well as the fact that the preceding six had been decided randomly so, by extension, the seventh was in fact random as well.

I then repeated the same cycle over and over.

Which is a long-winded way of saying that all panels were +/- 1 evenly distributed (we didn't have a multiple-of-seven number of participants)... BUT since we had a 33% flake-out rate, not all panels received FINISHED entries. Rosenberg's, inexplicably, had one, for instance, Bill Duncan's had two, Shaenon and Roger and Roy had quite a lot.

I can't account for anything except randomness regarding how people completed certain starter panels. Fame obviously wasn't a factor, because Jon is arguably the "biggest" of the starter panel contributors. Openness of the starter panel wasn't a factor, because Roy's was 100% open to interpretation, while Roger's was very, very directed. Artistic merit didn't really enter into it, as all starter panels were meticulously rendered and there was no completion bias on the basis of style, colour/B&W or computer/hand-drawn panels.

In the future, I think I might restrict the number of opening panels for this sort of thing more, or just skip this particular concept -- not through any particular dislike for it, but in favour of trying new things. Believe me, there is no shortage of event ideas on this end.

Randomization & Participation Rates

Shepherd's picture

Somebody asked in another thread why the panels weren't randomized such that we got an equal number of responses, but I figured I'd start a new thread on the subject in case people had any helpful suggestions re. the way this was done.

Regarding randomization... my randomization method was to randomly assign the seven panels, BUT to ensure that all panels were assigned equally at the same time. So the very first e-mail I received had a random 1/7 panel, the second e-mail a 1/6 randomization, and so on down to the seventh, where they were guaranteed a certain panel to make sure everything was even... but the randomness of receiving the e-mail at that particular time compensated for that, as well as the fact that the preceding six had been decided randomly so, by extension, the seventh was in fact random as well.

I then repeated the same cycle over and over.

Which is a long-winded way of saying that all panels were +/- 1 evenly distributed (we didn't have a multiple-of-seven number of participants)... BUT since we had a 33% flake-out rate, not all panels received FINISHED entries. Rosenberg's, inexplicably, had one, for instance, Bill Duncan's had two, Shaenon and Roger and Roy had quite a lot.

I can't account for anything except randomness regarding how people completed certain starter panels. Fame obviously wasn't a factor, because Jon is arguably the "biggest" of the starter panel contributors. Openness of the starter panel wasn't a factor, because Roy's was 100% open to interpretation, while Roger's was very, very directed. Artistic merit didn't really enter into it, as all starter panels were meticulously rendered and there was no completion bias on the basis of style, colour/B&W or computer/hand-drawn panels.

In the future, I think I might restrict the number of opening panels for this sort of thing more, or just skip this particular concept -- not through any particular dislike for it, but in favour of trying new things. Believe me, there is no shortage of event ideas on this end.

a 33% flake out rate is amazingly low. I am impressed.

I liked this idea, but I kind of like the idea that each year has its own flavor and its down set of rules. And from the array of entries that we say It looks like people can handle it with style.

Shepherd's picture

Actual figures: 58 signed up, 25 completed. So really, it's more like a 43% success rate.

Even 43 percent is a pretty good return. Better than most of these types of events.

ah, my bad... i forgot to take into account the drop-outs in the submission ratios.

i do like the idea of trying something new each year, as it keeps the event from becoming too predictable - perhaps a reason why there are so few returning participants!

Rudi Gunther
http://www.fuddafudda.com

I liked the idea lot, especially awarded winners with the panel they started with. Maybe now you know about how many people will participate, you could lower the number of starting panels to 5. I wouldn't suggest any lower because next year could be bigger, given that this year seemed to be the best publicized so far, and next year more people would have read the previous entries and be more excited.

Asking panel contributors to keep their panels more open ended might help as well, even if that didn't figure in with the drop outs.

Re: Randomization & Participation Rates

[quote:861fc63b4c="Shepherd"] Artistic merit didn't really enter into it, as all starter panels were meticulously rendered...

:roll:

(This is BoxJam).