461Views32Replies

Author Options:

Contest Judging? Answered

It seems that all contests have the same "Criteria"  originality, usefulness, simplicity, and clarity of instructions.
I have looked at some of the past contest winners and wonder how these "Criteria" were applied.
Some of the past winners have been anything but simple, and the instructable may have many questions in the comments about what is meant  in the instructions or asking for clarifications. Also the "usefulness" of some of the winners is very questionable.
It looks like the more complicated and outlandish the instructable the better chance it has to win.

32 Replies

user
sherrycayheyhey (author)2012-09-10

This is something that has bugged me too. I now work at Instructables but not much more than a month ago I was an average author on this site feeling the same way as you about the judging of the contests.
When I saw complex projects that were outside the means of the average user (whether it be due to cost, tools used, artistic ability, skimming over steps, etc.) win contests I felt like it had won due to it's coolness factor instead of being judged by the criteria outlined in the official rules.
The great news is that we're working on a few changes to the judging criteria so that Instructables can continue to have the best contests on the Internet!

Select as Best AnswerUndo Best Answer

user
daveda (author)sherrycayheyhey2012-09-10

Thank you.
This is what I was after, someone to actually look at what is going on with judging of the contests.

You would think a judge would actually take a thorough look at the entries they are judging, by reading the whole instructable and all comments about it, and then applying the "Criteria" what ever they may be.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)2012-09-10

"Simple" is both relative and contextual.

Usually, it means "simple to understand".

Do you have some specific examples of winner you do not feel match the criteria?

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-10

I think I remember a Tesla Coil that would cost over $600.00 to build and access to a machine shop. Useful? And no way simple. It would be fun to play with but how many of these were made?
This was from the last Epilog laser contest.
Look at the rest of the winners from this contest.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

Nowhere is there a "cost to make" criteria, and, as I said, "simple" usually means that the instructions are easy to understand.

The laser contests usually focus on large, "cool" projects, because judges want to give the large, expensive rewards to people who put real effort into their work. Note, I said "effort", not "money".

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-10

Then the "Criteria" should be large, cool, and effort, if that is how it will be judged.
What I am saying is the stated "Criteria" does not always match the results.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

Voting is based on people being impressed. Judging is based on the criteria.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-11

Actually I think voting is based on popularity and if your intructable is a featured instructable. The more friends you have and can get them to vote for you and get their friends to vote for you the better chance you have.
I am sure that featured instructables have more views and more votes simply because they are featured. If any part of the decision of picking finalists is based on the votes, then no contest entry should be a featured instructable.
At one time I was told that this was going to be changed so that voting had nothing to do with choosing finalists. I guess no one ever got around to changing this.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-11

Of course voting is based on popularity! If a project does not get looked at, nobody will vote for it. How can it not be?

Yes, featuring has an affect on views, but projects only get featured if they deserve more views, that's the point. Look at any set of contest entries, though, and far more entries get featured than reach the final. That's a non-argument, and your wish to ban featuring on projects entered into contests is deeply unfair to all those feature-worthy entrants that are not quite finalist material.

Who told you that voting would not affect the choice of finalists? What would be the point of voting? And would you then expect the judging panels to go through every entry in the kind of detail we do now? There are already over a thousand entries to the current laser contest, and I expect hundreds more - I have a full-time job, as do the other usual judges, so judging would take months.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-11

I believe it was Scoochmaroo that told me that they were looking at eliminating voting altogether, because it was becoming a popularity contest.

Instructables could be featured after the contest closes. By featuring an instructable before a contest closes means someone is prejudging entries.

If judging would take months, why not just do a simple drawing to choose the finalists. At least this way everyone would have an equal chance. Just change the rules so everyone knows how the contest will work. Then judge the finalists by some "Criteria".

I do understand why you have voting and why you have the broadest possible "Criteria" for all contests. Both of these help drive views of the web site and this improves the ability to sell advertising and this improves revenue.
So I know nothing will change. Why not just admit this, and say we will do as we wish, because this is what will happen.

It has been interesting though seeing the comments of an occasional judge admitting that they really can not judge all entries as it would take to much time.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-13

I know I said I would leave you to your trolling, but I had to answer such an implication.

When there are around twenty-thirty finalists chosen by the members, it takes me a minimum of a day to judge then against the criteria. I have a full-time job and a family - double the number of finalists, judging time goes up to a week.

You imply that I do not judge all entries because I can't be bothered to put in the time. That is a flat insult.

I think, instead, looking at the way you changed your arguments and moans through this topic, readers will be able to draw their own conclusions about the true reason you want the rules changed to prevent hard-working members from winning.

I am done with you.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-13

A simple draw would not work.

First: Nobody would be motivated to produce awesome projects, and the quality of the whole site would deteriorate. Why should anybody put the effort into an impressive major project, when it has no more chance to win than something cobbled together from a few scraps of plastic?

Second: It would be illegal in many countries. As it stands, contests are a challenge of skills, but drawing from everybody who enters makes it a lottery, which counts as gambling.

Select as Best AnswerUndo Best Answer

user
scraptopower (author)daveda2012-09-10

Where does it say the contest criterea is originality, usefulness, simplicity, and clarity of instructions ?

Select as Best AnswerUndo Best Answer

user
daveda (author)scraptopower2012-09-10

Go to any contest, click on " Official Rules ", scroll down to Judging and read the first sentence.

Select as Best AnswerUndo Best Answer

user
scraptopower (author)daveda2012-09-10

So it does, I never read that page... You are right, but I don't see it ever changing.

Select as Best AnswerUndo Best Answer

user
daveda (author)scraptopower2012-09-10

You are right, I do not think it will change either.
They use basically the same rules for every contest instead of making the rules fit each individual contest. To much work I guess.

Select as Best AnswerUndo Best Answer

user
scraptopower (author)2012-09-10

Contests are judged by humans, so whatever captures their interest/imagination is likely to get a better rating, even if it isn't too practical, useful, or simple. So long as humans are involved, the judging is never going to really adhere to these rules.

Select as Best AnswerUndo Best Answer

user
daveda (author)scraptopower2012-09-10

Exactly my thoughts on the rules not being followed. Why not just say anything goes, and we will pick what we like?

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

What you also need to understand, is that in almost every single contest run on this site, the finalists are chosen by the ordinary members, voting on the projects they like.

It is exactly a case of "we will pick what we like", but "we" means "the members who vote".

The judges only get to choose from the relatively small number of projects that make it to the final.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-10

Actually if you read the rules only a portion of the finalists are chosen by votes.
Less than half, the rest are chosen by judging.
The final winners are not supposed to be influenced by voting at all.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

And which portion do you think is chosen by the votes?

The judging of the finalists is not influenced by votes - the judging panel (of which I am occasionally one) are not told the voting results, and they are not told which finalists were selected by votes vs direct selection.

Plus, you are looking at the rules of one contest, and assuming they apply to all current and past contests. They don't.

The simple fact is that all winning projects are good, and involved a lot of effort and creativity on the part of the author, both in creating the object, and in writing up the instructable.

If you want to win a big prize, you've got to enter an impressive project.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-10

Read No. 8 in any contest, they all say "49% of finalists are determined by user votes," so I ques if rules were followed it would be 49%.
As far as the same rules applying to all contests, show me one that rule No. 8 Judging is different from the rest.

I believe that I stated that judging the finalists are not supposed to have anything to do with the popularity vote.

Again either the contest should be judged under the stated "Criteria" or change the "Criteria" to match the way it will be judged. If it takes "an impressive project"
to win this should be part of the stated "Criteria". If it has to "involved a lot of effort", again this should be part of the "Criteria" that it will be judged under.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

(How exactly do you choose 51% of 24 people?)

I am not going to go through all the rules of all the contests, when you can't even be bothered to read what I type here.

It does not matter what the rules say, the members will vote for whatever they feel like. The judges, however, *do* stick to the criteria.

Just because you do not think that one possible winner of one past contest did not match one of the criteria, that does not mean the result is wrong.

No entry ever meets all criteria exactly.

I really do not understand why you are complaining, it's not as if this [alleged] issue has affected any of your own projects.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-10

I did read what you typed here! This is from your previous comment "Plus, you are looking at the rules of one contest, and assuming they apply to all current and past contests. They don't."
I think this is saying that all contest rules are not the same. Is this what you are saying? If so prove it. Show me one contest where Official Rule 8. Judging is different from the rest of the contests. The only thing that varies is the number of finalist.

As far as your comment above "(How exactly do you choose 51% of 24 people?)"
That is a very good question, suppose you as a part of "the judging panel (of which I am occasionally one)" provide the rest of us non insiders an answer?
As an occasional judge I would think you would know the rules.

I am not complaining, just commenting on an inconsistency between "Criteria"
and out come of some of the contests.

And it seems as though I have hit a nerve.

I was pleased to read "sherrycayheyhey" comment that this is being looked at,
that is all I was after.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-10

Yes, you have hit a nerve - you are laying false accusations, accusing people if not doing their job.

As far as I can see, your real gripe appears to be that winning projects are too hard for you to do, or too expensive for you.

Unless you can start proving your assertion with some actual evidence (examples of winning projects that did not meet the judging criteria), then I am done with you.

Select as Best AnswerUndo Best Answer

user
daveda (author)Kiteman2012-09-11

No, my "gripe" is not with any winning project, it is with judging "Criteria"., and the fact that you as an occasional judge are not familiar with the rules that are supposed to be applied.
It appears that I am not the only one that feels this way, read "sherrycayheyhey" comment above, perhaps you should be in closer contact with the staff there.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)daveda2012-09-11

If you have no problem with any of the winning projects, how can you object to the criteria against which they are judged?

If you have no problem with any of the winning projects, how can you have a problem with the process by which they are chosen?

If you have no problem with any of the winning projects, how can you have a problem with the people that do the judging?

You are broadening your argument, pulling in anything that occurs to you, but making no valid points.

Feel free to have the last word, because I'm leaving you to your little tantrum.


Select as Best AnswerUndo Best Answer

user
Moem (author)Kiteman2012-09-11

Surely it's possible to object to elements of the process even if you don't object to the results?

Select as Best AnswerUndo Best Answer

user
Moem (author)Kiteman2012-09-10

I really do not understand why you are complaining, it's not as if this [alleged] issue has affected any of your own projects.

Sometimes we need to bring stuff up even if it doesn't affect us directly. This is pretty basic to me. And I see no need to get so defensive about it.

Select as Best AnswerUndo Best Answer

user
daveda (author)Moem2012-09-11

Thank you for your comment.
I was trying to point out that all contests, rule 8. Judging, have the same Criteria.
It would seem to me that the Criteria for judging should be made to fit each contest.

Some times the basic part of the rules are not followed, in the last Epilog laser contest the rules stated that 28 finalists would be chosen but only 17 were actually chosen. When I asked about this I was told that they some times change the rules after the contest entries close. Hardly seams fair.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Moem2012-09-10

The author is complaining that judges regularly fail to apply the criteria. This is simply untrue - the only "evidence" is one half-remembered example of a project winning that was complicated to make.

I still do not understand his actual gripe, since his main accusation is false. Is he annoyed that winning projects are outside his personal skill range? Or outside his budget?

Select as Best AnswerUndo Best Answer