437Views37Replies

Author Options:

Star Rating System Implemented Answered

Instructables can now be rated on a 5-star rating scale. So instead of saying that you like an Instructable, you can now say just how much you like it! Some fancy mathematical formulas were used to convert the previous ratings to the new scale, providing the scores that you see now. These formulas will be getting tweaked as we get some kinks out of the system and fine tune the results, but all of the new ratings will be saved for more accurate results in the future. So for now, go out and use the stars to more accurately rate new and older Instructables! More ratings will help fine tune future searches and encourage more people to post better Instructables.

37 Replies

user

It seems kinda buggy... found s bunch that says 1 or so ratings, but when you click on They like It, it shows a bunch of names...

Select as Best AnswerUndo Best Answer

user
Patrik (author)laminterious2008-05-09

Yeah, seems like, for now, the "fancy mathematical formulas" are still a little simplistic. For example, about half the instructables currently have a rating of 8.0 (4 stars), regardless of how many + votes it got. That seems to indicate that each + simply got translates to a 4-star rating. Unfortunately, that means that a new instructable with a single 5-star vote would get ranked higher than an old favorite with 300 + votes...

Most voting schemes like this use something called a Bayesian estimator, which essentially down-weights options with fewer votes, even if they were high votes. That way, 300 + votes would get ranked ahead of 10 + votes, and 10 + votes should get ranked higher than one 5-star vote.

Rating systems are a fairly active area of research - mainly due to their increasing use on the web. Imdb, for example has been using Bayesian estimators and various other tricks for a while, to rank movies based on user votes (there's big money in encouraging or discouraging people to go see movies).

Bayesian Rating - how to implement a weighted rating system
Internet Movie Database - Ranking
What does true Bayesian estimate mean in connection with the IMDb Top 250 ratings?

Select as Best AnswerUndo Best Answer

user
fungus amungus (author)Patrik2008-05-10

Right now the system is simplistic, it's true, as the new system hasn't been completely rolled out yet. There are a lot of 8s if you look around right now. As you guessed, the final system will be Bayesian. Averages get calculated a the end of each day.

Select as Best AnswerUndo Best Answer

user
gmoon (author)fungus amungus2008-05-13

One thing I noticed--if you made an instructable before the "-" minus button was removed, those negative ratings are averaged in. Those 'ibles all have somewhat less than an "eight" rating. More recent projects, of course, couldn't be down rated, and received the "eight" ...

Select as Best AnswerUndo Best Answer

user
Patrik (author)gmoon2008-05-13

Hmm - yes, it does look that way. Those ratings would have to be recalibrated separately. Best thing we can do right now, is to go back and re-rate the older instructables you're interested in, using the 5-star system. The more statistics available to compare the +/- votes, the + votes and the 5-star votes, the better the instructables crew can weight those old votes to be consistent with the new rating system. For example, instead of simply mapping all + votes to 4 stars, and all - votes to 1 star, you can map them to specific values, chosen such that the average rating (or distribution of ratings) based on +/- votes is the same as based on +-only votes, and the same as based on 5-star votes. That way, no instructables are penalized (on average) for having been published during the +/- era. For example, for the +-only era, you could set the value of a + vote equal to the average 5-star rating. You could even do this on a per-instructable basis, at least for those where you have enough + votes and 5-star votes. For the +/- era, you can calculate the total number of + and - votes, and set their value equal to the corresponding percentile of the 5-star votes. For example, if there are 25% - votes and 75% + votes, you would set the value of a - vote equal to the average of the bottom 25% of 5-star votes, and the value of a + vote equal to the top 75% of 5-star votes.

Select as Best AnswerUndo Best Answer

user
FrozenFire 99 (author)2008-05-10

i think this rating system is a lot better than the previous one but i agree with Patrik about the translating problem. also, can you change the score so that it will only show 2 or 3 decimals?

Select as Best AnswerUndo Best Answer

user
Patrik (author)FrozenFire 992008-05-10

I'm sure that'll get fixed asap, as fungus said - nobody wants to see that many digits! :-D

Select as Best AnswerUndo Best Answer

user
fungus amungus (author)Patrik2008-05-10

Nope. It'll be cut down to just 2 digits soon.

Select as Best AnswerUndo Best Answer

user

2 digits after the decimal place, that is

Select as Best AnswerUndo Best Answer

user
PointyOintment (author)2008-05-11

1. The main difference that I see between this and the old system is that the old system showed how many people like it, and the new one shows "how much the average person likes it." Also, there is no zero rating, so this is more like 0-4.5 than 0-5.
2. What happened to They Like It? I haven't seen it recently, even before the rating system change. I assume that anyone who rates above 2.5 is going to get on the list? What about a list of people who have the instructable in their favorites?

Select as Best AnswerUndo Best Answer

user
Aeshir (author)PointyOintment2008-05-11

Also, there is no zero rating, so this is more like 0-4.5 than 0-5.

The one star means that the ible still there to rate, that the author actually wrote something, even if it is still only deserving of one star.

Also, it doesn't matter if the top rating is 4.5, it's still the highest rating. It doesn't necessarily mean perfect or 100%, but it's the highest rating you can give it.

The rating system goes like this I guess:
1: Dog feces.
2: Poor.
3: Good/average.
4: Pretty good.
5: Awesome.

Select as Best AnswerUndo Best Answer

user
Patrik (author)Aeshir2008-05-11

Actually, the rating system - officially - goes like this: 0.5: Worthless 1: Poor 1.5: Some Flaws 2: Almost Average 2.5: Average 3: Above Average 3.5: Good 4: Very Good 4.5: Oustanding 5: Best Ever

Select as Best AnswerUndo Best Answer

user
Aeshir (author)Patrik2008-05-11

Whoops. I hadn't moused over the stars.

Select as Best AnswerUndo Best Answer

user

Oh. They Like It has moved. I was looking for it in the wrong place.

Select as Best AnswerUndo Best Answer

user
Yerboogieman (author)2008-05-11

nah, jk jk i did think about it sometimes though

Select as Best AnswerUndo Best Answer

user
Yerboogieman (author)2008-05-11

yeah, my idea came alive!

Select as Best AnswerUndo Best Answer

user
comodore (author)2008-05-11

I liked the old one better!

Select as Best AnswerUndo Best Answer

user
Kiteman (author)2008-05-10

I saw the star system, and I have abused it already!

Imagine what she'd have said with any lower score... :-/

Select as Best AnswerUndo Best Answer

user
Goodhart (author)2008-05-10

Who will be the first to get a rating of pi ? LOL

Select as Best AnswerUndo Best Answer

user
guyfrom7up (author)2008-05-10

I think this is better, but it makes my instructables look bad, I mean like 5.75 stars... I was happier with +8 or something like that

Select as Best AnswerUndo Best Answer

user
killerjackalope (author)2008-05-10

Interesting, only problem is one of mine has an obscene number of decimals, It's got 7 or so ratings so the number is going to be that way, interesting addition, I like it alot, only needs a little fine tunign as you said, surely you can add code to simply display the first digits or decimals, then use the most accurate formula for ratings, i know calculators have a way of doing this with their finite digit area so it must be doable...

Select as Best AnswerUndo Best Answer

user
gmoon (author)2008-05-10

I'm a bit confused how as to how "They Like It" will work now....

If you give an 'ible a poor rating, half a star for instance, does that add you to the "like it" category?

Or is "they like it" weighted also?

Select as Best AnswerUndo Best Answer

user
Lithium Rain (author)2008-05-09

Cool! That's a really good idea!

Select as Best AnswerUndo Best Answer

user
Firebert010 (author)2008-05-09

It's definitely a cool system. I can't wait to see how it turns out for the future.

Select as Best AnswerUndo Best Answer

user
DRADIS (author)2008-05-09

I saw one instructable with a rating of 7.2375340358. They should round ratings to the nearest tenth or something. Great idea though!

Select as Best AnswerUndo Best Answer

user
Labot2001 (author)2008-05-09

Kudos - another great idea from the HQ.

Y'all should move more often. ;)

Select as Best AnswerUndo Best Answer

user
Brennn10 (author)2008-05-09
user
Goodhart (author)2008-05-09

This will indeed make it easier for us to "rate" how well we like things...it may make contests a bit more of a challenge for you all though as far as calculating how well something is liked.

Select as Best AnswerUndo Best Answer

user
Dantex (author)2008-05-09

That's awesome!!! I as going to suggest it myself!

Select as Best AnswerUndo Best Answer

user
Keith-Kid (author)2008-05-09

Oh wow, you can give 1 1/2 and such!!! This is great!

Select as Best AnswerUndo Best Answer

user
Keith-Kid (author)2008-05-09

Oh wow, very good idea! This is much better than the old +1 -1 thing!!!!!

Select as Best AnswerUndo Best Answer