Star Rating System Implemented

Instructables can now be rated on a 5-star rating scale. So instead of saying that you like an Instructable, you can now say just how much you like it! Some fancy mathematical formulas were used to convert the previous ratings to the new scale, providing the scores that you see now. These formulas will be getting tweaked as we get some kinks out of the system and fine tune the results, but all of the new ratings will be saved for more accurate results in the future. So for now, go out and use the stars to more accurately rate new and older Instructables! More ratings will help fine tune future searches and encourage more people to post better Instructables.

Picture of Star Rating System Implemented
sort by: active | newest | oldest
1-10 of 37Next »
Very cool!
It seems kinda buggy... found s bunch that says 1 or so ratings, but when you click on They like It, it shows a bunch of names...
Yeah, seems like, for now, the "fancy mathematical formulas" are still a little simplistic. For example, about half the instructables currently have a rating of 8.0 (4 stars), regardless of how many + votes it got. That seems to indicate that each + simply got translates to a 4-star rating. Unfortunately, that means that a new instructable with a single 5-star vote would get ranked higher than an old favorite with 300 + votes...

Most voting schemes like this use something called a Bayesian estimator, which essentially down-weights options with fewer votes, even if they were high votes. That way, 300 + votes would get ranked ahead of 10 + votes, and 10 + votes should get ranked higher than one 5-star vote.

Rating systems are a fairly active area of research - mainly due to their increasing use on the web. Imdb, for example has been using Bayesian estimators and various other tricks for a while, to rank movies based on user votes (there's big money in encouraging or discouraging people to go see movies).

Bayesian Rating - how to implement a weighted rating system
Internet Movie Database - Ranking
What does true Bayesian estimate mean in connection with the IMDb Top 250 ratings?
fungus amungus (author)  Patrik9 years ago
Right now the system is simplistic, it's true, as the new system hasn't been completely rolled out yet. There are a lot of 8s if you look around right now. As you guessed, the final system will be Bayesian. Averages get calculated a the end of each day.
One thing I noticed--if you made an instructable before the "-" minus button was removed, those negative ratings are averaged in. Those 'ibles all have somewhat less than an "eight" rating. More recent projects, of course, couldn't be down rated, and received the "eight" ...
Patrik gmoon9 years ago
Hmm - yes, it does look that way. Those ratings would have to be recalibrated separately. Best thing we can do right now, is to go back and re-rate the older instructables you're interested in, using the 5-star system. The more statistics available to compare the +/- votes, the + votes and the 5-star votes, the better the instructables crew can weight those old votes to be consistent with the new rating system. For example, instead of simply mapping all + votes to 4 stars, and all - votes to 1 star, you can map them to specific values, chosen such that the average rating (or distribution of ratings) based on +/- votes is the same as based on +-only votes, and the same as based on 5-star votes. That way, no instructables are penalized (on average) for having been published during the +/- era. For example, for the +-only era, you could set the value of a + vote equal to the average 5-star rating. You could even do this on a per-instructable basis, at least for those where you have enough + votes and 5-star votes. For the +/- era, you can calculate the total number of + and - votes, and set their value equal to the corresponding percentile of the 5-star votes. For example, if there are 25% - votes and 75% + votes, you would set the value of a - vote equal to the average of the bottom 25% of 5-star votes, and the value of a + vote equal to the top 75% of 5-star votes.
i think this rating system is a lot better than the previous one but i agree with Patrik about the translating problem. also, can you change the score so that it will only show 2 or 3 decimals?
I'm sure that'll get fixed asap, as fungus said - nobody wants to see that many digits! :-D
fungus amungus (author)  Patrik9 years ago
Nope. It'll be cut down to just 2 digits soon.
fungus amungus (author)  fungus amungus9 years ago
2 digits after the decimal place, that is
1-10 of 37Next »