1489Views105Replies

Author Options:

Favorites and Ratings Answered

Picture of
In our recent update, one of the major changes is the removal of ratings and emphasis on favorites.

Key points:
  • We did this because most people rate either 1 star or 5 stars
  • The ratings ended up extremely close to one another and became meaningless
  • Our experience with a +/- system showed that having a down vote made for a toxic experience for everyone
  • We're now just using the favorite button now to simplify and keep it a positive experience for all

Read on for more details.

We've had a rocky relationship with ratings.  When we started Instructables in 2005, we had a "+/-" rating system where members could express either an up or a down vote for an Instructable.  Good Instructables generally received lots of "+" votes, and the list of the most negatively voted Instructables was pretty amusing (with this taking the bottom position for quite some time).  Our experience with this system was two-fold: first, negative ratings really didn't add much value to the community and were often given in a mean-spirited way rather than in a spirit of constructive criticism; second, the community expressed an interest in giving a more nuanced rating to an Instructable.  In fact, for a while members would comment on a project and assign it a letter grade from A to F. 

So, we implemented a 5-star rating system.  An Instructable's rating was calculated by the average of all of its ratings.  This method is fine when there were lots of ratings, but gives poor results when there are few.  For example, if the first two raters of a newly published Instructable both rated it 1 star (for whatever reason), that Instructable would appear to be terrible by its rating, and its rating would only improve as more people decided to give it a chance and perhaps rate it.  It is an awful experience for new authors to watch their work be initially down rated and then maybe improve over several days.

To address this, we built a Bayesian rating system with 5 stars.  The objective was to push the rating of an Instructable towards the average rating of all Instructables when it had few raters.  As more people rated it, the rating could diverge further and further from the average.  We spent quite some time optimizing this system: we built messages helping people understand what 1 star meant vs. 5 stars; we made it so the first couple dozen ratings of a new member had lower weight than later ratings (so it was difficult to change an Instructable's rating by rating it with lots of newly created accounts); and we tweaked all of the underlying knobs and ratios.  When everything had settled, a newly published Instructable would have a rating around 3, a very good Instructable could get to around 4, and an extremely popular and highly rated Instructable approached 4.8 or 4.9. 

It seemed like the system was working, and it gave a quantitative measure of the quality of an Instructable in relation to other Instructables.  However, it was ultimately deeply dissatisfying.  After all this work, and lots of input from the community through ratings, thousands of Instructables were all rated pretty much the same, with only slight differences in the rating's third digit.  The slight variations were most likely noise, and the ranking didn't give a definitive sense of quality to a new visitor nor a sense of accomplishment to an author.  What does it mean to say Sweet Potato Fries is rated 4.22 compared to Arduino-Controlled Robotic Drum at 4.23?  Is the rating going to help you decide which Instructable you might try your hand at?  Further, when we looked at how people actually rated, it turned out that ratings were heavily skewed towards 1 star or 5 stars.  Even after requesting more nuanced ratings, and despite how we described what various star levels meant, people were still only giving As or Fs -- either they liked the Instructable or they didn't. 

This system probably would have been ok if not for two factors:  star ratings appearing in Google searches and the rise of the "Like" button. 

If you search for "sweet potato fries" on Google you'll likely get a couple of recipes including one from Instructables.  Many first time visitors to Instructables find us through searches -- many of them food-related -- so it's very important how our content appears in search results.  A while back, Google started including star ratings in some of the search results, primarily recipes.  Many recipe sites have 5-star rating systems, so I guess the presumption was that including the ratings might help searchers better choose between results.  Unfortunately, this doesn't work because you cannot compare the ratings of one site to the ratings of another.  For example, many of the results for a "sweet potato fries" search have 5-star ratings using a basic average system (you can easily tell by the number of raters), while the result from Instructables might have more raters giving it 5-stars, but because of our Bayesian system, the overall rating might only be 4 stars.  On Instructables, 4 stars represents a very good project, while on another site, everything -- even garbage -- might be rated 5.  All of our work to create a meaningful rating system was being used to penalize us in search results -- why would a searcher click through to a 4-star sweet potato fries recipe when they could go to a 5-star recipe?

We couldn't tell how many people were choosing not to visit Instructables because our projects had lower star ratings, but we needed to do something.  I'm not proud to admit this, but our solution was to inflate all of our ratings by a full star.  Instructables rated 4.9 were suddenly joined by Instructables previously rated 3.9.  The ratings became even more meaningless because lots of Instructables were now all rated 4.5 or greater. 

At this same time, Facebook's "Like" button was gaining momentum.  The "Like" button is a pretty nice concept as it eliminates the ability to express a negative preference and it associates an individual with the rating.  I think it's more meaningful to know that 10 people like my Instructable than it is to have a 4.5 (or whatever) rating, even if that rating is the result of 10 people giving me 5 stars. 

Not everyone on Instructables uses Facebook, nor do we want to outsource our rating system to Facebook, but we wanted to learn from the success of the "Like" button and expand upon the concept to make our rating system less like giving a grade or being graded.  To accomplish this, we've removed our ratings and made the "favorite" action the way to express a preference on an Instructable.  Marking something as a favorite is more of a commitment than saying you like it or giving it a rating, but this is a distinction I see as important and worth developing.  At Instructables, we hope to inspire you to take action and build something great which requires a greater commitment than simply liking something.  In the future, we plan to expand what you can do with favorites, exploring concepts such as "I made it" and "I want to make it" among others.

Do we need something more akin to a like button in addition to favorites?  I'm not sure, and I'd love your feedback.  We'll be closely watching how people use favorites in its new, highly prominent position.  As with most things, this is an experiment, and we're collecting data to see if this change is something that helps our authors.  I hope you've enjoyed seeing a little bit under the hood about our rating system!  Let us know what you think.

50 Replies

user
justbennett (author)2013-06-03

I'm not proud to admit this, but our solution was to inflate all of our ratings by a full star.

Hahaha! Nice. Thanks for taking the time to explain this process. Very interesting. Personally, I've been using the "favorite" feature as a bookmark. I guess Pintrest is having an influence as well?

I've been wondering what the criteria is for projects that are listed under "popular." Is it views per hour? Views * Favorites/Comments + Search results...  No reason. Just wondering.

I agree that negative feedback is unhelpful especially from anonymous sources without explanation.  I would fully appreciate critical comments from people who are trying to help, but usually anonymous reviewers show no restraint and indulge in self-gratifying condemnation.

As to the "made it" "want to make it" discussion--Great idea. What if there were a way to assign points so that more difficult projects would have greater reward (whatever that reward may be let's call it MakerCred)? That way I wouldn't get as much MakerCred for kissing as I would for How-to-build-your-own-Jet-Engine.
 

Select as Best AnswerUndo Best Answer

user
Kiteman (author)justbennett2013-06-03

Oh, that last point could be linked to the difficulty slider that lots of people ignore when they publish their projects.

Select as Best AnswerUndo Best Answer

user
andrea biffi (author)Kiteman2013-06-04

I think that difficulty slide could be very useful, but it's a very subjective judgment, so it's hard to be valuate to distinguish universally a project from another. Maybe similar parameters as "tools needed" and "skills needed" could be used, with three choices each ("your hands" "a few ordinary tools" "professional tools", and "no skills at all" "jury-rig" "a genius degree".. just some brainchild)

Select as Best AnswerUndo Best Answer

user
darman12 (author)Kiteman2013-08-19

Maybe for cost you could enter a value, that way it isn't subjective at all.

Select as Best AnswerUndo Best Answer

user
artlife (author)Kiteman2013-06-09

I tried to use the difficulty slider on my instructable before I posted it, and it didn't work. Same with the cost slider. Did I miss something or is it down?

Select as Best AnswerUndo Best Answer

user
Kiteman (author)artlife2013-06-09

It doesn't seem to work on touch-screen devices.

Select as Best AnswerUndo Best Answer

user
artlife (author)Kiteman2013-06-09

Well, that's good to know. Now I won't have to buy a touchscreen device to try it. ;-)

I don't have touchscreen. I am using a pc laptop. I can try it with a mac.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)artlife2013-06-09

If it's not working with a mouse-based device, then that's a bug - could you start a fresh topic to report it? You'll need to include details of your browser, operating system and any add-ons you are using.

Select as Best AnswerUndo Best Answer

user
explosivemaker (author)2013-05-18

You DO realize that a lot of people have wanted/do want a dislike button on FB right?

I liked the old system. Now we can't even rate things at all. I just hate seeing really good guides being published and thrown in with everything else which ranges from mediocre to just plain sad. They deserve better. I don't see the point of "being nice" when the instructable lacks every single point that separates even the average from the bad. Sufficient pictures, a thorough description (WITH PROPER ENGLISH), and the fact that it's for something that actually has a point.

Either do it right the first time, or don't do it at all. There's no excuse for a complete lack of effort; which is what I would happily give a one-star for.

Select as Best AnswerUndo Best Answer

user
ALDIETZ (author)2013-06-12

I really wish you guys would bring back the "like" button from Facebook. Although I am not a facebook user myself, It helped me keep track of how many people were actually viewing (and liking) my hard work. It actually was a HUGE motivator and driving factor. The more "likes" I had (~133k before the takedown) the more propelled I was becoming to try and make a name and business plan for myself, something I had no aspirations to do before. I loved getting on my instructble weekly and checking how much (if any) the count went up. I had no idea until about a month ago that more than 20 people knew what I made, let alone LIKED it! You can’t just get rid of 133 thousand peoples likes.... its making me sad and unmotivated.

Please instructables... bring back a system of "likes" where anyone anywhere on any platform can view and "like" the things I am working hard to create. I don't care if its facebook "likes" or youtubes "thumbs up" I just want to know how many people are interested in what I have to offer the world.

Select as Best AnswerUndo Best Answer

user
vanweb (author)2013-01-14

Based on the concept of Instructables I think you should have an "Inspiration" rating... Basically the question "Did this Instructable Inspire you?" Then on the page it can say "This Instructable inspired 12 people". "Favorite" is too vague and "I would make this" sometimes does not apply as some instructables inspire me but I would not nessacarily replicate them... Thoughts?

Select as Best AnswerUndo Best Answer

user
artlife (author)vanweb2013-06-09

This sounds excellent to me. I see lot's of instructables that I would not make, but they inspire me to make a variation or change plans in other projects. Even though the site is called instructables, I am here for the inspiration!

Select as Best AnswerUndo Best Answer

user
RetroTechno (author)2013-01-14

I don't like the "Facebook" approach described here. As a general rule, I don't link external sites to my Facebook account. Those guys are all about gathering information from me.

Maybe what is needed is a "thumbs up" policy without the "thumbs down." That way, those who do outstanding work can still be easily found without hurting anyone's feelings.

Select as Best AnswerUndo Best Answer

user
liquidhandwash (author)2013-01-11

looks likes the stars have come back for some instructables, food has stars

Select as Best AnswerUndo Best Answer

user
frollard (author)2012-12-24

What if people had ownership of their star ratings in conjunction with what Ian01 says below --

Have a 1-5 star system but break it into
idea/originality/novelty:x/5
writing/presentation:x/5
photography/media/video:x/5
honesty/integrity/repeatability:x/5
Textual rating: pros/cons/constructive crits.
and if you comment on the instructable, your root level comment will be stamped with your ratings in the corner. Might get a bit messy - but it means you own what you say. Perhaps even a 'rating comments' section, like various websites product review sections. If you rate something high or low, justify. Include rate-this-rating system so that accurate ratings that are well thought out, helpful and constructive float to the surface while the trolls go back under their bridge.

Select as Best AnswerUndo Best Answer

user
Sparticles (author)2012-12-22

I am quite passive in rating and liking.
But personally, I feel that 'favorites' is something quite personal and I am quite reluctant to make something a favorite.

The star system was more comfortable because if you just like something you could press on the fifth star and you have anonymously contributed to the statistics.

But by pressing 'favorite' you add the instructables to your 'favorites' which means they mean something something to you.
(The two favorites I have now are just two instructables I really like, but not at all my favorites)
Also, the things you favorited are visible on your profile which makes it sort of personal.

Maybe (in my opinion) the best rating system would be a youtubeish 'thumbs up' button. Clean, simple, anonymously add to the statistics.
And a favorite button for you favorites.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Sparticles2012-12-22

"...I feel that 'favorites' is something quite personal..."

To me, that's kind of the point.

It's not like google or something, where the things you have already looked at influence what you see, or influence the adverts that pop up.

Select as Best AnswerUndo Best Answer

user
PointyOintment (author)2012-11-12

WALL OF TEXT! There's a summary at the end.

I have some suggestions for the rating system, and, since I noticed some things wrong with the comment system while reading the comments on this post, I have some suggestions for the comment system as well. (In particular: It's very difficult to follow the discussions when only one level of replying is possible, and some of the comments here are quite insightful, but they're hidden among the other comments.)

1. Ratings
Have you looked at Wikipedia's article rating and suggestion system? At the bottom of every article, there's either a "Rate this page" box (Article Feedback Tool Version 4) or a "Help improve this page" box (Article Feedback Tool Version 5).

1.1 The "Rate this page" box lets readers rate the article using five-star scales in four categories: trustworthiness, objectivity, completeness, and writing quality.

1.1.1 You said that the plus/minus rating system was not nuanced enough; I contend that even the one-dimensional five-star system that replaced it was not nuanced enough. A multi-dimensional rating system like Wikipedia's would have at least two advantages over expressing similar rating information in text comments (note: by ratings in comments I mean things like "Good photos, but hard to follow", which are not the same as suggestions): it would allow the opinions of many readers to be combined into a summary without conflating all of those opinions about different aspects of the instructable into one 'good or bad' rating, and it would allow instructables to be compared and sorted based on different aspects of their quality. I suggest the following dimensions: quality of finished product, inclusion of proper techniques, writing quality, photo quality, completeness, ease of following instructions, innovation, accessibility*, etc.

*Accessibility might not be a good thing to rate, because some people have 3D printers/CNC mills/whatever other tools, and some don't. I know I got a bit irritated a while ago when it seemed like almost every instructable required an expensive computer-controlled tool, and I wouldn't want such instructables to be rated poorly just for that. It would be similar for instructables requiring ownership of a sailboat, etc.

1.1.2 However, even a multi-dimensional five-star system would be susceptible to people voting at the extremes of the scales, as you mentioned. Therefore, I suggest substituting text-based scales in place of the stars. For example, the photo quality scale could have the following options: "All or almost all photos are clear and show all necessary details", "Most photos are clear, but some details are difficult to see", "Many of the photos are worth including, but some are not", "The photos add only a small amount of value/clarity", "This instructable would be no worse if the photos were removed". Each option would be given a point value of 1–5, but the point values would not be shown to the person submitting a rating, so that they would be more likely to rate honestly instead of just "1" or "5". Maybe the textual rating options could even be shown in random order. The point values would be used to calculate the average rating in that dimension (perhaps using that Bayesian algorithm).

1.1.3 It would be good to allow users to submit ratings in only some dimensions, so that they can still provide ratings even if they don't know how to or don't want to provide some of the ratings, or if one or more of the dimensions is not applicable.

1.2 Back to Wikipedia. The "Help improve this page" box asks readers "Did you find what you were looking for? Yes/No", and lets them write a suggestion for improvement.

1.2.1.1 The question "Did you find what you were looking for? Yes/No" or "Did this instructable help you? Yes/No" is fundamentally different from the quality ratings because it asks people whether or not they found the article/instructable useful, not how good they think it is. A usefulness score for each instructable (maybe number of "Yes" minus number of "No", or maybe use a Bayesian algorithm to compensate for low views) would be another way to sort instructables. It's somewhat similar to the old plus/minus system, but it's intended to measure just one aspect, usefulness, instead of condensing all aspects into one 'good' or 'bad', and it's probably easier for the users than picking from a range of usefulness values.

1.2.1.2 A similar suggestion feedback feature on Instructables might help new authors by letting other users submit suggestions for improvement without putting them in the comments section (which might not be monitored closely by the author—authors should be notified when they receive suggestions—and which would thereafter be more focused on providing helpful tips to other people, asking questions, and showing pictures of re-makes than on the original author's presentation). On Wikipedia, the suggestions and yes/no ratings are then displayed along with the names of users who submitted them on a feedback page, which, because Wikipedia articles are publicly editable, is visible to everyone. This raises the question of whether or not the feedback should be publicly viewable. Since an instructable is only editable by the author and collaborators, they're the only ones who actually need to see the feedback. On the other hand, making feedback public (not on a separate page, but beside the comments or below the info box, for example) would reduce duplicate suggestions. Wikipedia also allows editors to mark suggestions as resolved once the article has been edited accordingly.

1.2.2 Wikipedia only asks for suggestions after the "Did you find this article useful? Yes/No" question has been answered, but I think on Instructables they should be independent questions, so that people can submit suggestions even if they weren't actually looking for anything (in which case "Did you find what you were looking for?" would be meaningless) and so that people can say whether or not the instructable was useful even if they don't know how it could be improved (such as some very well-made instructables).

2. Comments

2.1 We NEED hierarchical comments back. Reddit has them for a reason. Instructables had them for a reason. They make it possible to have a coherent conversation with multiple branches that can easily be read at a later date and by other people. Even Facebook seems to have realized that flat comments make discussion difficult; they're starting to implement two-level comments like Instructables now has. It's better than one level, but only slightly. There are some blog CMSes that limit comments to four or five levels deep, and that's almost as bad because once you've gotten four or five messages into the conversation, it's difficult to continue without the subsequent branches getting mixed up. Now, on Instructables, it's difficult to have any conversation without the branches getting mixed up. Case in point: comments on this very post.

2.2 Let users publicly* mark comments that they think are helpful, insightful, or otherwise worth reading, and then allow users to sort comments by those ratings. This is what reddit does, and for the most part, it works well; the default comment sorting method there is based on ratings.** This would move comments that are worth reading closer to the top, where they'd be more likely to be read. Currently, the only way to express that a comment was good is to say so in a reply to it, which doesn't promote the good comment and only hurts the overall signal-to-noise ratio. Furthermore, I suggest incorporating Slashdot-style helpful/insightful/good-question/funny/etc. options*** (of which a user can select more than one) instead of just a 'like' or non-specific upvote, so that readers can sort comments by good questions to see if their question has already been asked, by helpfulness to look for any tips from other users they should take into account before starting a project, or by insightfulness if they're trying to find a way to improve upon the original project.

*I don't know whether the names of people who have given particular ratings to a comment should be public, but the total number ratings/total number of each type of rating a comment receives should be public; in my opinion, that's half the point of rating them.

**The only problem I see with sorting comments by rating by default is that new comments are unlikely to be seen. On reddit, threads don't usually last more than a few days, but on Instructables, I think the default sort should still be reverse-chronological, so that new comments get a chance to be read, replied to, and rated.

***I don't suggest incorporating any other aspect of the Slashdot moderation system. I think these rating options should be available to all users, because comments on obscure instructables would be unlikely to be seen by Slashdot-style moderators.

2.2.1 If you've found that worthless comments have been a problem—which I haven't—you could add another option, not worth reading, which would push that comment toward the bottom in all non-chronological sorting schemes. I feel that the wording of this option would be important, because many people on reddit use the downvote on things they disagree with, when it should be reserved for things that aren't worth reading. Maybe the "be nice" comment policy should include a point about appropriate downvoting if downvoting is implemented. On the other hand, the whole downvote/not worth reading option probably isn't necessary.

Summary of my suggestions: Multi-dimensional rating system for more clear and useful ratings, with textual scales instead of stars to promote honest and moderate ratings instead of just "1" or "5". Yes/no feedback on usefulness. Suggestions for improvement, separate from comments. The return of hierarchical comments. Ratings for comments and the ability to sort comments by rating.

P.S. Just tell Google that every recipe has five stars.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)PointyOintment2012-11-13

If the "dislikes" are meanspirited, so? That's the nature of the internet.

Other websites tolerate malicious negativity, do we ought to as well?

Sorry, that has never been the attitude here, and I and the rest of the Community Team put a lot if effort into making sure we do not descend into the cesspit of negativity that some internet users seem to think us an appropriate way to interact with the rest of humanity. This is not YouTube. A true Maker is positive and constructive.

If somebody cares enough to express their opinion through a complex, multi-optional rating system, then they care enough to post all that needs saying in the existing comments.

Select as Best AnswerUndo Best Answer

user
Javin007 (author)Kiteman2012-11-15

You're clearly missing the point, Kiteman. If someone "mean-spiritedly" votes 1 star on a person's 'ible, is it going to cause the site to "descend into the cesspit of negativity" that you describe? Of course not. Your extremism is neither necessary nor constructive.

If someone has a good 'ible, and someone gives it a negative vote out of "meanness," so what? It's not like it's a vulgar comment, and there's no knowing whether the vote was out of "meanness" or because they honestly didn't like the 'ible. With the CURRENT system, you give someone who's offended, or sincerely dislikes an 'ible only ONE way of showing their distaste:  Through a mean comment.  

Either way, if it's a "good" 'ible, then the positive votes will far outweigh the negative votes, and the end result is still the same.  The "mean spirited" individual got to throw his fit without having to devolve to making mean comments, and the ultimate impact to the site is nil.

Kiteman: "If somebody cares enough to express their opinion through a complex, multi-optional rating system..."
While someone else may have asked for a "complex, multi-optional rating system" it certainly wasn't me.  I asked for a simple like/dislike as has worked brilliantly for YouTube for about 6 years.

Kiteman: "...then they care enough to post all that needs saying in the existing comments."
Once again, you're utterly ignoring my previous statement.  Comments would be of absolutely zero use for any of the following (as I previously mentioned):

From previous post: ------------------------
"I've seen a number of sites like this go straight to hell after taking away any dis-incentive to post garbage.  With absolutely NO way for people to voice displeasure at having their time wasted even clicking on a lousy instructable that someone took ZERO time to produce, such as this:
https://www.instructables.com/id/wooden-sword-1/
or this:
https://www.instructables.com/id/cardboard-gun-prop/

Soon the site will be so full of absolute garbage, with no way to jump to the "highest rated" (and no, "most liked" is certainly not the same thing) that those who DO take days, weeks, or even months to create a thorough and impressive instructable will lose all incentive to participate.  Why would they want to waste their time if their 'ible is guaranteed to get buried beneath the ego stroking of 10 year olds?  And if I'm coming to search through the 'ibles, I'm going to be looking for those that ARE of the best quality, without wanting to have to click through and read several dozen to find the one that some effort was put into.
...
A [YouTube-Like] system only makes sense on a site like this where you may find two dozen different 'ibles on composting and wish to quickly find those that were the best designed, not just the ones that happened to be on the site for the longest, or that were posted by people who have 500 facebook friends."
------------------------------

This site was not (when I became a Pro member, at least) here for the sole purpose of allowing people to stroke their own egos, or to cater to pre-teens.  It was here as a resource for people who like to make stuff themselves share their knowledge and expertise, or to learn from those that have the knowledge or expertise.  Perhaps as Instructables as moved into the more commercial realm, the charter has changed.  If the goal is to placate everyone in order to drive up user counts in order to make as much money as possible, let's be honest about it at least.

By removing the rating system so as not to "hurt anyone's feelings" you've lowered the bar to the point that this is just another site where anyone can post anything (as evidenced by the above links), and trying to filter through the trash to find anything of value will soon become more trouble than it's worth.

This said, Kiteman, your post has made it clear that you've little interest in my opinion.  I'm sure this will quickly devolve to, "Well, if you don't like it, leave!" so this will be the last I have to say on the subject.

Select as Best AnswerUndo Best Answer

user
canucksgirl (author)Javin0072012-11-15

Perhaps a "needs improvement" button should exist along with a "like" button. It gets the same point across (imo) in a "be nice" sort of way, and lets the author know that something is lacking (plus tells other members that the project isn't quite up to par).

Select as Best AnswerUndo Best Answer

user

Unfortunately if you have a 'needs improvement' button you will then need a how to improve comments box otherwise it wouldn't work. There's nothing worse than going to an audition and not getting feedback as to why you weren't picked which is the same as why does this instructable need to be improved.....and who is the authority recommending this? Needs more thought.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Javin0072012-11-15

Woah - I hadn't even addressed a comment to to you, and you've managed to manufacture an entire row, all the way to putting words in my mouth and storming off in a temper at those words.

OK, you've had your last word, but in case other people are interested in my "extremist", fact-based view;

Down-rating has never prevented the posting of rubbish (see the very YouTube and FaceBook you wish to emulate). What has reduced the posting of rubbish has been the positive response to good projects, and the hours and hours of work Staff and the Community Team put in behind the scenes every day to helping people to improve their projects. Members who post poor projects without any interest in improvement quickly lose interest because of the lack of views and the lack of positive comments/ratings. Their projects slide into obscurity, only seeing the light of day when somebody drags them up to "prove" that loads of rubbish is being posted and drowning out the good stuff (which is nonsense, as any quick browse of the "recent" lists will show).

Historically, the ability to down-rate projects has encouraged trolling, with members posting low-quality projects, then using multiple sock-puppets to vote them up and to vote other, competing projects down. It got to the point where down-votes had to be excluded from contests where entries were selected as finalists on the basis of popularity, because the only projects that were coming up with positive totals were the dross posted by the trolls.

Now, you are welcome to think whatever you like (and you clearly do), but the above is posted from direct experience, and is a matter of record, not opinion.

Select as Best AnswerUndo Best Answer

user
Javin007 (author)Kiteman2012-11-15

"Woah - I hadn't even addressed a comment to to you, and you've managed to manufacture an entire row, all the way to putting words in my mouth and storming off in a temper at those words."

No, you'd only quoted me then proceeded to put words in my mouth and argue against your straw-man. (Projection much?)  That you quoted me and then responded to the wrong thread is hardly my fault.

"Down-rating has never prevented the posting of rubbish"
Once again, you conveniently ignore statements that have already clarified this.  At no point did I say it prevents this.  I've (repeatedly) stated that this would allow us to easily filter through the rubbish, and continue to stand by that statement.

"What has reduced the posting of rubbish has been the positive response to good projects"
Which is why I was very easily, and quickly able to show two (new) rubbish posts that I found while trying to find a good post on how to use cardboard to build props - posts that wasted my time as I had no context to know that they were rubbish without opening them.  Coddling people has never, in the history of the world, stopped anyone from slacking off.  

"Historically, the ability to down-rate projects has encouraged trolling, with members posting low-quality projects, then using multiple sock-puppets to vote them up and to vote other, competing projects down."

Yes, and I've been the "victim" of this exact type of trolling on YouTube.  Do you know how much it impacted the overall net result?  Zero.  Zilch.  Nada.  It's a failed argument from the start.  Even moreso if you limit the ability to vote to those that have Pro memberships.  A quality 'ible will overwhelmingly get enough positive votes to make such trolling pointless, and virtually non-existent when the "right" to vote is limited.  (This is posted from direct experience, and is a matter of record, not opinion.)

"It got to the point where down-votes had to be excluded from contests where entries were selected as finalists on the basis of popularity, because the only projects that were coming up with positive totals were the dross posted by the trolls."

Cool story, bro.  Still, it has nothing to do with the conversation at hand.  The contests already have their voting completely separate from the rating system that would be used for said filtering.  

As I do business metrics for a living, I can assure you that there are patterns that can be discerned any time someone may attempt to "game" the system that are easily thwarted.  This is a matter of both record and professional opinion.

"Now, you are welcome to think whatever you like (and you clearly do)"
Thank you for your permission.

Back to my original post - it was simply to state my (very strong) opinion that a like/dislike option is necessary for a site such as this.  I was not attempting to get into this "flame war" but I will certainly defend my position, particularly when it's responded to in such a dismissive and condescending manner by one of the very people claiming that this site should have a strict "be nice" policy.  This is a matter of record and opinion.

And yes, I did say I wouldn't respond again, but God forbid I be accused of "storming off in a temper."

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Javin0072012-11-16

What flame war? The only one flaming here is you, but, you can spout and fizz as much as you like, you are not going to change anything on this site by TL:DRing at me.

(Oh, and please pay attention - when I mentioned "trolling", I meant here, not YouTube. Do you see it now? No. Do you know why? Because we took away the incentive to troll by removing the ability to down-rate. This is not YouTube, this is not FaceBook, and I, personally, am proud of that fact.)

Select as Best AnswerUndo Best Answer

user
bricabracwizard (author)Kiteman2012-12-16

I totally agree with you, let's keep this a positive site! We have enough 'crap' in the world without adding to it!

Select as Best AnswerUndo Best Answer

user
Lithium Rain (author)Kiteman2012-11-13

Yeah, no true Scotsman is negative! :P

Select as Best AnswerUndo Best Answer

user
PointyOintment (author)Kiteman2012-11-13

I agree with everything you said, but it seems to me that you might not fully understand what I meant. (Of course, I understand that you're busy and probably don't have time to respond to everything.) I'll try to clarify:

Other websites tolerate malicious negativity, do we ought to as well?

I carefully chose the words "not worth reading" to mean exactly that, and to discourage malicious use (though it would be impossible to prevent malicious use). I did say that a downvote option probably isn't necessary. I didn't mean to advocate for it. I just included it for completeness. Feel free to ignore it while reading the rest of my suggestions; it has no bearing on them.

A true Maker is positive and constructive.

I agree. To clarify my intentions, the downvote/"not worth reading" vote would serve to reduce the prominence of comments that are not positive and constructive. (I personally think a lot of people don't want to report comments just for being unhelpful, because it feels like censorship or wasting the team's time.) Again, though, I don't think it's necessary, and I just included it for completeness.

If somebody cares enough to express their opinion through a complex, multi-optional rating system, then they care enough to post all that needs saying in the existing comments.

True, but there's a difference: computers can understand ratings, but they can't understand comments. Ratings like I described would allow people to search and browse based on rated end-product quality, writing quality, etc. When I search for something specific (and I try to search thoroughly), I usually get a lot of low-quality, unhelpful results that just get in my way and waste my time. Even just separating the presentation quality and product quality into separate ratings might help with that.

(Tangent: I hope it's not the case, but it seems like you and/or others might think that instructables should not be ranked or compared numerically. If you don't think that, you can ignore the rest of the paragraph. If you do think that, I strongly disagree. Some are just better than others, and we shouldn't pretend otherwise. LED Throwies is unarguably better than BLOW UP A GLASS SNAPPLE WITH YOUR HAND!!!!!, for example.)

So, to sum up, section 2.2.1 can be completely ignored, and complex ratings do have some advantages over comments. I hope this has cleared up my intentions on those two points. :)

Select as Best AnswerUndo Best Answer

user
wotboa (author)2012-12-14

I agree that most ratings systems on the web have become meaningless. Your five-star method works well as a project climbs toward 5, but you realize that most projects are rated over 4.5 with, as you say, the second decimal point deciding the tie-breaker. (At what, 0.2% full scale?)

If viewers are inclined to give any project a rating, encourage them to do more than simply push a button. In other words, discourage the lookie-loos who have no opinion. Any meaningful rating should be thoughtful and complete.

Ian01 posted a comment Nov12:
"I suggest the following dimensions: quality of finished product, inclusion of proper techniques, writing quality, photo quality, completeness, ease of following instructions, innovation, accessibility*, etc."

Ian01 has made a great preliminary list (5 stars!). Take a poll to get a consensus of what facets viewers might think are important for a project to receive a high rating. With six or eight or ten categories, the viewer who truly wants to give a rating will take the time to do so. The fly-bys will opt out and move on.

Maybe you can out-wiki Wikipedia and start a new trend. Make ratings meaningful, not just push a button to vote. (Then sell your new 'ratings software' to other sites wanting to use it. With embedded links to Instructables, of course. Heheheheh.)

Select as Best AnswerUndo Best Answer

user
wotboa (author)wotboa2012-12-14

I made a short list of some of the categories that I use before starting my own projects.
Perhaps the list will spark some interest in creating a new 'category' rating system.

Parts/material availability
Tools required (related to average home workshop)
Skills required to finish the project
Cost (time and money) versus usefulness
Complexity versus usefulness
Difficulty versus usefulness
Safety - assembly techniques
Safety - materials
Safety - finished product

On the Instructables project page also rate:
Written instructions
Photo quality
Innovation of design
Level of difficulty
I want to build this

Select as Best AnswerUndo Best Answer

user
canucksgirl (author)2012-11-20

I was just looking at my YOU page under "Following" and I'm now confused...

If we removed the Star Rating System, why are there NEW projects showing a Rating that were published AFTER the site changes? Some have ratings, some don't. I thought it might have something to do with the number of times the projects were selected as "favorites", but that can't be the case because some Instructables with NO Rating have been favorited...

Any idea why or how some projects are getting a rating without a rating system???

Select as Best AnswerUndo Best Answer

user
Jayefuu (author)2012-11-16

I saw this on the oatmeal today. I thought it relevant to some people's comments about the new lack of allowing giving negative ratings.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Jayefuu2012-11-16

Haha, I haven't Oatmeal in ages!

Select as Best AnswerUndo Best Answer

user
The Green Gentleman (author)2012-11-13

I really like the idea of the "I made this" and "want to make this" buttons.  That sounds like it would be highly complimentary to authors, and would allow me to give a little more feedback than just favoriting something.  How about a tribute page for each instructable, where members can post pictures of their own variations and attempts?  I've been impressed with some of the variations given in the comments of (for example) MrBalleng's instructables, but going through a couple hundred comments to find pictures is a little daunting.  Also, don't use "it" maybe so it doesn't sound like braggin' or "missed opportunities" on Craigslist.  Or maybe do.  I could see people being very entertained by hitting the "I want to make it" button over and over again.

Select as Best AnswerUndo Best Answer

user

I would love a "want to make it" button.

Select as Best AnswerUndo Best Answer

user
bajablue (author)2012-11-13
user

I miss "Project Of The Day".

Select as Best AnswerUndo Best Answer

user

The Project Of The Day is still alive! You can see it by hovering your mouse over the Explore tab up top.

Select as Best AnswerUndo Best Answer

user

Thanks mikeasaurus, that made my day!

Select as Best AnswerUndo Best Answer

user
makendo (author)2012-11-13

Glad to see the end of the rating system. The addition of favourites is a nice touch. A metric for non-members like FB shares or tweets might be useful also (I don't know if you even have this data, but it at least allows would be raters to choose their own brand of endorsement).

Select as Best AnswerUndo Best Answer

user
JamesRPatrick (author)2012-11-13

How about a +/- rating system in which the displayed rating will never go below zero? For example:
How to Build an Ark gets 3 upvotes and 7 downvotes. The rating would read "0" until the invisible downvotes are canceled out by more upvotes.

OR

A +/- rating system in which the rating shown is a colored bar(just like YouTube), but grows in thickness with each rating until it hits a maximum thickness(say, 50 votes). People would instantly know the rating(Percentage, NOT weighted) of the instructable and the validity of the ratings.

OR

A +/- rating system in which a user will be required to give an explanation for downvoting, presented by a list of checkboxes.

Ideally we would use YouTube's rating system.
If lots of people think my instructable is shit, I want to know about it with absolute clarity so that I can improve upon myself. Negative reviews are extremely meaningful, and hiding them doesn't help anyone.

Select as Best AnswerUndo Best Answer

user
strods (author)2012-11-10

I'm fine with the favorite and agree with the reasoning. It would be nice to have a talley of "I made this". if we have 10 "Made" besides the orginal author and the comments are all fair or positive, then the chances are it is legit. But if there are a lot of comments or recommendations then maybe the 'ible wasn't the best.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)strods2012-11-10

An "I made this" button is a regular request, and one I heartily support.

Select as Best AnswerUndo Best Answer

user
Lithium Rain (author)Kiteman2012-11-10

I feel strongly about the I made this button. Currently, Instructables is not about making per se so much as inventing or creating; only something that can be made into a *new* instructable will really give you something to talk about/share on the site. To me, it would be just as awesome to have a stat on how many instructables you have recreated, recognizing making - not inventing - as the true activity driving the site.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)Kiteman2012-11-10

(Although I predict, on that metric, that recipes would become the main focus of the site...)

Select as Best AnswerUndo Best Answer