229Views33Replies

Author Options:

Instructables content - moderation and peer review Answered

Following a discussion in this forum I thought is might be a good idea to create a new thread.

Adrian Monk
"That's one reason I'm a little apprehensive about the future of Instructables-you don't want a police state, but if it gets humongous you start to lose control unless you moderate well. It's quite selfish, but that's why part of me wants Instructables to stay (relatively) small, and not become a phenomenon."

Kiteman
"That's the thing, we aren't heavily moderated - there are no people who have a specific moderating responsibility."

Lemonie
Some time ago, people were talking towards things like peer-review (I can't remember who). Allowing anyone to post anything does have some drawbacks.

As a suggestion - new posts would submit to a "review" stage, where administrators and their delegates would have the opportunity to click "publish" or "reject".

"Publish" would move the new post to it's normal published state, "Reject" would force a comment to be added to that effect, leaving it in the "review" stage.

It would mean changing the system, but a group of people would be filtering new posts.

It would preempt having to flag things and post comments asking for withdrawal or fixing afterwards.

With a group of people having the option to publish, no one person would be able to hold a new post back, and the group would be able to see "reject" comments and add to them. Users posting would also be able to see comments against their submitted posts.

By "the group" I mean people who are flagging and commenting already.

Safety catch: after X-days with no action, posts publish automatically.

Forum for general ideas, do with it what you will...

Regards

L

34 Replies

user
super moderator (author)2009-05-21

There should be dedicated moderators who can go around flagging items but their flags have a higher priority OR they they can temporarily ban items pending future review. This would not work on it's own but it would be good with the other measures proposed.

Select as Best AnswerUndo Best Answer

user
frollard (author)2009-03-30

I'm in on this late: How about a buffer page? You hit publish your ible: Based on what robot finds, it could do a few checks. For example, spelling check, grammar check, image 'sharpness' check, word count, etc. "STOP! We really think you should give it a once over for spelling, grammar, and photo comments. Read it through and see if you could follow the steps with no knowledge of what you're doing. Once you're totally sure, hit publish again" Another thing, I'd like to see the reactivity of the voting more representative of a true average, without such a strong weighting toward 2.5. Voter apathy is huge, and thus we have a TONNE of unrated or 2.5 rated ibles. Encouraging more people to vote and flag is important as well. Adding extra 'advanced' voting categories would be cool. Categories like instruction quality, picture quality, difficulty, overall. Moving the 'rate' box (or a duplicate of the existing rate box) to just above the comment box would be a blatant reminder for users to rate the ible. A reward for voting with something as simple as a post-count-esque title in your profile for how many ibles you've rated, and your average rating (to show if you're just up or down thumbing ibles for no reason.

Select as Best AnswerUndo Best Answer

user
lemonie (author)frollard2009-04-23

I seem to have missed this comment.
There's already a Robot filter for missing images and empty steps, adding in a spell check might be a bit erratic (lists of parts, tools, code, proper English spelling etc.)

The site default rating (as illustrated by my data here) is actually 3.0

L

Select as Best AnswerUndo Best Answer

user
NachoMahma (author)2009-02-17

. I think something needs to be done, but I'm not sure what needs to be done. There do seem to be a lot of iBles that aren't quite up to snuff.
. I think Kiteman is on the right track - it should be reactive (vs proactive). The resources required for Robot to quickly review and approve all iBles and posts is just too much.
. I'd like to see something like a Reverse Feature Team. If a RFT member flags an iBle/post (or maybe have a threshold where 2-3 members have to flag), it gets kicked to a holding area for review by Robot.

Select as Best AnswerUndo Best Answer

user
gmjhowe (author)NachoMahma2009-02-17

I agree, this is a feature that we keep suggesting, but still doesn't seem to have come into fruition. All they would need to do, is pick up on us from the inner sanctum, which actually includes most the people on this thread.

Select as Best AnswerUndo Best Answer

user
NachoMahma (author)gmjhowe2009-02-18

> All they would need to do... . Well, it's not quite that simple to implement. Code has to be written and/or modified, then debugged. I think something like it will eventually be necessary, but it won't happen overnight. . And it seems to me that they need to be much more careful in selecting users that will have the capability of making an iBle/post disappear than those that just flag an item as Featured.

Select as Best AnswerUndo Best Answer

user
threecheersfornick (author)2009-02-17
I have definitely noticed a decline in the average quality in new instructables.

Things I'm noticing:
  • bad spelling / grammar
  • "stolen" pictures
  • duplicates, duplicates, duplicates

Other than that, things are just like they've always been.

My suggestion is that these things are added to the list of things you can flag an instructable or comment for. If an admin thinks that they are a problem, and the instructable in it's current state is not a positive addition to the community, the instructable is unpublished and the author is given the chance to fix it up.

I strongly believe that people should be able to post instructables about whatever they like, knex weapons included.

Select as Best AnswerUndo Best Answer

user

There are some things that pop up, that really ask the question "why did you bother?" Even for the sake of having one it hardly seems worth the effort - testing the system?

Bad English is extreme laziness, when even things like browsers will spell-check things.

Unoriginal pictures tend to get used with verbatim reproductions from other sites (e.g. Wikihow) or otherwise 'empty' posts, so they're symptomatic of a low standard.

I think there's a lot of flagged stuff that hasn't been addressed?

L

Select as Best AnswerUndo Best Answer

user
lemonie (author)2009-02-16

Checking back on The-Worlds-worst-Instructables only one of the top 6 seems to have gone

L

Select as Best AnswerUndo Best Answer

user
DJ Radio (author)lemonie2009-02-17

How to blow up a glass snapple with your hand doesnt have the lowest rating anymore.

Select as Best AnswerUndo Best Answer

user
lemonie (author)DJ Radio2009-02-17

Well please tell us which one does? L

Select as Best AnswerUndo Best Answer

user
DJ Radio (author)lemonie2009-02-17

"extreme mini knex gun" has a lower rating that "blow up a glass snapple with your hand". Its .03 rating points short of a less than 1 star rating.

Select as Best AnswerUndo Best Answer

user
lemonie (author)DJ Radio2009-02-17

Hmm, that's fairly recent (and fairly crappy) but about 2 years after the Snapple bottle. Prior to that you could give -ve ratings, which is different to 0.5 minimum.

That Knex is poor but not really bad. And both are still there...

L

Select as Best AnswerUndo Best Answer

user
ll.13 (author)lemonie2009-02-17

I miss the old rating system, however things must go on. =]

It doesn't stop falling... -46 it's still the most moronic post on the site (?):
BLOW UP A GLASS SNAPPLE WITH YOUR HAND!!!!!
https://www.instructables.com/id/EVBGUDRGU9EUBYRQ3B/

Select as Best AnswerUndo Best Answer

user
lemonie (author)ll.132009-02-17

Yes, that one's still there - has nobody flagged that?
I'm amenable to letting a 'joke' kick around a bit for the laugh, but not that long.

L

Select as Best AnswerUndo Best Answer

user
bumpus (author)2009-02-16

Kind of like flagging, but it would be temporarily removed, viewed by a certain group of special people, and either republished or rejected.
Only for instructables, and forum topics.

That is, if someone asks it to be reviewed, kinda 'Flag and ignore'

Select as Best AnswerUndo Best Answer

user
lemonie (author)bumpus2009-02-16

A lot of stuff slips by, and adds to the general junk content. If people are only picking up on new stuff as it comes in, some of it gets missed. Do I have to go back and flag stuff, or are we only interested in what happened in the last 24 hours?

L

Select as Best AnswerUndo Best Answer

user
bumpus (author)lemonie2009-02-16

Oh, I was just thinking for future ideas. You can if you want to though. :-)

Select as Best AnswerUndo Best Answer

user
Kiteman (author)bumpus2009-02-17

Flagging would work - maybe three (5? 10?) flags and it's automatically hidden until reviewed? That would cover the site 24/7, because there is always somebody online, somewhere.

Select as Best AnswerUndo Best Answer

user
Tool Using Animal (author)2009-02-17

Groupthink pre-censorship, count me out of Instructables if any such system is instituted. I'm not going to post if I have to think "Gee, is the hive mind going to like this?" each time.

Select as Best AnswerUndo Best Answer

user

That is my fear - the site is currently attractive to the independent free-thinkers that make the better / most original Makers. Pre-mod would drive them away.

Select as Best AnswerUndo Best Answer

user
PKM (author)Kiteman2009-02-17

Doesn't that depend on what exactly is being filtered for? If we were just filtering rubbish content (obvious spam, Instructables that no thought has gone into like the classic "ghetto hacks") then most currently published Instructables would get through, it would just filter out the odd thing that is usually un-published by mods reasonably quickly anyway. If it were to be extended to "objectionable content", however, there would be problems- you just have to see how much controversy arises around well-written and well-meaning instructables that happen to feature a little nudity or something to do with dead animals to imagine that eventually good content would be vetoed by ideologues. Ultimately we would either end up with excessive arbitrary editorialising (as TUA mentions) and lose much of what makes the site good, or a more labour intensive version of what the site does already.

Select as Best AnswerUndo Best Answer

user
Kiteman (author)PKM2009-02-17

So you're agreeing with me? That is, you agree with reactive moderation (check it when somebody complains) rather than proactive moderation (hold it back from publication until it has been checked)?

Select as Best AnswerUndo Best Answer

user
PKM (author)Kiteman2009-02-17

Well.. in an ideal world I think pre-moderation would be good, but ultimately there are sufficiently many pitfalls for it not to be a very practical idea. Actually, I believe it already happens to a degree- the staff might not comment on exactly how but the "red flag words" list was mentioned.

If it was implemented
- purely on the basis of non-'Ibles/spam/plagiarism (mostly objective) rather than objectionable content (subjective)
- open to the feature team and mod squad as well as just staff
- non-fatal (ie the list of currently provisionally rejected Instructables is visible to other mods)
- multi-level (passed Instructables are published, rejected ones go to staff for return to sender or publishing so the staff have minimal workload but ultimate control)

.. then it might be feasible, but that is a large and complex scheme. While I'm not a great fan of "if it ain't broke, don't fix it" as a universal maxim, I don't think the current system is bad enough to warrant a fundamental shakeup like this- I don't actually think it's bad at all. The level of unwanted content seems to be actually so small that this proposed layer doesn't seem worth the hassle and potential for driving away valuable contribution.

/essay

Select as Best AnswerUndo Best Answer

user
lemonie (author)PKM2009-02-17

As your opening, I was thinking of a process where someone (out of many who have the option) waves the OK stuff through but says "no" to the spam, sub-standard, not-Instructables and plagiarism. These would still be there to see in the reject-pile, anyone else (with the option) could give the OK if they thought the judgment unfair. And comments could be made "off-air" so to speak L

Select as Best AnswerUndo Best Answer

user
Kiteman (author)2009-02-16

No, I'd much rather reactive moderation than pro-active. To do otherwise would rob the forums of an essential spontaneity.

Similar measures were instigated by the BBC on several of its message boards, and those boards died.

Plus, given the international usage of these boards, you would need a team active round the clock. Presuming that moderators would be recruited from the current "core" regulars, those who had proven themselves reliable, then you would probably need to recruit the entire core, especially to cover the European time-zones.

Appoint moderators if you must (I guess we'll need them eventually), but make them reactive, with the power to remove or hide inappropriate material when other members object to it.

If we force members to wait before their threads even see the light of day, never mind get a response, then we will lose those members.

Select as Best AnswerUndo Best Answer

user
lemonie (author)Kiteman2009-02-16

Just Instructables then? I am thinking of e.g. things that get lifted off the internet and republished verbatim, many of which have not yet been pulled. L

Select as Best AnswerUndo Best Answer

user
Kiteman (author)lemonie2009-02-17

Maybe that requires an extra flag option - copyright infringement (which would require extra details to be added). (Were you here when somebody posted the original Throwie as their own idea? They had ripped it off verbatim from another site that had ripped it off another site that had ripped it off here. Kind of funny.)

Select as Best AnswerUndo Best Answer

user
lemonie (author)Kiteman2009-02-17

I may have been, like the Anarchist's Cookbook stuff that's all over the internet. (And people still said "here's my e-mail can you send me a copy") And Wikipedia, and Wikihow etc. L

Select as Best AnswerUndo Best Answer

user
Goodhart (author)lemonie2009-02-16

You mean in a similar way that an Ible is reviewed now, when it is entered into a contest?

Select as Best AnswerUndo Best Answer

user
lemonie (author)Goodhart2009-02-17

MM, that's more of a judging process. I was thinking of a tick-box push button process where someone waves it through or says "no, because-" General ideas for discussion, not a plan. I know people have in the past put up Knex forums asking for new makes to be previewed there, to "cut-out-the-crap" so to speak. But that only works if people do it. L

Select as Best AnswerUndo Best Answer

user
kelseymh (author)2009-02-16

Hi, Lemonie. I think the problem with idea is volume. The sheer number of created instructables would require a dedicated staff member (maybe multiple) to do the filtering. They already have the Robot catch and hold I'bles with specific keywords or combinations of keywords, for review before publication. That process can take more than a day if someone publishes on Friday night California time (and G*d help you if you publish right before Christmas). There are numerous posted threads from people whining that their I'ble didn't appear instantaneously. With a global review system as you propose, that problem would only inflate dramatically.

Select as Best AnswerUndo Best Answer

user
Lithium Rain (author)2009-02-16

I'm not sure...if it gets bad like youtube, it'd be better than letting it go to pot, but instinctively this seems like not such a good thing. However, as Kiteman has said I think it'll have to be done eventually. I'm on the fence here...

Select as Best AnswerUndo Best Answer