Author Options:

Faster Overnight Website! Answered

Hi All,

Many of our most dedicated community members have complained about various errors occurring for logged in users during the night (for us, in SF). This has been consistent for a long time, and the code causing problems has been around for years. It precedes everyone currently on the dev team. We have automatic database maintenance tasks that run overnight, some of which are responsible for these problems.

We have heard your pleas! I believe we have just one feature that will break if we completely remove all of these tasks. So we're going to try an experiment this weekend. We hope for these results:

1) Posting should be just as smooth from 12am to 6am PST as it is during other times of the day.
2) These little stats icons will start to show inaccurate data:

This will not affect total view counters.

If our theory proves correct, we'll proceed to find another data source for that button. And you who are awake when we are asleep will have a much more usable site going forward.

Just wanted to give everyone a heads-up, since we're deliberately introducing a bug.... but I promise it's for your very real benefit!

~ Your Friendly Devs


So... anyone notice any difference this weekend?

Just made posting that totally disappeared :(
No error, no warning, just back to what the page looked after loading it.
Had to create the posting again.
Happened less than 5 minutes ago if that is of any help.

If I can actually post this comment, then that is a good sign.


When it was working, it did seem faster. Otherwise, see my messages.

This just in:

Our hosting company has just informed us that they are replacing some network hardware which may affect users of Instructables. The maintenance window is from 4pm - 8pm PST on Sunday, July 12. We expect a brief outage during that time, and possible intermittent slowness affecting primarily logged-in users.

So our test to speed up the site may feel as though we sped it up in one place only to slow it down in another. The slow isn't us :)

It's always the way, isn't it? You buy a faster car, and they dig up the road...

Hi Phil. I note that the number of photos you can now include on a step before they get truncated and moved to the dreaded "Show All Items" status (i.e. never get seen) is now 4 instead of 6. I'm curious - is this because (a) you want authors to provide less photos or (b) more steps? Or is there some other reasoning? It's a headache to design an instructable when this keeps changing, because I already felt like I was struggling to get it down to 6 and 4 will have me sulking again and going back to the style I tried in https://www.instructables.com/id/Built-in-wardrobe/. Well, probably not. It was a real pain. But I am interested in the why.

A small handful of photos and about 200-400 words of text per step would seem to be the sweet spot according to the numeric engagement metrics we see. As for the number of steps, the sky is the limit - it should tell the story of your project as clearly and energetically as possible. What this effectively boils down to is pixel height of the page: audience members seem to like scrolling a particular distance through the page. Less and they bounce, more and they get bored and never click to the next step. We're trying to help everyone find the sweet spot, since it's good for everyone if everyone hits it.

Apologies this messed up your deliberate photo arrangement though. That wasn't our intent, but it's hard for me to guarantee we'll leave this alone in the long term. If you want to screen cap or link to a particularly not good photo set condition that we should improve, I'll happily share it with folks around here to make sure we cover all common cases. We're very actively invested in this part of the site.

Phil - presumably you are aware of the frequency with which readers hit "Show All Items". Mind sharing that with your authors?

I published https://www.instructables.com/id/How-to-install-a-wall-hung-vanity/ today, in which I was careful to add no more than 3 photos per step. It worked OK. However, it's less clear to me how to improve past projects. Delete photos down to no more than 3 per step? Add steps? Suggestions welcome.

This data is surprisingly slippery. Suffice it to say, it's not a lot. Sometimes 1 or 2% of users will click "view more" on the first photoset on the page, but this varies widely across pages (down to numbers so small I don't see them in analytics anymore) and drops significantly for photosets other than the first one on the page.

Some compelling images in the first photoset do get significant clicks, I've seen up to 25% sometimes. From there, the viewer will find himself in a slideshow and may well click through all the images in the photoset, not just those shown. I've found that the most-clicked images in photosets tend to be good images with more detail than you can see at a small size. One of my Instructables actually does well in terms of getting users to click for larger images, and enter the slideshow: https://www.instructables.com/id/CNC-Scissor-Chair-Plywood/

Is it possible the term "Show All Items" on the expand button throws people off?

I've had users wonder how I did key parts of project, even when I clearly showed how in a series of photos (with detailed photo notes). It turned out they simply hadn't clicked to expand to examine all the photos of a step.

1-2% can be pretty effectively approximated as none. Good to know, because this (now) means >4 photos/step = waste of effort.

I guess it depends on context.... in the world of engagement measures on the internet, that doesn't sound tiny to me. Also: we see a larger than 2% bump in the number of people who make it to the bottom of the page because we limit the number of photos per step. Fewer will see your 5th photo, but more will see your next step (this is the purpose of the limitation).

If 2% of our audience went pro, we could easily spend half our time developing pro-only features.

But it's true in the abstract I guess. Lots of photos per step would be better deployed as lots of steps.

Here’s what
I don’t get. Instructables.com relies on authors providing free content,
yet you’re not upfront about what you know makes a good project. You just
change stuff and you leave us to guess. I now know that any photos that get
auto-hidden essentially don’t get seen, and that if we want to include lots of
photos we should do so by adding additional steps. That’s useful, and it makes
me wonder why we have to drag this information out of you and why it is buried in the forums. Your interests and
those of your authors largely coincide – we all want better projects and more
traffic. Why don’t you tell us how to achieve that end? We have some metrics –
we can look at views and favorites and comments, but that’s it. You’ve killed
the stats so we don’t know anything about where viewers are coming from, much
less how long they stay. I have no idea whether a view is meaningful – one of my projects has 370k views but only 6 comments. Is everyone who visits just
bouncing straight out of there? Or is it especially effective at getting views because
I deliberately circumvented your default layout and took care of it myself? Or
is it just an SEO powerhouse? I have no idea.

I get the
distinct sense you and your team are rolling their eyes at the stupidity of authors
doing dumb stuff that you can see plainly from your stats is ineffective, and
you are forced to intervene on our behalf to make them better. But throw us a
bone! TELL US what works and what doesn’t. Don’t just make a change to the way
projects are presented and say nothing about why you did it. We’re not psychic.
We have none of the information you have at your fingertips. Deploy some of it
to the people who are writing your content.

We do share "what works" for most people, it just appears that it's not enough detail for this crowd. Here are some examples:





I wish we knew how to maximize everyone's views in all cases. That would certainly be information worth sharing! We don't know; we just run an endless series of experiments, tweaking things for everyone's benefit as we can. Many things we discover accidentally. This particular issue -- the scroll length of the page -- was a hypothesis we formed after doing letting everyone see the "All steps" view by default. Most of us prefer the "all steps" view, so we figured others would, too. Turns out they don't have the attention span to get to the bottom of the page, so we turned around and started shortening the page. The difference here to a single Instructable may be small, but when aggregated over hundreds of thousands of Instructables, it matters. That's usually why we choose to make changes first, and ask questions later.

As you point out, we're with you! I'm trying to participate more in the forums as a way of giving this elite group of authors some access to more detail. An answer to the question "why," which you asked above. I can't commit to a blog that lists every hypothesis and failed experiment we discuss, but if you have questions, please ask.

As an aside: Referral stats will be returned to the way they were in 2016. They're powered by Google right now, which I thought would be better than our own data. As observed, it's way worse. Message received.

I'm not asking for a blog that lists every hypothesis and failed experiment. But when you make a change that impacts negatively on the majority of my past projects (because they were optimized for a different default image number), I think you need to justify that to your authors. Something like "Hey, our stats tell us it is more effective to have fewer photos per step, so we're hiding all but 4 of them. In future, consider writing projects that have more steps and fewer photos per step, because only ~2% of readers will use the "Show More Items" button."

There is a certain irony in the fact that your "Feature Worthy" instructable uses a collaged photo layout rather than the default. I suggest you add the above information to that instructable, and also say "An effective way of presenting many photos in a compact form is to use a collaging tool to combine them in a single image".

You asked, we listened!

Looks like our experiment with shorter pages didn't help engagement quite as much as we expected, and since there's a clear preference for more images, we'll be bringing back the third row of photos in photosets.

Given the timing and our desire not to break things right before holidays, we'll probably release the change just after new year's. But I wanted to let you know as soon as the decision was made, so you don't spend time adjusting Instructables. I hope it's not too late!

ah, that's great news. Thanks for the heads up

Ok, sounds good. I'll make note of it in the forums the next time we make substantive changes to the Instructable page layout. "substantive" is a little bit of a spectrum, so if we make smaller changes that you notice and would like to understand, please reach out.

I'll talk with the editorial team about mentioning collages in the feature worthy Instructable. Pixlr does what pixlr does (it's a separate product), but for those who want collages, I'm sure there's a resource somewhere we can point to.

Thanks for hashing this out with me!

What would be less confusing?

Perhaps simply "View More" as you called it above. Or maybe "Show All Photos," or "Additional Photos"?

I think the
best example of one which I’ve thought very carefully about photo layout and
which you have now messed with is


Note that
there are certain numbers of photos that work. 1. 3. 5. These generate (or at
least did) a fairly attractive layout; one big one, and 0, 2 or 4 smaller ones.
2, 4 and 6 all look poor IMHO, and you’ll see in these cases I have made
collages to compensate. Your new layout has now rather spoiled my steps with 5
photos in them, because now only half the visual story is being told (and in
some cases, the punchline is missing – step 13 now hides the sharpened ax,

Your new
design ensures I will not ever bother to write a step with more than 4 photos
in (because you hide subsequent ones), and because I *really* don’t like your
layout for 4 photos (1 big photo and 3 tiny photos doesn’t work for me) I likely won’t write one with more than 3. FWIW, see steps 3 and 9 for how I
would prefer to see steps with 6 and 4 photos in them, respectively.

Here’s the
thing. Why would an author include photos that are just going to be hidden?
Might as well just leave them out. So your design decision will necessarily
reduce the photographic content on the site. Maybe that’s OK, I don’t know, but
I feel it’s an inevitable result.

also has examples of steps where I’ve tried to get around the default layout
and optimize to 6 or less photos. It really needs revisiting, because a bunch
of those steps now look weird with the no-more-than-4-photos restriction, at least to me.

Pseaton / makendo,

Here's a thought - why not let [pro] authors select how many images are visible before the "show all"?


Get the PixlR people to add a tiling option, where a small number of photos get fixed together as one image then, no matter how many images are viewed, their relationship to each other remains fixed?

That's a good idea, but it's fair to say that when I suggested authors ought to have some control over this (last time they changed it), that suggestion was met with considerable resistance and indignation (and ignored). I've kind of resigned myself to a one-size-fits-all policy, but would really like the defaults to work better.

In principle I think you're right, it's something we could consider exposing for pro authors. It's a fair bit of work though, maintaining separate layouts (across 4 platforms, remember) for Instructables published by authors meeting various criteria. Probably more work than is likely to happen for a change that would be meaningful to so few, in all honesty.

If you do want to cut us out of the loop in terms of layout control, pro authors are able to do so. You can "inline" your images in the html by uploading a file to Instructables, clicking through to the file page where you can get the url for the image, then including that via an <img src="xxxxxx"> tag in the "code" mode in the editor. I don't really recommend this in general, but for power users who want more control than we allow through the UI, this gives you more layout control than just about any other site on the internet.

I think I'd rather stick with the collaging I used in https://www.instructables.com/id/Built-in-wardrobe/. I note that mikeasaurus has taken this approach in his recent cheese-cutter project, presumably to dodge the photo limitation, so I'm guessing it's sound (if more work). Any chance of Kiteman's Pixlr suggestion of auto-collaging getting traction?

23:44 EAST it happened again.
Made a comment and the result was a virgin website looking the way like I loaded it before.
No sign of my comment, no error message, nothing.
As there was a quite a delay before all was gone I copied the text into the clipboard.
That way I was able to paste it for the second try of posting, which worked just fine.
There is a bug somewhere....

<happy dance>I have just successfully posted, twice, in the time-slot when the forums usually give me a Backend Error response!</happy dance>

Also, spam-busting powers are working again!

Not all fixed it seems...
Just tried to check a thread that I replied to earlier and instead of it I got the error page.
It is nice to see that something unexpectedhappened, but I don't think this message helps solving the issue.
After loading the thread again it showed up but my comment was there twice.
Decided to delete one but both replies disappeared.
Happened less than 10 minutes ago...

Hm, that's frustrating. Suggests there are several different issues at play here; makes testing theories challenging. But generally I'd consider a change in the error you see to be progress. It's possible we made something worse in principle (so you're not even getting as far as you're used to) but since the change was to literally turn something off, that seems hard to believe.

In this case, your comment did go through eventually, but you saw an "Instructables is down" page when you hit "Make Comment" ? That's a more straightforward timeout, versus the database errors you saw before.

If I am not mistaken I saw something like "Instructables is experiencing some difficulties..." - next time I will take a screenshot to be sure!
Thing is that it seemed to be fine for the past few days, so it might have been a one off error that is not really related.
Will try again tomorrow around the same time and see what happes.

No problems at all last night.
No messages lost, no errors, just worked :)

Thank you so much for this!
Not just the attemp to fix it but also the information itself.
Can't wait to see the "Backend read error" gone :)

happy to share! I'll try to make more regular small posts in the future. Was there any improvement this weekend?

Well, I only had one posting that was hanging for a few minutes but still published.
So I would say it is better, although I was not too active at the usual times.
In any case I did not get any backend read error, which is a big improvement :)

That's strange the comment box wasnt here yesterday but it is here today

Yeah, I forgot to allow comments originally. Not intentional.