366Views54Replies

Author Options:

4 New Pro Features - UPDATED - 1, 2 & 3 now complete! Answered

As some of you may have seen, recently I've been playing with Perl to manipulate Instructables and gather data from profiles. More recently I developed these 3 perl scripts for use by the pro community. Since they proved fairly popular we thought we'd move them to the public forums so you can see what we've been up to. (By we I mean pro members, not staff, I'm not staff)

I thought of 3 new potential features I could develop, this time only available to pros. The fourth you can read about below, it's more of a teaser for something Lithium_Rain and I are working on.

1) Download all own instructables: Backs up ALL of your instructables to your hard drive as pdfs.

2) Download all favourite instructables: Downloads all your favourite instructables!

3) Download all X's instructables: Big fan of Kiteman? Or Ewilhelm? Or Canida? Or even Jayefuu? *gasp**shock**horror*. Going away for a week? No problem. Download all of one person's instructables as pdfs in one easy step and be set for quality reading for the duration of your trip!

4) This is more of a teaser for a work in progress. It should be out in the next month if Lithium_Rain and I aren't too busy. It's more of a server run application that uses ibles to gather interesting data. More on this in a few weeks maybe.

Using the Perl scripts  yourself: (you need to be a pro member)
1) Download Perl from www.cpan.org
2) Install Perl
3) Download WWW::Mechanize from www.cpan.org. This will let the script access instructables. It's used to fetch web pages.
4) Install Mechanize by extracting it into your Perl directory (or just follow the instructions included with it)
5) Download download_instructables_pdfs.pl or download_instructables_favourite_pdfs.pl from this forum thread and change the extension from .txt to .pl
6) Edit download_instructables_pdfs.pl. You will need to update your instructables account name and password so it can log in. The variable $target is the name of the instructabler you want to download the instructables of. Those are the only 3 things you need to change. If you like you can also edit the pdf properties to include comments, larger pictures and other things. Depending on the amount of ibles you're doing, it may take longer. It takes about 3 minutes to download my 17 ibles.

Can't be bothered to install Perl but still want a copy of your instructables?:
1) Subscribe to me (for the effort it takes me to run the script)
2) PM me your email address and whether you want your instructables or your favourite instructables or both!
3) I'll run it for you and email you a zipped file containing your instructables.

OR - wait a few weeks and Lithium_Rain and I might have a web interface ready.

People who've had their ibles sent: Zieak, Goodhart, Kiteman, Jayefuu

Discussions

Anyone else want theirs done? Just send Gjdj3 his favourite 170 instructables :D

My parents say I need more time off the computer so... yeah! Sending pm now!

Unashamed "bump"

0
user
Gjdj3

8 years ago

Ah, this is cool! I can't believe I missed it before. I just subscribed and PM'd you.

Downloading them now :)

If moved to the non-pro forum does anyone think this might inspire some people to join?

Yes, if you persuaded Eric to make the ability part of the site.

If you make it accessible via downloadable code, or through aother website, then it will be used by non-pros with as much ease as pros.


But you need to be a pro to download pdfs. And since this uses the user's login credentials to access the site, if they're not a pro they won't be able to use the script. But knowing what's inside this thread (putting it in the non-pro area) might convince some people to join if they can see what these features are?

Ah, of course, you are correct!

I forgot about the PDF restriction.


I'd have to scrape the html and construct my own pdf if I wanted to do this for free members.

Ouch.

I think you're best off not doing that.

Do you want to move this topic into the general forums, see if it would tempt a few more new pro memberships?

Yeh I'd agree, a) it'd be a lot of work b) I paid for pro already :p

If you can, then yes please. :D Wherever you think is most appropriate sir.

>moved to community blog<

(You may want to edit the OP to explain the topic to non-pros?)

OP updated :D After spending several minutes trying to work out what that obscure acronym was. :) Thanks Kiteman.

Subscribed, and Lira already has my email address.

(I'm not sure I'd dare install the thingie, I might break thye intertubes. Again.)

Thanks =]

Suprisingly all 100 of yours only come to 9.5Mb zipped. Ready to whiz over to you once I get your email address. Lira's away for a few days so.....

OK, stupid question - what software do I need to open a "rar" file?

O cool. I'll switch to that then, winzip is a pain with its "Your free trial is over" rubbish

WinRAR should do you

http://www.rarlab.com/

One comment, Jayefuu. Macs and Linux boxes come with Perl as part of the distribution, so its only the poor little Windoze users who should have to install anything.

Which version of Perl are you assuming in your code? MacOSX Leopard comes with 5.8.8.

Yeah, I didn't have to install anything. Ubuntu ftw. :D

Ummm..... I think I use ActivePerl5.10.something. I don't know enough about it to say whether it would work on previous versions.... but 8 isn't far from 10 :p

Linux/Mac users may need to install the perl module called Mechanize. That's a really easy bit though :D

Personally, I would like to see all three options available (I would use 1 & 2, and I am sure others would use 3)

My question, though, is "how"?

Would it be a piece of software we download and install?  A Firefox add-on?

I ask because (as Lira will affirm) I am no code-monkey.  I like "click the button and it works" concepts.

As an extra, would we be able to select options on the downloads (comments, etc), or would you set a default?

Oh, and...

Could you code be integrated into the site itself?  The concepts you outline would make great pro features that might encourage others to turn pro.

Is it worth talking to Rachel along those lines?

(And I do hope you aren't planning on drinking that OP straight from the bottle - it needs to be in a straight glass to fully appreciate it...)

I did indeed pour the OP into a glass after a few greedy sips from the bottle. :D

It would be implemented in Perl. At first release for those who are keen it would be a "install Perl, download script and run" jobby. At a later date if demanded I could turn it into a webpage that runs the perl script and emails the results.

With respect to it being integrated into the site, it COULD be, in the finished implementation by me it COULD be, but it wouldn't be the most efficient way of doing it. With database access it would be more than 100 times quicker, and the code would be a lot simpler to use too. The way this script would work would be by me logging in to ibles through the script, downloading the favourites page of a member or the ibles page of a member and then downloading all the pdfs with perl. If this was made into a site feature it would be far easier to write it in javascript and query the instructables database directly.

Go for it, then!

PM Rachel right now!

Ha ha. I think unlikely that they would give me database access, even if only to selected tables and read only.

If you don't ask, you don't get.

Do you think Adnams invited me to tour their brewery out of the blue?

Lira and I are going to get the demos ready first then ask we thinks. :D

Unless Eric or Rachel read this first and think it's a good idea. ^.^ *hint hint*

FWIW, you can
1) point to this thread for context
2) ask for as much of a database schema as the team are willing to give
3) write "pseudocode" for what your script would do- not the exact SQL, just "select all PDFs where the author is the selected user" or whatever
4) give code back to team, if they are willing to implement it then you can supply it in "fill in the blanks" style and it's a small(er) amount of work for them to put in the genuine database code.

database schema = relevant table and field names?

Yep.  The schema tells you what tables there are, what their fields and indexes are and how they relate to each other. (If you need more on "field"/"index"/"record", look up an introduction to databases, but you sound like you know what you're talking about).

Jayefuu's already answered it but what we were thinking is, we'd code it in perl and throw it up on a test site, and if people appreciated the feature after testing it we would submit the idea to the Team for site intergration(even help in the implementation if they desired).

0
user
zieak

8 years ago

I'll wait for a web interface if you get around to it.  But let me just say i like the work you're doing.  Keep it up!

Thanks. =]

I downloaded your ibles, they come to 2mb zipped. I can email you them if you PM me your email address.

Thanks for the sub!

It is my username at yahoo.  Thanks!

wwwwwwwwwwwoooooooooooooo ^5 !!
Well done, Im not much of a scripter so i think i will wait for the web interface :P
Great work

Since you're already subscribed to me you could just pm me your email address or give me it on irc and I'll email yours to you.

Scripts for 1 and 3 now complete.

#1 would be much appreciated, I always get bored and distracted when i try to DL all my pdf's, I'd be nice to have a local backup, just in case.......

Cool. Check back in a couple of weeks. I tried write number 1 last night but I was having problems getting my head around logging into the site with Perl. :( Some problem with cookies or some such.

I think your ideas are great!

It would be useful.  Develop them or perish.

Awesome Ideas !
All three are great but I think 1 would be fantastic, a really easy backup !
Also these would make great pro features :D