4 New Pro Features - UPDATED - 1, 2 & 3 now complete!
As some of you may have seen, recently I've been playing with Perl to manipulate Instructables and gather data from profiles. More recently I developed these 3 perl scripts for use by the pro community. Since they proved fairly popular we thought we'd move them to the public forums so you can see what we've been up to. (By we I mean pro members, not staff, I'm not staff)
I thought of 3 new potential features I could develop, this time only available to pros. The fourth you can read about below, it's more of a teaser for something Lithium_Rain and I are working on.
1) Download all own instructables: Backs up ALL of your instructables to your hard drive as pdfs.
2) Download all favourite instructables: Downloads all your favourite instructables!
3) Download all X's instructables: Big fan of Kiteman? Or Ewilhelm? Or Canida? Or even Jayefuu? *gasp**shock**horror*. Going away for a week? No problem. Download all of one person's instructables as pdfs in one easy step and be set for quality reading for the duration of your trip!
4) This is more of a teaser for a work in progress. It should be out in the next month if Lithium_Rain and I aren't too busy. It's more of a server run application that uses ibles to gather interesting data. More on this in a few weeks maybe.
Using the Perl scripts yourself: (you need to be a pro member)
1) Download Perl from www.cpan.org
2) Install Perl
3) Download WWW::Mechanize from www.cpan.org. This will let the script access instructables. It's used to fetch web pages.
4) Install Mechanize by extracting it into your Perl directory (or just follow the instructions included with it)
5) Download download_instructables_pdfs.pl or download_instructables_favourite_pdfs.pl from this forum thread and change the extension from .txt to .pl
6) Edit download_instructables_pdfs.pl. You will need to update your instructables account name and password so it can log in. The variable $target is the name of the instructabler you want to download the instructables of. Those are the only 3 things you need to change. If you like you can also edit the pdf properties to include comments, larger pictures and other things. Depending on the amount of ibles you're doing, it may take longer. It takes about 3 minutes to download my 17 ibles.
Can't be bothered to install Perl but still want a copy of your instructables?:
1) Subscribe to me (for the effort it takes me to run the script)
2) PM me your email address and whether you want your instructables or your favourite instructables or both!
3) I'll run it for you and email you a zipped file containing your instructables.
OR - wait a few weeks and Lithium_Rain and I might have a web interface ready.
People who've had their ibles sent: Zieak, Goodhart, Kiteman, Jayefuu
Comments
11 years ago
Anyone else want theirs done? Just send Gjdj3 his favourite 170 instructables :D
Reply 10 years ago
My parents say I need more time off the computer so... yeah! Sending pm now!
11 years ago
Unashamed "bump"
11 years ago
Ah, this is cool! I can't believe I missed it before. I just subscribed and PM'd you.
Reply 11 years ago
Downloading them now :)
11 years ago
If moved to the non-pro forum does anyone think this might inspire some people to join?
Reply 11 years ago
Yes, if you persuaded Eric to make the ability part of the site.
If you make it accessible via downloadable code, or through aother website, then it will be used by non-pros with as much ease as pros.
Reply 11 years ago
But you need to be a pro to download pdfs. And since this uses the user's login credentials to access the site, if they're not a pro they won't be able to use the script. But knowing what's inside this thread (putting it in the non-pro area) might convince some people to join if they can see what these features are?
Reply 11 years ago
Ah, of course, you are correct!
I forgot about the PDF restriction.
Reply 11 years ago
I'd have to scrape the html and construct my own pdf if I wanted to do this for free members.
Reply 11 years ago
Ouch.
I think you're best off not doing that.
Do you want to move this topic into the general forums, see if it would tempt a few more new pro memberships?
Reply 11 years ago
Yeh I'd agree, a) it'd be a lot of work b) I paid for pro already :p
If you can, then yes please. :D Wherever you think is most appropriate sir.
Reply 11 years ago
>moved to community blog<
(You may want to edit the OP to explain the topic to non-pros?)
Reply 11 years ago
OP updated :D After spending several minutes trying to work out what that obscure acronym was. :) Thanks Kiteman.
11 years ago
Subscribed, and Lira already has my email address.
(I'm not sure I'd dare install the thingie, I might break thye intertubes. Again.)
Reply 11 years ago
Thanks =]
Suprisingly all 100 of yours only come to 9.5Mb zipped. Ready to whiz over to you once I get your email address. Lira's away for a few days so.....
Reply 11 years ago
OK, stupid question - what software do I need to open a "rar" file?
Reply 11 years ago
. I use 7-Zip - OSS and free
Reply 11 years ago
O cool. I'll switch to that then, winzip is a pain with its "Your free trial is over" rubbish
Reply 11 years ago
WinRAR should do you
http://www.rarlab.com/
Reply 11 years ago
Thanks!
11 years ago
One comment, Jayefuu. Macs and Linux boxes come with Perl as part of the distribution, so its only the poor little Windoze users who should have to install anything.
Which version of Perl are you assuming in your code? MacOSX Leopard comes with 5.8.8.
Reply 11 years ago
Yeah, I didn't have to install anything. Ubuntu ftw. :D
Reply 11 years ago
Ummm..... I think I use ActivePerl5.10.something. I don't know enough about it to say whether it would work on previous versions.... but 8 isn't far from 10 :p
Linux/Mac users may need to install the perl module called Mechanize. That's a really easy bit though :D
11 years ago
Personally, I would like to see all three options available (I would use 1 & 2, and I am sure others would use 3)
My question, though, is "how"?
Would it be a piece of software we download and install? A Firefox add-on?
I ask because (as Lira will affirm) I am no code-monkey. I like "click the button and it works" concepts.
As an extra, would we be able to select options on the downloads (comments, etc), or would you set a default?
Reply 11 years ago
Oh, and...
Could you code be integrated into the site itself? The concepts you outline would make great pro features that might encourage others to turn pro.
Is it worth talking to Rachel along those lines?
(And I do hope you aren't planning on drinking that OP straight from the bottle - it needs to be in a straight glass to fully appreciate it...)
Reply 11 years ago
I did indeed pour the OP into a glass after a few greedy sips from the bottle. :D
It would be implemented in Perl. At first release for those who are keen it would be a "install Perl, download script and run" jobby. At a later date if demanded I could turn it into a webpage that runs the perl script and emails the results.
With respect to it being integrated into the site, it COULD be, in the finished implementation by me it COULD be, but it wouldn't be the most efficient way of doing it. With database access it would be more than 100 times quicker, and the code would be a lot simpler to use too. The way this script would work would be by me logging in to ibles through the script, downloading the favourites page of a member or the ibles page of a member and then downloading all the pdfs with perl. If this was made into a site feature it would be far easier to write it in javascript and query the instructables database directly.
Reply 11 years ago
Go for it, then!
PM Rachel right now!
Reply 11 years ago
Ha ha. I think unlikely that they would give me database access, even if only to selected tables and read only.
Reply 11 years ago
If you don't ask, you don't get.
Do you think Adnams invited me to tour their brewery out of the blue?
Reply 11 years ago
Lira and I are going to get the demos ready first then ask we thinks. :D
Reply 11 years ago
Unless Eric or Rachel read this first and think it's a good idea. ^.^ *hint hint*
Reply 11 years ago
FWIW, you can
1) point to this thread for context
2) ask for as much of a database schema as the team are willing to give
3) write "pseudocode" for what your script would do- not the exact SQL, just "select all PDFs where the author is the selected user" or whatever
4) give code back to team, if they are willing to implement it then you can supply it in "fill in the blanks" style and it's a small(er) amount of work for them to put in the genuine database code.
Reply 11 years ago
database schema = relevant table and field names?
Reply 11 years ago
Yep. The schema tells you what tables there are, what their fields and indexes are and how they relate to each other. (If you need more on "field"/"index"/"record", look up an introduction to databases, but you sound like you know what you're talking about).
Reply 11 years ago
PM them!!
Reply 11 years ago
Jayefuu's already answered it but what we were thinking is, we'd code it in perl and throw it up on a test site, and if people appreciated the feature after testing it we would submit the idea to the Team for site intergration(even help in the implementation if they desired).
11 years ago
I'll wait for a web interface if you get around to it. But let me just say i like the work you're doing. Keep it up!
Reply 11 years ago
Thanks. =]
I downloaded your ibles, they come to 2mb zipped. I can email you them if you PM me your email address.
Thanks for the sub!
Reply 11 years ago
It is my username at yahoo. Thanks!
11 years ago
wwwwwwwwwwwoooooooooooooo ^5 !!
Well done, Im not much of a scripter so i think i will wait for the web interface :P
Great work
Reply 11 years ago
Since you're already subscribed to me you could just pm me your email address or give me it on irc and I'll email yours to you.
11 years ago
Scripts for 1 and 3 now complete.
11 years ago
#1 would be much appreciated, I always get bored and distracted when i try to DL all my pdf's, I'd be nice to have a local backup, just in case.......
Reply 11 years ago
Cool. Check back in a couple of weeks. I tried write number 1 last night but I was having problems getting my head around logging into the site with Perl. :( Some problem with cookies or some such.
11 years ago
I think your ideas are great!
Reply 11 years ago
thanks :D
11 years ago
It would be useful. Develop them or perish.
Reply 11 years ago
Develop them it is!
11 years ago
Awesome Ideas !
All three are great but I think 1 would be fantastic, a really easy backup !
Also these would make great pro features :D