Introduction: Backing Up Your Instructables Inbox

Picture of Backing Up Your Instructables Inbox

This instructable contains instructions on how to set up and use a Perl script I wrote for Steveastrouk to allow him to download all of the messages in his Instructables inbox and open them in Thunderbird or any email client that can handle .eml files.

Once you've used the script you'll be able to refer to them while offline and organise your messages by date, sender, title and shoe size and never again worry again that Robot will lose your messages!

While I've tried to make it as easy to use as possible, this does require some installation and setting up of Perl and some modules. It's not really easy, but I've tried to explain how to use it as best I can and a relatively competent computer user should be able to figure out any problems. If you get it right first time it should take you no longer than 10 minutes to set up, and a second or two per message to download them.

The script uses the modules LWP and Mechanize to log in to your instructables account and then fetch web pages and then Perl regular expressions to separate out the useful content. Even if you don't want to download your entire inbox it might prove useful to see how to use Perl to scrape web pages for information.



Step 1: Installing Perl and Appropriate Modules

Picture of Installing Perl and Appropriate Modules

To use this Perl script you need an installation of Perl and the modules LWP and Mechanize. All of these should be free, if you're asked to pay for them you're doing something wrong.

I recommend ActivePerl for Windows. I've never used Perl on a Mac or Linux so I can't help you there. If you have success running this script on either, please let me know so I can update this with how you did it.

So you will need:

ActivePerl - an easy to install version of Perl
LWP - a Perl module used for fetching web pages
Mechanize - a Perl module that uses LWP and expands on it

Instructions for installing Perl modules are available here on cpan.org. Some users have had real problems with installing these and getting mechanize working, dependencies etc. All I can say is cross your fingers and toes, I had no problems so hopefully you wont either.

Step 2: Checking Your Installation!

If you've not used Perl, LWP and Mechanize before this step is highly important!

Please check Perl and the modules are installed correctly before trying to run my script. It'll save a lot of hassle running these two simple scripts to check all is working well before moving on.

To run a Perl script in Windows, navigate to the command line:

Start > Run > "cmd"

Then type Perl, followed by a space followed by the path to your test scripts. This path and name can be dragged and dropped.




If everything is working correctly:
perl_test.pl should print "Hi I'm working!"
mech_test.pl should print "I found instructables!"

You will need to download these scripts and then change the extension from .txt to .pl. Robot doesn't like me uploading scripts!

Step 3: How It Works...

These are roughly the steps that the code follows to get the result, a great big folder full of all of your messages....

It's not important that you read this, but I added it incase anyone was interested and wanted to follow along with the code.

1) It takes the values from the settings section and uses these to log you into your account
2) It checks that your password is right by visiting instructables.com/you and checking the title of the page contains your username
3) It visits your inbox and grabs the total number of messages (it's listed near the bottom)
4) Next the script iterates through your inbox pages, lapping up all of the links to your messages and storing them in an array for later
5) The script now has a big list containing the URL of each of the messages in your inbox. At this point it prints the total to the screen to let you know it's found them all.
6) Now the waiting begins. This bit can take a long time depending on how popular you are. The script loops through the array of URLS, visits each one, extracts the date, sender, subject and message from the HTML and adds it to a load of code that's saved as a .eml file.


Click to biggerate:

Step 4: Running the Script

Picture of Running the Script

To run the script, download the file from this step and rename it from inbox_export.txt to inbox_export.pl.

Open the script in a text editor and enter your username, password and the folder that you want your messages to be saved to. This folder must exist before you run the script.

Run it as you did those in step 2.

Wait a bit, if you read the previous step you'll know it could take a while! For the more active/popular/spammy (delete as appropriate) instructables users it has thousands of web pages to visit! Ouch!

Step 5: Using the Files

Picture of Using the Files

Now you've got a whole bunch of files, what're you going to do with them?

Most email clients will let you import a folder, if you're very lucky you can just drag and drop the messages into a folder created in your client.

The "Reply" link at the bottom of each email should take you to the message in your inbox when you click on it, providing you're logged in of course.

You can now organise your messages by date, sender, title and shoe size and need never worry again that Robot will lose your messages! Enjoy!

If you'd like to be notified of a new Perl script coming soon that will allow you to mark all of your messages as read, or delete all, please subscrube.

Step 6: Support, Bugs & Credits

Thanks to:
Steveastrouk for the idea
gmjhowe for the title picture
The Ideanator for the title graphic
Instructables Robot for having patience with me crawling his pages

If you have any problems or ideas, feel free to post in the comments. I'll try my best to help.

Comments

Chankey Pathak (author)2011-12-26

Can we make it in LWP?

Jayefuu (author)Chankey Pathak2011-12-26

Oooof. It's been a while since I wrote it. IIRC it uses LWP _AND_ Mechanize. There was a problem with logging in that I couldn't use JUST LWP for it.

Chankey Pathak (author)Jayefuu2011-12-26

Okay :) I am new in Perl. Completed reading "Learning Perl". Now want to learn something related to web development. Your tool for example is exactly what I like to make. Can you guide me what should I learn next in Perl?

Jayefuu (author)Chankey Pathak2011-12-27

Find a fairly simple web page and make sure you can use Perl to grab the source code. Then use regex to pick some information out of it.

Half the battle is first understanding the web page, Instructables is more complicated than a lot of sites as it uses so my javascript.

So I'd suggest you find your favourite site, use LWP to grab the html then regular expressions to parse out an interesting detail. Then progress to forms or grabbing and processing/saving pictures.

Chankey Pathak (author)Jayefuu2011-12-27

Okay, thanks a lot :)
Do you recommend any source to learn Perl?

Jayefuu (author)Chankey Pathak2011-12-27

Nah, I just think of a problem I want to solve with Perl then start thinking about how I'll do it, then if I get stuck I google for a more specific tutorial.

Chankey Pathak (author)Jayefuu2011-12-27

Okay, thanks for guiding, have a good day! :)

iminthebathroom (author)2011-05-24

sweeet

Jayefuu (author)iminthebathroom2011-05-24

You tried it? It was written quite a long time ago now and if the Instructables HTML has changed since then it *may* not still work.

iminthebathroom (author)Jayefuu2011-05-24

I will tonight when I get home, I'll see what patches may need to be made and report back

siliconekeypad (author)2010-12-20

Very cool project in deed. Thanks

Jayefuu (author)siliconekeypad2010-12-20

Thanks, did you get it to work?

J

Awesome! I'd be interested to see how much load this put's on the Instructables servers.

kelseymh (author)2010-09-16

Tsk, tsk, tsk. Half your steps have no pictures, and you still got Featured! My, how standards are slipping among today's youth...

Jayefuu (author)kelseymh2010-09-16

Shockin' huh! They should really have a word with whoever featured it ;)

scoochmaroo (author)Jayefuu2010-09-16

That guy's new. We'll have a talk with him.

gmjhowe (author)scoochmaroo2010-09-17

I reckon he featured it because the first picture was that excellent, it made up for the rest of the steps.

Hiyadudez (author)gmjhowe2010-09-19

Hehe.

kelseymh (author)2010-09-16

Very cool project; nice implementation! Rated...

NachoMahma (author)2010-09-16

. I just backed-up 321 PMs. Thanks!

steveastrouk (author)2010-09-16

Wa Hey ! Works for me.
Thanks
Steve

About This Instructable

2,326views

10favorites

License:

More by Jayefuu:Beech Tray for Salt & Pepper MillOmas Grünes Tomaten ChutneyDisabling Contactless Payment on Debit Cards
Add instructable to: