Top Trumps inspired Instructables Card Game

Step 2 Data Mining - by Jayefuu

Once we had our selected group of players to go on the cards, we needed all the data about them! Visiting 104 profiles and writing down their credentials did NOT sound like a lot of fun. To speed up the process and to make it more repeatable, Jayefuu wrote a Perl script to visit the profile of each of the chosen few to scrape together their vital statistics. It takes a while to run because it has to visit a few thousand pages.

The Perl script collects:

• Ibler name
• Number of Ibles
• Answers
• Best Answers
• Answers Ratio (calculated)
• Featured Ratio (calculated)
• User's total instructables views
• Number of subscribers
• Date joined
• Comments

The output is spat out into a .tsv (tab separated value) ready for pasting onto the card templates.

Profile images were gathered with a similar Perl script after messaging all of the chosen Instructablers to check they were happy with their current avatar.

In case you'll find them interesting, the Perl scripts have been included below. They won't run without some fiddling as a crucial line has been omitted to stop people running them. They load a fair few instructables pages. If you really have need to run it, PM Jayefuu to get the line you need.

Comments

[wareneutron]

that card have my name

[dombeef]

No it doesn't, did you read the infomation?

[KnexFreek]

 Im on a mac (this is on step 2) and i dont know how to view the document, or even what to run it in! LOL HALP!

[Jayefuu]

It's a Perl script. You don't need it unless you want to see how I scraped all the data from the website. It shouldn't run anyway as I took out a crucial line. But it's there for interest.