Instructables

Step 2: Data Mining - by Jayefuu

Picture of Data Mining - by Jayefuu
Once we had our selected group of players to go on the cards, we needed all the data about them! Visiting 104 profiles and writing down their credentials did NOT sound like a lot of fun. To speed up the process and to make it more repeatable, Jayefuu wrote a Perl script to visit the profile of each of the chosen few to scrape together their vital statistics. It takes a while to run because it has to visit a few thousand pages.

The Perl script collects:
  • Ibler name
  • Number of Ibles
  • Answers
  • Best Answers
  • Answers Ratio (calculated)
  • Featured Ratio (calculated)
  • User's total instructables views
  • Number of subscribers
  • Date joined
  • Comments

The output is spat out into a .tsv (tab separated value) ready for pasting onto the card templates.

Profile images were gathered with a similar Perl script after messaging all of the chosen Instructablers to check they were happy with their current avatar.

In case you'll find them interesting, the Perl scripts have been included below. They won't run without some fiddling as a crucial line has been omitted to stop people running them. They load a fair few instructables pages. If you really have need to run it, PM Jayefuu to get the line you need.

 
Remove these adsRemove these ads by Signing Up
that card have my name
KnexFreek4 years ago
 Im on a mac (this is on step 2) and i dont know how to view the document, or even what to run it in! LOL HALP!