thoughts on a project i am toying with

Hey guys...little project I am working on, very very beta.
But I have an led sign displaying my Instructables stats (instructable coming soon) and while building it, I realized that Instructables didn't have an API of any kind to see our stats all together.

So I rolled my own...check it out

click the pie chart on the result page to get the url for a xml output...very, very beta, but you can fetch the data via python or processing via the xml url data, or parse the results page.
It's buggy, but it is sorta working.

sort by: active | newest | oldest
1-10 of 52Next »
schumi235 years ago
btw: I found a bug... it isn't telling the views per instructable, so the area under the numbers is empty. for an example (yours too)
fjordcarver (author) 5 years ago
started fleshing out a scoreboard

and also a more graphical leader board

no rss yet for these stats, but coming soon
fjordcarver (author) 5 years ago
Wow, lots of changes going on. Hard to keep up, I'm still having a blast playing around with stats.
I am now collecting some. Although I imagine it will break in a few hours....again. Broken twice now. And unable (last I checked) to get at some stats now. Sigh, oh well my first forage may be short lived, but it is fun and challenging.
Things I have learned over the last few evenings.

cron jobs - didn't even know what it (cron) was. Huzzah!
task queue in app engine - I am now using a task queue to fetch results, which is beneficial for spreading the work load around...also huzzah!
memcaching results - getting better at this, increases site response time enormously....knew this a bit, know it better....huzzah?
rss- sort of broken right now, (still functions and parses as xml, but there are some commented out bits that mess with the retrieval a little (breaks node data it into a list where present) I will fix this...maybe when instructables changes have all smoothed out.))....early for a huzzah.

So I encourage you (please...batting eyes) to give it a once over if you haven't already.

fjordcarver (author) 5 years ago
lol. It looks like a roll out for some new functionality is in the works. I can't wait to see what there is. I imagine this is going to break my app. Oh well it was fun for awhile.
I guess I will wait and see what is in the works.
So no guarantees on whether my app will be broken by days end or not. I may never see if the new cron and db will populate properly.
fjordcarver (author)  fjordcarver5 years ago
Half-Broken.......waiting till the rest has changed to see what needs redoing. Should be fixable, but maybe a real api is in the works.
fjordcarver (author)  fjordcarver5 years ago
But the new stuff looks mighty interesting.
fjordcarver (author) 5 years ago
I just started memcaching results. The cache persists for up to four hours for now (No guarantees) . This improves the latency quite a bit on searches for stats that people have already looked up. Try looking up yourself then someone else and then yourself again. The second visit to yourself (or the other user) will be lighting fast while the data persists in the memcache.

This also means that you can't have a new result for up to four hours.

The obvious next step is to store results in a database and do the parsing before a request is even made. That way when data is not in the memcache, I can hit the database for it's most recent results. Which will further improve user latency.

I am not sure how I want to do this. One option is to create user accounts and only automate the capture of their data, which is nice and can offer people options on how to display stuff etc......

or I could just add any member that is searched for to the db, which is kinda fun in and of itself, it lets the rest of us see too. The ultimate scoreboard. I mean this data is all available anyways right. People already said 'ya sure, you can share that'

both options have benefits that i can see

one feature I want to add soon is a random unranked 'ible in the db (or memcache maybe) to the front page with the prompt of "Rank Me!!". So when you visit, there would be a little button of sorts that prompts you to check out and rank an unranked ible. Because they are sad when unranked. They cry little 'ible tears when they see their sad totalviews compared with their siblings.

I love all the suggestions/input/support it makes what started as a little diversion and learning experience into something exciting to boot.
This all sounds brilliant stuff.

I've thought of something else - when you publish an ible, there are sliders for difficulty and cost - could you grab this data so that, for instance, I could find Randofo's cheapest robot project, or Schooch's easiest recipe?
Kiteman Kiteman5 years ago
Oh, and if we're really piling on the wish-list, it would be cool if you could hover over the slices of the pie chart and see which one was which, and then that slice be a link to either:
  • The project data, further down the page.
  • The project itself (on the site)
  • A graph for the project data.
fjordcarver (author)  Kiteman5 years ago
This is in the thought pile (actually how the sunburst pie thingy came into being). Currently just using the depreciated static image api, but google has a nice chart api that i plan on exploring more.
I like the idea of a slice leading to the project data a lot. Added to the idea list. I can't see the data anywhere on cost and difficulty, and have wondered where it is buried. For now the data needs to be acessible from a non-logged in perspective, but I was toying with the idea of creating a login for the site and giving it a pro-membership, so it could hand out patches and suchlike (comments maybe). But in all honesty I have not yet looked at getting a script to login.
1-10 of 52Next »