Skip to content

Un-learn and Re-learn!

October 21, 2013

I am new to blogging, I wish I had started a lot earlier. What am I doing these days?…. sighh…. I’m really tired (never physically, but mentally YES! HELL YES).
Thesis work hasn’t yet kicked off as well as I hoped (although I’m gunning for it now);
I’ve been inspired by a couple of friends to Read and NOT Watch (books/movies in this case);
I still continue looking for a research group willing to fund summer-2014 for my internship (its frustrating at times);
And yes programming will always be there on my mind.

So I’ve thought of trying my hands at PHP-MySQL, nothing complicated, just building a database of something (No you aren’t supposed to know 😛 ). Found an awesome python library ‘Scrapy‘ which will help me in scripting my web crawler in order to scrape out data from on-the-line ( 😛 have been watching THE INTERNSHIP, yes its damn funny). Scrapy‘s documentation does NOT let you export the data out as a MySQL database. But HOW LONG can you ignore MySQL huh? C’mon you can let us write items to JSON, XML, CSV, blah blah! And NOT MySQL? You gotta be kidding me! Well I’m really late to see Scrapy-Dappy-Doo, but then came a Messiah (in 2009! I’m so late!) who edited the usual Item Pipeline procedure and showed us all how to write scraped data to a MySQL database.

Its my first day with this, and I’m still working on my first Spider-Pipeline. I also came to know about a cool chrome extension (yes it is FREE!) from a friend Vipul. Its called ‘Selector Gadget‘ and its actually something you can call cool! Once you install it, you click on its icon at the top-right corner, and boom! up comes a cool widget kind of thing (whatever you call it):

Selector Gadget

Once you go through the basics of Scrapy, you’d know that your crawler/spider/scraper needs the XPath of the data field you’re looking for. So you can easily select the data you want, deselect any additional data coming along, and look at its XPath with Selector Gadget!

XPath of the 'price' field

Follow the Scrapy at a glance and then its Tutorial to get a basic idea about it. I’ll keep you informed with new developments. Oh yes, it seems Android has officially launched the BBM App.

But after going through this post, you must be wondering WHY the title to this post. Its because just a few minutes back, I’d come across a blog post by an awesome, fantastic personality (yes he’s my old pal! from the IIT-JEE days) Gaurav Ojha. Guys, have a look at this post (Its a page-turner, or rather, a page scroller). He’s made his own webpage, and it is an exhibition of true creativity. I’d heard a few poems from him back in the JEE days (its been 4 years now) and I knew he was quite enthusiastic. The title of this post comes from his description about himself 🙂

That’s all!

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: