A web crawler with a web user interface which shows crawling stats in realtime. Uses Node.JS, jQuery, Socket.IO, Express, Twitter Bootstrap.
To use node web crawler, you need to have installed Node.js and MongoDB. When you have them both installed, just clone the repo:
git clone git@github.com:ecdeveloper/node-web-crawler.git
And then install dependencies:
npm install
That's it. Run the app:
node app