How crawler/spider based search engine works?

Hi, I think you are asking about Data mining. For a brief description, go to http://en.wikipedia.org/wiki/Data_mining . But really, it is offered at the university as a full module and that's just the tip of the iceberg. I think you need to depend on textbooks or professors to find out more extensively on this subject.

Hi tk4159, you need to learn more about SEO. Then i am sure after you learn it, and apply inside your website. Then spider will work effective for your site. Some me friends say, Loading your website all effect this spider based search engine. More fast your site load, easy the spider to crawl.