Web Page Scrapper / Spider

Web Page Scrapper / Spider I need a coder who is proficient in data scrapping and crawlers. Below I have explained what I want to be done. You will translate what I want into a software technical spec. The Process 1) I will enter keywords to search in Google/Yahoo. 2) This will provide me results for web page,blogs,groups,news. 3) I want to store the webpage content of the first 25 results returned in the search for web,blogs,groups,news. How point 3 should work... The crawler will visit each web page and extract the web page content. This content will be stored in an access database. Following fields... search result rank domain url content date of search search engine search type (web,blog,groups,news) search keyword This process can be automated or I can run it manually anytime I want. Development Considerations We are speiclaized in Active Server Pages, Visual basic 6. Therefore to manage the application the future we would like to have this application developed in ASP script or Visual Basic 6. The backend for this app will be sql server but for this demo you can use Access. NOTE---------------- 1) We would like to have this script delivered beta tested within 75 days. If you are not in a position to deliver within 75 days then do not bid as we will cancel the project if not delivered in 75 days. 2) We will only entertain those bidders that have the ability to take on this project from both functional and technical aspect. I have explained what I want but I would like coders to offer a better way of doing what I want. 3) Before I select a bid I would like to have a technical and functional document of how you will be approaching to build the script and the project timeline. Thank You. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform Activer Server Pages and/or Visual Basic 6