C# .net 3.5 - crawler and parse

I want to be able to pass a url or urls to a service and have it analyze webpages for some key optimizations

I will pass a specific url to the service and it will process just this 1 page

or

I will pass a starting url to the service and it will crawl the site for all its pages (dedup please) . dont build your own craler code, check codeplex for some if you need it. (also make sure craler stays on same domain)

then it will grab all html/css and images from the site locally to the server and run some tests against it

1-Flag the page and image name that was scaled in html. for example someone uploades a image 1000x1000 and then in a wysiwyg they drag it down to 200x200.

2-Uploaded a image greater than some threshold value like 500kb. I should be able to set this in some config area in the code

3- List each page, # of images per page , size of each images, total of all images, number of css files on the page, number of js, size of all js and css per page

This will be what we use to start. Over time i will want to register more things to check, almost like modules. I can think of many more for example checking resources for gzip etc...I will want to register some new module in the system (via code) and have it be able to run against those modules as well. So let me know your methods here,

Maybe workflow foundation fits nice here with wcf. Im picky about code and want this to be done really well so let me knwo your plan

## Deliverables

1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.

2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables):

a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment.

b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request.

3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement).

## Platform

.net 2.5 wcf wwf

.NET Amazon Web Services ASP C# 编程 MySQL Odd Jobs SQL

项目ID： #3771304

关于项目