Developing Compilable WebCrawler written in Java
$30-250 USD
货到付款
Hello there,
I have a java project which I want to develop for my report. The project is simple WebCrawler with no interface included. I have managed to finish most of the parts. All I want is optimisation and changing some features of the project. The reason I want to hire a person is, I have no time to optimise this project for the following 1-3 months.
What I want is;
1st) The database kept on text file. I want to change it into MySQL database to make it more efficient and easy to understand.
2nd) Threads. I am using threads but I want them to work more efficient than current version. What threads will be doing is, they will be assigning to one job only ( for example; Start crawling: [url removed, login to view], dont pick any other job like [url removed, login to view], [url removed, login to view] until it finishes the [url removed, login to view] ). The trick point is, when the thread is crawling the links in [url removed, login to view], it should check the links in the another database ( pool DB for links ) whether it exists or not. If it exists, it shouldnt add it into database to prevent extra work and prevent repeated data.
3rd) I want this code to run properly. It doesn't work efficient currently there are some stucking points and other stuff. I have been using several testing tools but some internal code should be corrected.
4th) I am using [url removed, login to view] file, update it with the never version. ( Pretty simple.. )
----
Finally;
I will be giving the whole source-code to you. You will have plenty of time to understand and achieve the goals stated above. As I am giving the source code to you, you would keep it and use it in the future, so please consider this before offering a price. The codes, other stuff and information will be updated as soon as I hire the professional.
The due date is : May 20th 2014 (±5 days)
Thanks in advance!
项目ID: #5813558
关于项目
有17名威客正在参与此工作的竞标,均价$220/小时
Hello, I have 12+ years experience as Java developer. One question, did you developer the complete crawler yourself or is it based on a certain software product? gr, Jacco
Hi, For crawling are you currently making use of any prebuilt api. is it allowed to use any api or should we use the hardcoded java for this. Let me know if interested Thanks
Hello, I'm a Java, J2EE application developer with over 10 years experience. Check my profile, see how other project owners said about my service. Contact me if you were interested. Thanks, Rick
I write clean code with TDD. So this is your chance to get a clean and working solution of your problem.
Hi! I had seen carefully your proposal. Your proposal is very interesting and possible for me. I can help you. I think that we can go long time together. Please send me your code. Thank you.
Hi I have done a similar kind of work in one of my project . For that i have used NoSQL for keeping track of duplicity , Also i have used executor framework for parallel processing of these websites. Let me know if 更多
This is my first bid on a freelancer project, and we all want the first things to be associated with happy memories, so that we can cherish them. I will make sure i will do anything to get the job done for you. I hav 更多