Developing Compilable WebCrawler written in Java

已取消 已发布的 Apr 16, 2014 货到付款
已取消 货到付款

Hello there,

I have a java project which I want to develop for my report. The project is simple WebCrawler with no interface included. I have managed to finish most of the parts. All I want is optimisation and changing some features of the project. The reason I want to hire a person is, I have no time to optimise this project for the following 1-3 months.

What I want is;

1st) The database kept on text file. I want to change it into MySQL database to make it more efficient and easy to understand.

2nd) Threads. I am using threads but I want them to work more efficient than current version. What threads will be doing is, they will be assigning to one job only ( for example; Start crawling: [url removed, login to view], dont pick any other job like [url removed, login to view], [url removed, login to view] until it finishes the [url removed, login to view] ). The trick point is, when the thread is crawling the links in [url removed, login to view], it should check the links in the another database ( pool DB for links ) whether it exists or not. If it exists, it shouldnt add it into database to prevent extra work and prevent repeated data.

3rd) I want this code to run properly. It doesn't work efficient currently there are some stucking points and other stuff. I have been using several testing tools but some internal code should be corrected.

4th) I am using [url removed, login to view] file, update it with the never version. ( Pretty simple.. )

----

Finally;

I will be giving the whole source-code to you. You will have plenty of time to understand and achieve the goals stated above. As I am giving the source code to you, you would keep it and use it in the future, so please consider this before offering a price. The codes, other stuff and information will be updated as soon as I hire the professional.

The due date is : May 20th 2014 (±5 days)

Thanks in advance!

数据挖掘 数据处理 Java MySQL 软件开发

项目ID: #5813558

关于项目

17个方案 远程项目 活跃的May 13, 2014

有17名威客正在参与此工作的竞标,均价$220/小时

trustus

Hello , We have a team of Skilled Java-J2EE professionals with experience up to 8 yrs. You will be able to directly communicate with our technical expert. Our Expertise is J2EE: 1) Frameworks: Struts 更多

$199 USD 在10天内
(41条评论)
6.7
waverick

Hello, I have 12+ years experience as Java developer. One question, did you developer the complete crawler yourself or is it based on a certain software product? gr, Jacco

$250 USD 在30天内
(27条评论)
5.1
csanuragjain

Hi, For crawling are you currently making use of any prebuilt api. is it allowed to use any api or should we use the hardcoded java for this. Let me know if interested Thanks

$166 USD 在10天内
(23条评论)
5.2
chinbochen

Hello, I'm a Java, J2EE application developer with over 10 years experience. Check my profile, see how other project owners said about my service. Contact me if you were interested. Thanks, Rick

$526 USD 在40天内
(16条评论)
5.1
chyconsl

HI, I have done similar projects in the past. I will like to use my experience to serve you better. You can trust my sincere indulgence both in quality and time frame. If you will give me a chance, I will ensure that y 更多

$555 USD 在40天内
(8条评论)
4.3
gfvici

Hi, i'm basicly new in Java, but your project sound interesting. I think i can manage the 1,2,4 part easily and i can learn a lot if i could get this project. I have already some work what i have done with the My 更多

$122 USD 在40天内
(1条评论)
1.0
srbr

I have more than 6.5 Years of Extensive design and development experience using Java/J2EE, Spring, MySql, JPA,Hibernate and various Java derived technology and framework. I have worked on so many multi-threaded java a 更多

$200 USD 在12天内
(0条评论)
0.0
callshubhan

A proposal has not yet been provided

$66 USD 在3天内
(0条评论)
0.0
crash256

I write clean code with TDD. So this is your chance to get a clean and working solution of your problem.

$277 USD 在7天内
(0条评论)
0.0
dishantk

Hi Scarlett, I have been working with Java for the last 4 years. Although multi-threaded application is something I haven't worked on before, but I already have started my hands-on over the same and request this opp 更多

$200 USD 在10天内
(0条评论)
0.0
iphonedev311

Hi! I had seen carefully your proposal. Your proposal is very interesting and possible for me. I can help you. I think that we can go long time together. Please send me your code. Thank you.

$150 USD 在5天内
(0条评论)
0.0
nmdien

Hi Scarlettj, Nice to meet you. I am a senior software engineer of an outsourcing company. I am working on a crawling Internet tool that is used to crawl all meta data of movies, series, episodes and so on. I 更多

$155 USD 在10天内
(0条评论)
0.0
kyyash

A proposal has not yet been provided

$194 USD 在3天内
(0条评论)
0.0
nikunjgupta9

Hi I have done a similar kind of work in one of my project . For that i have used NoSQL for keeping track of duplicity , Also i have used executor framework for parallel processing of these websites. Let me know if 更多

$111 USD 在30天内
(0条评论)
0.0
rahulbabbar1

This is my first bid on a freelancer project, and we all want the first things to be associated with happy memories, so that we can cherish them. I will make sure i will do anything to get the job done for you. I hav 更多

$166 USD 在15天内
(0条评论)
0.0