Find Jobs
Hire Freelancers

Build a Python Based Web Scraping Application

$750-1500 USD

已关闭
已发布大约 9 年前

$750-1500 USD

货到付款
I'm looking for an expert in Python and Web Scraping to take on a very specific project. The ultimate project will actually NOT be Django, but rather using a microframework called Flask (If you have Django experience, Flask is about a one day learning experience). For Python, we prefer people with Scrapy experience, and would like to stick with Scrapy and Flask. Essentially, you'll be building a spider that will hit a specific website, and your job is to crawl approximately 20K pages. The first time, we do a full scrape, if the page exists in the Database then you'll check for changes and updates. Accuracy is 100% vital. All data should be collected using JSON, the spider will of course run server-side, but there needs to be a corresponding web interface for manually performing CRUD operations. You should also note, that we expect the collected data to be stored in a MySQL database (which should be appropriately indexed and normalized). Finally, it is an absolute must that you have GIT experience. Not just doing code checkins 1x per day, but routinely doing pushes to our repository. If you don't understand GIT and how team development works with a DVCS, please don't waste the time bidding. Also, if you don't get anything herein, again don't waste our time. If you send me 20 useless website URLS I'll passover you entirely. I'm a little jaded by people that keep submitting the same "blanket response" to every project. You have enough information to accurately bid within reason, but ask questions if you don't.
项目 ID: 7108070

关于此项目

25提案
远程项目
活跃9 年前

想赚点钱吗?

在Freelancer上竞价的好处

设定您的预算和时间范围
为您的工作获得报酬
简要概述您的提案
免费注册和竞标工作
25威客以平均价$1,269 USD来参与此工作竞价
用户头像
Dear Sir, I'm a seasoned web scraper. Have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. Please give me the opportunity to do the work. Warm Regards, Debdulal Roy Proshanta
$1,250 USD 在20天之内
4.9 (72条评论)
7.3
7.3
用户头像
Hi, I can do this for you. Please send a massage in the PMB for details.......Best Regards flashsaiful
$1,250 USD 在20天之内
4.8 (127条评论)
6.6
6.6
用户头像
A proposal has not yet been provided
$1,250 USD 在20天之内
5.0 (56条评论)
6.3
6.3
用户头像
hello i have good experience in writing crawlers in python . And also with scrapy and crawlera . i can work with django,flask and git . thanks
$1,400 USD 在10天之内
4.8 (77条评论)
6.2
6.2
用户头像
Hello, I would like to help you with this project. I've worked with scrapy before, and 20k items is certainly doable with it. I must say, if I were to choose web framework, it would be django rather than flask, as it's got more batteries included and you end up being more productive. But yeah, it can be flask if you have some good reason to choose it. One issue that needs some attention is - what should happen to manually edited items during next run? In order to keep your changes, we have to store them in a separate table (otherwise they'd get overridden). I'm not sure why did you mention collecting data in JSON, do we have to scrape JSON rather than HTML pages? Best regards, Stas.
$1,400 USD 在15天之内
5.0 (23条评论)
5.7
5.7
用户头像
A proposal has not yet been provided
$750 USD 在20天之内
5.0 (107条评论)
5.9
5.9
用户头像
Hi, I have experience working with flask, scrapy and mysql and GIT. I'm also proficient in regular expressions and general web development which come in handy for web scraping. I've done web scraping projects in the past and have always been able to retrieve the required results. I've been working as a freelance software developer for over a year specializing in python and web development. I have an engineering degree and excellent problem solving abilities. I pride myself if delivering high quality maintainable code and in working with my clients to ensure they are 100% satisfied with the products I develop. I would be happy to discuss the project in more detail and learn more about your requirements and what exactly you want scraped. If your interested in my services feel free to contact me at any time and I'll get back to you as soon as possible. Thanks, Joshua Cruikshank
$1,250 USD 在20天之内
5.0 (13条评论)
5.2
5.2
用户头像
Hello, sir. My name is Alex, i'm professional python developer. About my bid first - i'm just making portfolio on freelancer, so my rates are low now, later it will cost twice =) I like your project and i'm good candidate because: 1. I have 5 years of web development experience, last 2 years i was working as system architect. 2. I was working with flask, django and scrapy. 3. I have experience about building muthithreading appilcation making mass page downloads (i've made 1000000 google result downloads in 3 days). 4. I'm damn good. I have some questions we can talk about if you like my bid. Looking forward to work with you!
$833 USD 在15天之内
5.0 (3条评论)
4.9
4.9
用户头像
Hello, I have experience with Flask/SQLAlchemy, will use Scrapy to do the crawling. My solution will keep the historical downloaded data, and will provide a web interface to view it. Updates will be done using HTTP HEAD requests (checking Date and If-Modified-Since headers, to lower total traffic). Not sure why you need JSON here (do you want to parse the content in some way?), I'd rather store whole pages as strings in the database.
$1,000 USD 在10天之内
5.0 (1条评论)
2.7
2.7
用户头像
Hello Sir, I can develop your web scraper, I have already developed this type of work using BeautifulSoup. It's nota problem for me to use Scrapy. For example, I get the entire database of citizens in France using my web scraper (with address, phone number, name, etc), 60 millions entries, The accuracy is also my goal. I work with passion, that is why I work the day, the night even the weekend (and I'm not a freelancer flooder, with blanket responses) Best Regards,
$1,250 USD 在25天之内
0.0 (0条评论)
0.0
0.0
用户头像
I have substantial experience developing services and applications in Python, Django and Flask which would be ideal for this project. I've around 5 years commercial experience with Python, and over the past year have used Flask to develop complete API's from scratch, which persisted using MySQL or mongodb. I also use git on a daily basis, and believe regular commiting is of the utmost importance. Please let me know if you have any questions about my bid. Thanks, Tim
$1,111 USD 在7天之内
0.0 (0条评论)
0.0
0.0
用户头像
This is a expert python developer with experience in Django and Scrapy. Hope to establish cooperation relationship. Regards.
$1,250 USD 在15天之内
0.0 (0条评论)
0.0
0.0
用户头像
I used Python and Scrapy to extract trading card data from four different HTML-based sites. The data was extracted in JSON format, and the script was manually run from the server side, with no available CRUD web interface aside from the default Django admin site used alongside the project. If I may ask, what site will we be extracting data from? Thank you!
$1,400 USD 在15天之内
0.0 (0条评论)
0.0
0.0

关于客户

UNITED STATES的国旗
Aliso Viejo, United States
4.9
34
付款方式已验证
会员自12月 9, 2012起

客户认证

谢谢!我们已通过电子邮件向您发送了索取免费积分的链接。
发送电子邮件时出现问题。请再试一次。
已注册用户 发布工作总数
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
加载预览
授予地理位置权限。
您的登录会话已过期而且您已经登出,请再次登录。