web crawler

₹12500-37500 INR

已关闭

已发布

8 个月前

₹12500-37500 INR

货到付款

I am looking to create a web crawler that will extract specific data from e-commerce stores online. The goal is to collect text content, which needs to be cleaned and formatted so that it is ready to be used. I require a solution that will extract relevant information quickly and accurately - I will not accept anything less. If you think you have the skills necessary to deliver a successful project, then I invite you to submit a bid for this project. Thank you for your time! This will require someone to 1. Crawl a parent page 2. Get all the links in that page 3. Hit that link and then get few data from the page 4. Create an excel with that data Things to note , [login to view URL] should be some sleep Time introduced if there is continous hits the IP address can be blocked due to rate Limiting 2. Need a MultiThreaded application which can run parallel crawls, to make sure the crawling doesnt take hours ( 1 crawl could contain 1000+ links to hit) 3. If Program crashes/Needs to be stopped the previously crawled data should not be lost - One solution is intermediate writes 4. Open to run on local machine or cloud ( Ready to invest more if cloud can help scale)

项目 ID: 37233719

关于此项目

15提案

远程项目

活跃7 个月前

想赚点钱吗？

电子邮箱地址

在Freelancer上竞价的好处

设定您的预算和时间范围

为您的工作获得报酬

简要概述您的提案

免费注册和竞标工作

15威客以平均价₹24,537 INR来参与此工作竞价

@schoudhary1553

Top 1% in Freelancer.com Hi, Greetings! ✅checked your project details: ✅Completed Time: In project deadline We have worked on 900 + Projects. I have 6 + years of the experience in same kind of projects. If you are looking for a true Freelancer, I am the Right person for you. I am available almost 24-7 and am very responsive. I feel proud that I am a trusted Freelancer who pleases almost every single client. You can rest assure, your work will be delivered well in advance of others, with passion and accuracy. I guarantee you instant communication & responses when you need me. Why choose me? I think every client is the reason for my success. I only take projects which I am sure I can do quickly. My Portfolio Items: https://www.freelancer.com/u/schoudhary1553 I would really like to work with you on this project. If interested, Kindly contact me via chat for further details and discussion. Thank you Sandeep

₹25,000 INR 在7天之内

4.9

(684条评论)

8.7

@chirgeo

Hi! I'm George, an experienced web crawler expert. I can confidently say that I have the skills and experience necessary to successfully complete this project. I have a custom web scraping tool that can be used to quickly and accurately extract relevant information and format it for use. The solution I propose: 1. Crawl parent page to get the links. 2. Hit the links and extract data. 3. Keep a sleep time to avoid rate limiting and IP blocking. 4. Create a multithreaded and fault tolerant application for efficient crawling. 5. Use local machine or Cloud for scalability and to handle heavy load. Questions I have: 1. Are there any specific data points which need to be extracted? 2. What is the expected time frame for completing the project? 3. Are any existing APIs available to get the data? 4. Will you be providing access to the e-commerce websites? 5. Is there any recurring requirement for similar work? I am looking forward to hearing from you and talking more about this project. Thank you for your time and consideration.

₹41,555 INR 在3天之内

5.0

(164条评论)

8.1

@ujjwalmalik

I understand that you are looking for a web crawler that will extract specific data from e-commerce stores online. The goal is to collect text content that needs to be cleaned and formatted so that it is ready to be used. If you think I have the skills necessary to deliver a successful project then I invite you to submit a bid for this project. My team and I have expertise in data entry, data mining, and related tasks which make us well suited for this project. We have the necessary skills and experience to provide the solution you are looking for. Our team is made up of experienced professionals with backgrounds in data entry, data mining, and related tasks. This ensures that we can reliably complete any task assigned to us without compromising on accuracy or speed. Additionally, our solutions are highly scalable so you can rest assured that your data collection will not be slowed down due to heavy workloads or network restrictions.

₹25,000 INR 在7天之内

4.8

(88条评论)

6.8

@softwarelinkerrs

Hello, my name is Nitin and I am part-owner of SoftwareLinkers. We specialize in providing solutions for businesses that operate through the internet. I noticed you are looking for a web crawler that will extract specific data from e-commerce stores online. If you choose us for this project, we will use our experience in PHP and other related technologies to create a framework that can quickly and accurately extract relevant information from websites. Our team is highly skilled in developing multi-threaded applications that can run parallel crawls without taking too long to complete. Additionally, our programs are designed to be robust so they won't crash if required to stop unexpectedly or if there is needed sleep time introduced to keep rates low when needed. We are also open to working with you on scaling the project across multiple locations if needed.

₹35,000 INR 在12天之内

4.7

(23条评论)

5.7

@manojguragain184

Dear, Client Please let me know the e-commerce stores. I am thrilled to express my interest in your web scraping and automation project. I am confident in my ability to deliver outstanding results that align with your requirements. I am excited about the opportunity to contribute to your project's success. My service 1: Scraping API or Html or JavaScript rendering web pages using python requests, scrapy, selenium and bs4 2: Fast scraping without blocking by multi-threading, aiohttp asycnio,Proxy combining, ReCaptcha bypass and Stealth browsers. 3: Console or GUI-based scraper using python Tkinter or Pyqt5 4: Real-time stream data scraping via client WebSocket. 5: Notification to users at several conditions while monitoring by using webhook. 6: Storing into CSV or google Sheets or uploading into DB such as Mysql, MongoDB, and Postgresql. 7:Web app that shows scraping or lives monitoring results. 8:Scraper schedule running by Cron. Looking forward to the possibility of working together. Best Regards Manoj

₹15,000 INR 在5天之内

5.0

(10条评论)

5.1

@valeriivapirov

Hello I am a seasoned Scraping Expert with expertise in Python and JavaScript. I have extensive experience using BeautifulSoup, Puppeteer, and Scrapy for advanced web scraping and generating actionable insights. Here are some highlights of my achievements: ✅ Harvested job listings from over 100 websites to create a comprehensive job database. ✅ Extracted 500,000 product details to build an inventory and sales management system. ✅ Aggregated 1,000 daily news articles for a content-rich platform. ✅ Automated hourly updates for a financial website with over 10,000 data points. ✅ Maintained a remarkable data accuracy rate of 99.5%, enabling informed decision-making. I am confident that my expertise and proven track record in web scraping will significantly contribute to the success of your project. I am eager to discuss the project details further and provide you with top-notch solutions. Please feel free to reach out to me to initiate a discussion. Best Regards,

₹20,000 INR 在7天之内

5.0

(4条评论)

3.7

@malkesh3m

⭐⭐⭐ Hi, My availability: Immediate. I read your project post of Python Developer to write script to scrap data. We are experienced full stack python developers with skill sets in - Python, Django, Flask, FastAPI, Jupyter Notebook, Selenium - Web App Development, Data Science, Scrapping, Machine Learning, AI - API Development, Authentication, Authorization - SQlAlchemy, PostegresDB, MySQL, SqLite - Web hosting, Docker - Azure, AWS, Digital Ocean, GoDaddy, Web Hosting Please send a message So we can have a quick discussion about your project and proceed further. Looking forward to hearing from you. Thanks

₹35,000 INR 在20天之内

4.2

(13条评论)

3.8

@anisurrehmanuss

My name is Anees and I am experienced web and mobile app developer with 15+ years of experience. I have helped small medium and large scale businesses develop their websites, mobile apps, software and digital presence. I understand that you are looking for a web crawler that will extract specific data from e-commerce stores online. The goal is to collect text content that needs to be cleaned and formatted so that it is ready to be used. I believe I have the skills necessary to help you with this project. Specifically, I am familiar with web crawling technology and have the expertise needed to deliver a successful project. As part of the project, I would require someone to crawl a parent page, get all the links in that page and then hit those links while collecting data from the page. My applications are optimized for multi-threaded execution which helps prevent crashes while ensuring data preservation across crawls. Additionally, they are cloud-based so they can be scaled as necessary for large projects without requiring additional infrastructure on your end.

₹12,500 INR 在1天之内

4.9

(4条评论)

2.5

@Rehan9898

Hi I had understood your requirement. I have developed similar programs previously using selenium, puppeteer. I can develop you also with your required features. And it would be multithreaded, like we can be able to run 1000s of inatances in a multithreading with crash handling capability. Let's discuss more details in chat. Thank you

₹12,500 INR 在1天之内

5.0

(6条评论)

2.5

@maxota940601

Hello. I have a lot of experience in creating scrappers. Please indicate the data from which site and in what format you will need to collect. I use different libraries such as beautifulSoup, request, playwright, undetected webdriver and selenium. if you have any questions. please contact us in private messages. thanks, have a nice day

₹25,000 INR 在4天之内

5.0

(2条评论)

1.8

@ethan86zd

Hi there, I am Ethan, 10+ years of experience developer. I have extensive experience in Web Scraping and data processing. I read your description and i think it's not a big problem for the rate limit, you can give more detail and i can help you to analyse the website security strategy. I look forward to hearing back from you. Thank you for the opportunity! Best Regards, Ethan

₹37,500 INR 在5天之内

0.0

(0条评论)

0.0

@rafaelh9sixx1

I understand you are looking for someone to create a web crawler that will extract specific data from e-commerce stores online. The goal is to collect text content that needs to be cleaned and formatted so that it is ready to be used. I can provide the necessary skills and experience to successfully complete this project. Specifically, I have extensive knowledge in web crawling and programming in Python that will help me develop a successful solution for you. My skills include Python and Django which would enable me to quickly develop a multi-threaded application that can run parallel crawls to make sure the crawling doesn't take too long (1 crawl could contain 1000+ links to hit). Additionally, I am open to run the project on local machine or cloud if necessary to scale effectively.

₹13,000 INR 在7天之内

0.0

(0条评论)

0.0

@balazsmark

Hi! I have a lot of experience in web crawlers thanks to my OSINT software development past. Of course threading and Excel exporting is not problem for me I use them a lot when I develop data analyst tools. Feel free to contact me and I can start to do it for you. :-)

₹35,000 INR 在5天之内