I have been doing large scale website scraping since 2017 so I know every loopholes and tricks to scrape almost every websites out there, small or big. I like challenges so feel free to invite and test me.
I'm a full time Data Mining engineer, I have 5 years of experience in web scraping, data mining, data extraction and API integrations, through working on a wide range of projects. I've developed and maintained scraping engines that extract data from over 800 different websites with 20 million data point every single day.
I only accept a contract if I'm 100% sure I can finish the project. So you dont need to worry about getting your time wasted with half done works.
⚙️ Scraping tools: Requests, Selenium, Scrapy, BeautifulSoap
⚙️ Deliverables: Excel, CSV, JSON, XML, MySQL, MongoDB, SQLite
⚙️ AWS: EC2, RDS, S3, Lambda
⚙️ Google Cloud: Cloud function, BigQuery, CloudSQL, Compute Engine (cloud server)
Scraping methods:
⚙️ Proxy to prevent IP ban
⚙️ Multi-threading to speed up the scraping process
⚙️ Cookie injection to access websites that requires authentication.
⚙️ Plus lots of web scraping tips and tricks that I have learnt over the past 5 years