Crawl data from Pinterest

Hello! I'm great at web crawling. If you look at my past projects on Freelancer.com you'll see that I've successfully completed many similar projects before. I can create such a Pinterest crawler in Node. Please contact me so we can discuss further details.

Dear Customer! I am an expert PHP developer with over 6 years of experience and very interested to work on this project. Available to start immediately and finish as soon as possible. My bid is for fast professional service exciting my customers. Please contact in PMB to discuss details. Best Regards, Zeke

Thanks for reading this! I have few recommendations and questions. Please let me know a suitable time for a chat. Regards

While I am new to freelance work, I have designed large scale data crawlers for two different employers, gathering data from a variety of sources ranging from APIs to infrastructure to Web scraping. Unfortunately, both are still in use at private companies, and I cannot show you the code. Nevertheless, I can give their architecture in greater detail if you want. I am highly confident I can complete this quickly, as I have a store of existing experience in exactly this area. Technology wise, I highly recommend Python with GTK bindings. Manipulating the DOM directly is much cleaner than parsing the source, although both work well. Process would be as follows: - Design relational schema in DataBase (MySQL I presume), that incorporates the data points you intend on gathering. Correctly model all objects, events and relationships. - Built data crawler in bottom up, modular process. Built the data extraction modules with the same hierarchy as your data model (ex. User extractor is primary unit, containing a board reader, pin reader (which itself contains a comment reader), follower/following reader, etc.) - Built data transformation layer, which is greatly simplified if the crawler and data source are similar in structure. - Finally, determine launch point for crawler. More sophisticated solution is to build a hub service for scheduling and execution management. Spring Batch Tomcat Webapp works well, if you want the crawler to run on a schedule.

I am working on a BI project with python at the moment using the pyquery module. The application I am building is using the multiprocessing module as well in order to benefit from multi core processors. The back-end is an oracle database but I would recommend an approach where the storage should be mongodb. The project is not in production yet, so I could not give more details or direct you to the live example. I am very interested in similar projects and I have in mind for the creation of a related service running on the cloud. The price I am asking is talk-able.