进行中

Scrape products, import to mySQL and develop PHP database management frontend

We are looking for a qualified Web Development Company with good skills in PHP, mySQL, web scraping and database management frontend.

In short:

1) Scrape data from wholesalers website

2) Build product database backend based on scraped data

3) Develop admin panel for database management

1.1) Initial scraping

A ~5000 product product wholesaler database needs to be scraped and transferred into a database backend.

The website where the data needs to be scraped requires user/password and is based on ajax/html

1.2) Database backend admin panel

Develop a management frontend where products can be (re)named, translated, described, priced, grouped, categorized and illustrated with pictures by our staff.

1.3) Update function

Synchronisation function to update the database and check for new and discontinued products if needed.

1.4) User management of backend admin panel

The management frontend should allow user to create descriptions, groups, upload images etc. but not give admin rights to normal user. Entries created or changed by normal users must show up at administrative level for review where admin have the ability to correct or undo changes if needed.

Only bid if you have experience in and already completed similar projects. To make sure we do not waste our time and that you read the full description and our requirements, send a URL to a similar project already completetd by you and shortly explain how you did it. Bidders who do not comply waste their time and will fail. Thank you.

1.5) Auto grouping function
An automatic grouping function of products should be implemented.

Background: The products database contains of many products of which many single products are from the same kind.

For example "sampleproductname" is available in different colors, as whole corns or grinded. The system must have auto function to group products on the basis of the first word of product name. So in this example the product group "sampleproductname" is created automatically and connected to all sampleproductname articles.


Scope of full system
Basically the product database and backend are used to feed multiple shops later on. Every shop will have a different inventory, different prices, minimum order etc.
In the first step we need to duplicate the suppliers database as the core database.
In the next step the admin panel must be created to enhance this database with different product names, descriptions, images etc. The enhanced database is then used for our shops.
Of course the products are still connected by their ProductID to the core DB, especially regarding core features like avaibility and minimum order amounts. But it should be also possible to fall back on the core, for example if no own description is created yet.
Please note that the product ID from our supplier is only used internally. For the customers the system must generate our own product Ids to be displayed on the shops.
If we implement more product characteristics in future, the equivalent DB fields should be easy to setup.

Design of Product Groups and Categories
Definitions:
A Category is defined as containing certain Products groups or Sub-Categories
Categories for example are used in the menus of our shops.
A Product group is defined as containing a certain set of products
A Shop category set is defined as a data base containing different product categories used in a certain shop.
It is important to note that products may be connected with with more than one product group. Product groups may be connected with more then one category or sub-category. Categories may be connected to more than one shop category set.

Keeping track of suppliers database changes
We need to keep track of changes in the suppliers database and need a page on the panel called "change log" displaying all changes and aditionally display product dependend changes on each product page. No need for notifications.


We should split the initial scraping into parts:


1st Step)

Scrape only the product list without loggin in (all 3 languages they offer)
This task you can do on your server and easily use proxies if needed.
(I don't think they have some kind of protection, but not sure about that)

Yes we include Images but only the highest resolution picture. I think we better store them as file not mysql - "productID.jpg" should do it

All other images are not interesting as we have to do image processing anyway at a later stage...

That Should be very easy, because the product list as well as the images do not require login

2nd Step)

Detailed scraping while being logged in

We have still some Product details missing. However they are not vital and can be collected later and more slowly. We we can collect these details while somedy is working on the management panel. When the management panel is installed on our server we log in to the suppliers system from server side and every time somebody edits a description or article name, the server connects to the suppliers system and collects the missing information.

Generally I think we should change from "whole database update once a week" to "update on request".

Whole database update is then only needed for new products and price changes, but this can be done without being logged in.

技能: 数据库管理, MySQL, PHP, 软件构架, 网页搜罗

查看更多: php mysql simple project database, php mysql equipment rental database, mysql php projekt management, php import mysql dump, import mysql database mysql database, php browse file import mysql, php csv import mysql, remove products existing mysql database, php clients management database, best import products magento mysql database, import mysql dump php script, sample proposal develop php front end mysql database back end, database management mysql php html, inventory management mysql php database, school database management system php mysql

关于此雇主:
( 1条评论 ) Koblenz, Germany

项目ID: #17921798

授予:

tunapolje

Hello, I am new here but I have great reviews in other platform if you need... can you answer me on this questions before I give you access to my similar work. 1. Have you username / password from target website? 更多

$600 USD 在10天内
(0条评论)
0.0

55 威客就此工作平均出价 $531

omsoftware

Hello, Greets from OMSOFTWARE!!! Yes,we can definitely do data scrapping from the website and expertise in PHP, mySQL, and database management fronted. Reference link [login to view URL] http://www.s 更多

$750 USD 在25天内
(142条评论)
8.9
xiangang

100% JOB SUCCESS!!! Hello client, I read your job posting carefully and am very interest your project. I am a full stack developer with great experience and skills in Web/iOS/Android development for over 10 years 更多

$555 USD 在10天内
(56条评论)
7.2
phpdeveloper100

Send me url from where you scrape the products, Please open Message box so that we discuss it in detail Expertise ================== - PHP - Cryptocurrency - Code Ignator - Magento - Laravel - Wordpress - Z 更多

$786 USD 在10天内
(73条评论)
7.2
fullymagento

Hi, We are a team of 65+ experts in PHP and Mobile apps and have more than 10+ years of working experience in all of the categories that you need assistance [login to view URL] to your job requirements, I have done many 更多

$555 USD 在10天内
(74条评论)
7.4
ifactorsolu

Hello I am Sumeer from India having more than ten years of experience in various PHP project. I have integrated various APIs and I believe that the scope of the work you have mentioned falls in my domain of expertis 更多

$540 USD 在15天内
(218条评论)
7.4
langlangFan

Hi, thanks for your job posting. I have developed several websites similar with your requirements. I am good at PHP, Laravel and Wordpress. I also have much experience in web scrapping My previous target was eComme 更多

$750 USD 在10天内
(43条评论)
7.1
$450 USD 在12天内
(44条评论)
7.0
phpXpertbd

Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database 更多

$500 USD 在22天内
(84条评论)
7.3
AzzkaNoor

Greetings! Here is the link to my previous project: [login to view URL] Everything is explained in the video. It seems like an interesting project. I w 更多

$555 USD 在10天内
(23条评论)
6.8
hawkscodeaus

Hi, I can start work on your project once you confirm me and i am also open for further discussion if you want. I have 8+ years in web design & development with good experience in following skills ... CMS - Wor 更多

$300 USD 在3天内
(110条评论)
7.5
techwelfare

We are a qualified Web Development Company with good skills in PHP, mySQL, web scraping and database management frontend what you are looking for:We have some questions to clarify,please reply us so we could discuss th 更多

$630 USD 在10天内
(160条评论)
7.1
zekovicm

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within 更多

$744 USD 在7天内
(65条评论)
6.8
polarjin2017

Hello? How are you? I have good skills in So I can complete your job in time. Hope to work with you. Thank you.

$500 USD 在3天内
(92条评论)
6.6
kanika6665

I have 7+ years experience in this field. After carefully reviewing the experience requirements for the job description, I feel that I am suitable for the job.

$555 USD 在10天内
(64条评论)
6.8
Batista111

Hello, I have read the job and understand the requirement. I have a very good team to work with you. I will provide you full time expert developer who can work with you. Please have a look at my freelancer prof 更多

$555 USD 在10天内
(108条评论)
6.6
schoudhary1553

This is Vibrant Webtech and I was glad to see that you're looking for help for project Scrape products, import to mySQL and develop PHP database management frontend. I've delivered more than 400 + projects in the las 更多

$500 USD 在5天内
(60条评论)
6.4
CrazyWebGuru

Hello! Thanks for your posting. I've read your post carefully and I am very interested in your project. I am an experienced and talented web developer and I have rich experience with AJAX, MySQL, PHP, Software Archi 更多

$555 USD 在10天内
(41条评论)
5.7
sharmarahulraman

Hello, Hope you will be doing well!! I have read your description and I understand your requirement and I can handle your project. Please discuss with me for more clarification. Here is some example of my 更多

$467 USD 在13天内
(82条评论)
6.1
bizdigitalbrain

Hello, Hope you are doing great !! I have seen your given requirement that you need Scrape products, import to mySQL and develop PHP database management frontend. >> What We Can Do:- --- We Can Create A Sepa 更多

$555 USD 在10天内
(22条评论)
5.6
yongjin818

Dear, I am an expert in web scraping. In before, I developed 400+ spiders using scrapy, php, selenium. For example, I scraped data for Many products information from these sites(ex: ebay, amazon, welivv, [login to view URL], e 更多

$555 USD 在10天内
(54条评论)
5.7