I require a Python script which will perform the following actions:
===========
Requirements
===========
1) Open/read a .txt file (perhaps any/all .txt file(s) which appear in the same directory.)
2) Creates new text file (i.e. [login to view URL])
3) For each line in .txt file from #1 above (which is a search term), performs a google search
4) Scrapes # of search results
5) Scrapes all 10 titles (titles only, not description or url) for each search result on the first page (only first page required)
6) If the results page also shows 'related searches', scrapes those too
7) Appends results of #4, #5 & #6 above to the results text file
8) Adds 5 carriage returns & moves on to next search term
======================================
Output would be displayed in [login to view URL] as follows:
======================================
[Search term 1] (search - [no. results] results)
[result 1]
[result 2]
[result 3]
etc.
Searches related to [search term]
[related a]
[related b]
[related c]
etc...
(5 carriage returns)
[Search term 2] (search - [no. results] results)
[result 1]
[result 2]
[result 3]
etc.
Searches related to [search term]
[related a]
[related b]
[related c]
etc...
(until all search terms exhausted)
==============================
Real life output would appear as follows:
==============================
Build a beach body see results in just 10 days (search - 47,500,000 results)
30 Day Beach Body Challenge - 30 Day Fitness Challenges
P90 - Beachbody
Insanity Workout - Beachbody
About Beachbody Products - Innovative In-Home Fitness ...
Make These 15 Small Changes to See Big Results - The ...
10-Minute Trainer - [login to view URL]
21 Day Fix Results - [login to view URL]
Workout programs from beginner to advanced | John K ...
Just How Fast Can I Get the Body I Want? | Nerd Fitness
10 Day Workouts on Pinterest | 10 Day Challenge, Beach ...
(next result)
ANY QUESTIONS, please ask.
Hi, I have developed many Scrapers in .NET and Python. I have read the description & understood most part of the project...I would definitely like to discuss further details...
Looking forward to your response..
Hi, expert programmer and web/data scraper here with over 19 years experience in programming and RDBMS.
Please see my reviews.
I'm using Perl or python for this kind of jobs.
I'm able to finish fast.
hey there
i am pretty keen on scraping and i find your project very interesting.
i have a lot of experience with python and i think you will be satisfied with the result.
the only thing that i want to know is that if google blocks our IP, are we going to use a proxy
server?
p.s. I can start working on the project from tomorrow morning, since i have to catch a plane today.