Web scraping source code written in a scripting lang, Python, Ruby, Javascript etc.

We need the following: Source code to scrape the site of all available business entity data. Preferably written in Python, Ruby, or Javascript (or other modern language), and executable from the command line in Linux. - Program should continue scraping even if a particular request fails. The program should also have a configurable wait period between web requests to avoid swamping slow web servers. - The program will output incrementally to disk as either CSV, JSON, or to an open-source database. Code will be reviewed and tested for accuracy. Supply all directions for installing dependencies and executing the program, as well as the first 100 records scraped by the program as a sample. If desired, we have sample programs available for three states, and can consult on a strategy for a given site. Some states will likely require the use of browser automation: we have a sample script using Selenium for one of the states.