Find Jobs
Hire Freelancers

Scrape Wayback for AU domain name and first archive entry, output to CSV

$15-25 AUD / hour

已关闭
已发布将近 7 年前

$15-25 AUD / hour

Unless one of the Freelancer community knows different, there doesn't seem to be a way of querying the AU TLD whois records for the creation date of Australian domain names, even with DIG or nay other commandline tools. My imperfect solution is to harvest the earliest Wayback archive entry . Image 1 shows the two data elements I need to extract from the zonefile csv I have : (1) the domain name (2) the date the domain name was first archived in Wayback. Using a script I found in the Wayback APIs I have built a [clunky] batch script [[login to view URL], attached] that captures data into a series of files that I batch rename to a csv [[login to view URL] example attached]. Each capture from [login to view URL] produces a file with the oldest archive date on line 1. The domain name is obvious from each line, the date is in the YYYY:MM:DD:HH:MM:SS format. It will need a chunk of regex written into the script to filter off the first line of each successful data capture, clean up time data element to readable format (eg 19981202014938 becomes 02/12/1998) and appends the domain name and date to an external csv file. Some of the domains have no entry in Wayback but the script will still need to write the URL to the csv with a 'nul' value as the date element. so I can see which have no Wayback records. The script will preferably draw the urls from an external text file. The successful bid can use the URLs listed in the '[login to view URL]' file attached as a test list for the script. I dont need the scrape, just the script (there are 2.1 million domain names to query). Autobids will be deleted if the proposal is not read, and acknowledged within 12 hours. My script took approx 2 hours to write with minimal experience so bids over 4 hours will be automatically deleted. Any other questions, DM me. Some resources: [login to view URL] [login to view URL] [login to view URL] [login to view URL]
项目 ID: 14358495

关于此项目

14提案
远程项目
活跃7 年前

想赚点钱吗?

在Freelancer上竞价的好处

设定您的预算和时间范围
为您的工作获得报酬
简要概述您的提案
免费注册和竞标工作
14威客以平均价$36 AUD/小时来参与此工作竞价
用户头像
Hi sir, I am scraping expert, I have did more than 350+ scraping project, please check my feedback then you will know. Can we discuss more details about this project? then I will provide example data/script for you. Thanks, Kimi
$250 AUD 在5天之内
5.0 (317条评论)
7.7
7.7
用户头像
Greetings sir, i am an expert freelancer for this job and your 100% satisfaction is assured if you allow me to serve. Here is the reason. Why you should pick me? a) I am a very expert and have the same kind of experience of 5 years. b) I work very hard (16+ hours a day and 7 days a week) and also very fast so... it will be done very soon than most of the other providers c) And most important part is my policy: "I will give you (to my client) life time support (as long as you keep relation with me). And fix any bugs/problem without any cost. So, don't ever worry about me” Please sir, leave a reply ASAP, as I am waiting for your kind reply
$22 AUD 在40天之内
4.9 (222条评论)
7.8
7.8
用户头像
Hello. I can do it the BEST for you. I have read the details completely. Please have a look onto my employer's reviews on my profile so you can get an idea of my quality work :) Inbox me so we can discuss and continue!
$21 AUD 在40天之内
5.0 (6条评论)
4.9
4.9
用户头像
I am highly interested to do data research and extraction work. I can guarantee you that I will extract such data that are missed by others.I can start your work right away very soon. Please note that I will collect all the data MANUALLY. I can do the bulk work within very short time. I tried to offer very attractive price. Waiting to serve you.
$15 AUD 在40天之内
5.0 (15条评论)
4.2
4.2
用户头像
Good day, My name is Nikola Nuspahic and I am business manager at BlueViSion IT Solutions. It is a team of web developing and market research experts from East Europe. Our expertise are Web Development, Database Development. We are also good in data collection and segmentation. Beside that our records speak for them self. You can see on our profile that we are dutiful and fair - we stick to the budget and deadlines and have 100% of completed jobs. Trust, besides skills, is important for any kind of collaboration. I like your project. Can we talk more about it?
$15 AUD 在40天之内
0.0 (0条评论)
0.0
0.0
用户头像
Hey, how is it going today? My name is Michael, I'm a web designer/developer from Boston. I've had a look at your project description and feel that my skills match your requirements perfectly. In fact, I did a very similar project recently which I can share with you. Let me know when you have a minute to discuss.
$22 AUD 在40天之内
0.0 (0条评论)
0.0
0.0
用户头像
Hi I am Zubair, expert in web scrapping,Data Processing, JSON, Shell Script, Data mining, and web research. I have scraped over 1000 sites from last 6 years during which I have undertaken many challenging aspects. I have gone through your project and I am interested in your project. I have my own set of scraping tools and techniques. we can have a detailed discussion on the project over chat… Thanks and Regards Zubair Ashraf
$15 AUD 在40天之内
0.0 (0条评论)
0.0
0.0
用户头像
Hello, It's a pleasure to let you know that I've Completed and Delivered similar project before. All I need to work on customization part if we can proceed towards more discussion. I have gone through your project description and confident to accomplish your project. I am an individual developer and you will be working directly with me if we proceed work on this project. My key skills are – 1) Shopify 2) Angular JS 3) Node JS 4) PHP with Codeigniter and Laravel Framework. 5) App development 6) Shopify templates 7) Amazon/ Ebay API's Let's initiate our chat so we can proceed towards the conclusion of this project scope and give it a start as soon as possible. Thanks. :-)
$20 AUD 在40天之内
0.0 (0条评论)
3.4
3.4
用户头像
Hello, My name is Ishtiyaq Lone, I am expert in web scrapping, data crawling and data mining, I have 6 years+ experience in data mining, website scraping, screen scraping and I am certified web scrapper expert. I have completed 100+ projects web scraping -- for web scraping projects. My key Skills are: Web scraping, Data mining, Data crawling, Ms Office(FULL) JavaScript, Go, PHP, SQL, HTML, Python, Perl, CSS. Platforms: Linux, Amazon Web Services (AWS), Google App Engine, Windows, Mac OS X I have completed more than 100+ projects in Web Scraping. You can test the quality of my work if needed, and provided that I can also show you my sample works. Thanks.
$15 AUD 在10天之内
0.0 (0条评论)
0.0
0.0
用户头像
Hello! I have got acquainted with your requirements and have a proposal as well as some questions that we should discuss. The price and timeline are negotiable. We will agree on final estimation after discussion. My name is Alex. I am a representative of web agency Pixel based in Kyiv, Ukraine (GMT+3) We have 10 years of work experience and more than 450 of various complicities projects implemented. Our programmers work with Word Press, Drupal, Open Cart and also with custom admin systems. Besides, in our team we have great designers. Creation of websites, logos and elements of corporate identity are ones among our main objectives. We know how to promote websites, effectively schedule money of the client and calculate invested funds with the help of E-commerce google analytics. We are targeted on long-term partnership and receiving a positive result by the client. You can be sure that your order will be performed with high level of quality within scheduled time and approved budget. Best regards! Alex
$25 AUD 在40天之内
0.0 (0条评论)
0.0
0.0
用户头像
Hi, I was going through your project specs and I have some questions to ask you. Are you available to have a quick chat now? Thanks and Best Regards Krish
$20 AUD 在40天之内
0.0 (1条评论)
0.0
0.0
用户头像
Hi, I have a good amount of experience in web scraping using Python,Django, Phantomjs and nodejs. This is my latest project on web scraping using python: Scraping using Python: Electronics Parts Intelligence Processing eProductScrapper is mostly scraping & data-mining oriented project, which is based on scrapy and lxml plugins, along with Celery distributed environment via redis. This is mostly focused on electronics parts to fetch information like product details, sku, technical datasheet(pdf), product stock, price history. which will be used to make product life-cycle in a highly presentable manner to make non-authorized seller, brokers, after market sellers more aware of the market requirements of the products. Technology & Framework Used: Python, django, celery, scrapy, nodejs, phantomjs, mongodb, mysql. I would love to have ongoing relationships with your team and ready to work on your time schedule 40-50 hrs per week as per requirements. Thanks & Regards,
$15 AUD 在40天之内
0.0 (0条评论)
0.0
0.0

关于客户

AUSTRALIA的国旗
Melbourne, Australia
5.0
2
付款方式已验证
会员自10月 28, 2013起

客户认证

谢谢!我们已通过电子邮件向您发送了索取免费积分的链接。
发送电子邮件时出现问题。请再试一次。
已注册用户 发布工作总数
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
加载预览
授予地理位置权限。
您的登录会话已过期而且您已经登出,请再次登录。