Trigger Spark Jobs based on various conditions

$10-30 USD

已完成

已发布

将近 6 年前

$10-30 USD

货到付款

Suppose there are two folders A and B. A folder will be loaded with Employee data files and B folder will be loaded with Department data files. Every minute, the java program should monitor these folders and trigger SparkA(based on folder A) or SparkB(based on folder B) programs(Spark-Submit or equivalent). After successful completion, the java program should move the processed files to a different location. The condition is either for every five minutes or for exceeding the size of the folder to more than 2GB.(Spark 2.1.1) You can use below code for reference: [login to view URL]

Hadoop

Java

Spark

项目 ID: 16753314

关于此项目

6提案

远程项目

活跃6 年前

想赚点钱吗？

电子邮箱地址

在Freelancer上竞价的好处

设定您的预算和时间范围

为您的工作获得报酬

简要概述您的提案

免费注册和竞标工作

颁发给：

@sandeepfreax

Hi I am interested in this job. I believe that the skills and experiences I have gained at this position make me an ideal candidate for this position. I have extensive experience in implementing Spark-core, Spark-sql and Spark-Streaming. I have developed strong programming and debugging skills. I have completed dozens of PoC for my customers and received praise for the before time completion and that in Scala, Java and Python. Beside Apache Spark, I have also gained extensive experience in Jenkins and Automated / Unit Testing. My experience has afforded me the opportunity to become familiar with the some of the best used architectures in the programming. In particular, through my work I have become heavily involved in writing the good applications and debugging the high severity issues. It has also helped me in designing the best application model. I am always eager to learn about new emerging technologies in the world of programming and Devops Integration. I would love to bring the passion for programming and sharing the good practices with your application. I am confident that my experience in the Big Data and my programming skills in Scala / Java / Python will qualify me for consideration. If you would like, I can discuss my experience with you. I look forward to meeting with you and discussing my qualifications in more detail. PS : I am certified Spark, Scala and Java Developer. Beside that I held many certificates in different domains. Regards Sandeep

$55 USD 在4天之内

0.0

(0条评论)

0.0

6威客以平均价$73 USD来参与此工作竞价

@Gaosong2017

Hello, dear How are you? I have experience in Hadoop in pyspark with python. I think that i can help you as well. Hope your contact me. Best Regards.

$100 USD 在1天之内

4.9

(18条评论)

5.2

@AmolZinjadeP

Hi, I have more that 4+years of experience in hadoop and data mining technologies like HDFS, MapReduce, python, Spark, Hive etc. Please review my profile for skills Contact me

$100 USD 在1天之内