PySpark Expert

已关闭 已发布的 5 年前 货到付款
已关闭

We are looking for a PySpark Expert to help optimise our PySpark pipeline. We are using Kafka for ingestion, PySpark for Stream processing and Elasticsearch for storage. Our pipeline has a couple of bottlenecks, which we want to unclog.

We want someone who understands how to use the various optimisation techniques in Spark, knows how to set the configurations for hardware resources and understands YARN.

If you are an expert in this space, please reach out to us.

Elasticsearch Python Spark Yarn

项目ID: #18170398

关于项目

6个方案 远程项目 活跃的5 年前

有6名威客正在参与此工作的竞标,均价$37/小时

matlabxperts

Hi. I have been working on pyspark since 5 months, In one of my project i had to made pipeline which get data from mysql and perform some parallel operation and submit back to the mysql, the purpose of doing that to 更多

$41 USD / 小时
(18条评论)
6.6
shadabkhan92

https://www.freelancer.com/projects/software-architecture/Elastic-search-kibana-cuckoo-API https://www.freelancer.com/projects/javascript/Webdevelopment-Project-for-Shadab/ https://www.freelancer.com/projects/softwar 更多

$47 USD / 小时
(29条评论)
6.2
deytps86

Hello I am working in Bigdata/Hadoop technologies for years. I worked in Hadoop, Mapreduce, Spark (Streaming & MLLib), Kafka, ELK Stack, Cassandra, MongoDB, Postgre. I implemented above using Python as well as Java 更多

$30 USD / 小时
(25条评论)
5.0
dirisalagopal

Hadoop expert

$30 USD / 小时
(2条评论)
2.9