Big Data Engineer at PicsArt
PicsArt is looking for a Big Data Engineer. The ideal candidate will work on Big Data infrastructure, determine technical requirements, make architectural decisions and insure the best possible performance.
- BS/MA degree in a highly quantitative field or equivalent;
- Experience with complex distributed systems and service oriented architectures;
- Experience with big data tech stack (Hadoop, HDFS, HBase, Kafka, Spark, Spark Streaming, Presto, Hive, Flume, Samza, Yarn);
- Experience with Python and Scala;
- Knowledge of SQL;
- Knowledge of concurrency patterns;
- Understand fundamental design principles behind a scalable application;
- Experience in building stream-processing systems, using solutions such as Spark-Streaming is a plus;
- Knowledge of various ETL techniques and frameworks, such as Flume is a plus;
- Knowledge of Java (or other OOP language) is a plus;
- Knowledge of Bash Scripting is a plus;
- Ability and willingness to learn new technologies;
- Advanced written and verbal English communication skills.
- Various design and architectural patterns;
- Big data processing architectures;
- Code versioning tools, such as Git;
- Build tools such as Maven or Gradle;
- Continuous integration.
- Help drive the long-term big data architecture and infrastructure by working in conjunction with Product Management to align with business goals;
- Act as a part of the architectural team;
- Develop and operate our data pipeline & manage data infrastructure;
- Work closely with analysts and data scientists and help to develop data-driven dashboards and systems;
- Take on some of the most significant problems in high-performance, scalable analytics;
- Monitor performance and advise on any necessary infrastructure changes;
- Leverage big data processing and automation with Python, Scala, Spark, Hive, Spark Streaming, and more;
- Translate application storyboards and use cases into functional applications;
- Analyze and optimize query/script performance;
- Identify bottlenecks and bugs, and devise solutions to these problems;
- Train and mentor new and junior team members.
What we seek and value most in our candidates:
Professionalism in both soft and hard skills; motivation to grow, learn and share, positive attitude, flexibility, transparency, ownership and the most important - passion and commitment.
You will be working in an environment with more than 1000 servers in 4 data centers on different continents, delivering over 95 billion monthly requests and 10Pb monthly traffic.
PicsArt is the world's largest creative platform for social media editing, with more than 130 million monthly active users and influencers. PicsArt community creates, remixes and shares nearly 1 billion images every month; the company has amassed one of the largest open-source collections of content in the world, including free-to-edit photos, stickers and more. PicsArt is available in 30 languages for free and as a subscription on iOS, Android and Windows devices. Headquartered in San Francisco with offices in Yerevan, Armenia, Los Angeles, and Beijing.