Pyspark Technical Lead

About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the future-a future in which trial-and-error business decisions have been replaced by informed choices and data-supported strategies.

By applying AI and data science, we help leading companies to prototype, refine, validate, and scale their AI and analytics products and delivery models. Cognizant's AIA practice takes insights that are buried in data, and provides businesses a clear way to transform how they source, interpret and consume their information. Our clients need flexible data structures and a streamlined data architecture that quickly turns data resources into informative, meaningful intelligence.

Job Summary :

We are seeking a Pyspark Technical Lead with 7 to 11 years of experience to join our team. The ideal candidate will have expertise in Pyspark , Cloudera HUE, Cloudera Data Platform, Apache Iceberg, Spark. This role involves working in a hybrid model with day shifts. The candidate will play a crucial role in driving our research and development initiatives, leveraging their technical skills to deliver impactful solutions.

7to11yrs

Required Skills :

Cloudera HUE,PySpark,Spark Pyspark,Cloudera Data Platform,Apache Iceberg

Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform

Supervise the integration of Apache Iceberg and Spark into existing data workflows

Provide technical guidance and mentorship to junior team members on PySpark best practices

Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions

Ensure data quality and integrity through meticulous testing and validation processes

Develop and maintain documentation for data architectures and workflows

Optimize data processing pipelines for performance and efficiency

Conduct code reviews to ensure alignment to coding standards and best practices

Solve and resolve complex technical issues related to data processing

Stay updated with the latest advancements in data technologies and incorporate them into projects

Drive innovation by exploring new tools and techniques in the research and development domain

Communicate effectively with stakeholders to gather requirements and provide project updates

Contribute to the overall success of the company by delivering high-quality data solutions that drive business value

• Possess strong expertise in Cloudera HUE and Cloudera Data Platform
• Demonstrate proficiency in Apache Iceberg and Spark
• Have extensive experience with PySpark for data processing
• Show a solid understanding of data architecture and data engineering principles
• Exhibit excellent problem-solving skills and attention to detail
• Have a background in research and development is a plus
• Display strong communication and collaboration skills
• Be able to work effectively in a hybrid work model
• Have a proactive approach to learning and staying current with industry trends
• Be capable of mentoring and guiding junior team members
• Demonstrate the ability to deliver high-quality solutions within deadlines
• Show a commitment to continuous improvement and innovation
• Possess a strong sense of ownership and accountability for project outcomes.

Benefits: Cognizant offers the following benefits for this position, subject to applicable eligibility requirements:

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability
• Paid Parental Leave
• Employee Stock Purchase Plan

Disclaimer: The salary, other compensation, and benefits information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.