Sr Data Engineer

Role: Sr Data Engineer

Duration: 7 Months

Location: Mclean, VA (Hybrid)

Job Description:

The candidate should have experience building robust data pipelines, working with large-scale datasets, implementing real-time streaming solutions, and developing REST APIs.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Python and Scala.
• Build and optimize data processing solutions leveraging AWS services, including EMR and AWS Glue.
• Develop and maintain ETL/ELT workflows for large-scale data processing.
• Implement real-time data streaming and event-driven architectures using Apache Kafka.
• Develop, integrate, and maintain REST APIs for data exchange and application integration.
• Write complex SQL queries and optimize database performance.
• Work with distributed computing technologies to process and transform large volumes of structured and unstructured data.
• Collaborate with cross-functional teams to deliver high-quality data engineering solutions.
• Explore and leverage AI-powered tools and technologies to improve data engineering efficiency.

Required Skills

• Strong hands-on experience with Python and/or Scala.
• Extensive experience with AWS cloud services.
• Hands-on experience with AWS EMR and AWS Glue.
• Experience with Apache Kafka and real-time streaming.
• Strong SQL development and query optimization skills.
• Experience designing and developing REST APIs.
• Strong understanding of data engineering concepts, ETL/ELT pipelines, and distributed data processing.

Preferred Qualifications

• Experience working with AI/ML technologies or AI-powered development tools.
• Experience building scalable cloud-based data platforms.