Big Data Engineer

Genesis10
life insurance, 401(k)
United States, North Carolina, Charlotte
Nov 20, 2024
Genesis10 is currently seeking a Big Data Engineer with our client in the financial industry located in Plano, TX, Charlotte, NC, and Pennington, NJ. This is a 12 + month contract position. Responsibilities: Innovate and transform the systems integration landscape for the Technology Infrastructure organization, while following industry best practices and providing capability maturity in support of Enterprise Data Management standards Involves hands-on innovation, development, and support of integrations with multiple systems and ensuring accuracy and quality of data by implementing business and technical reconciliations Analyze current Hadoop based Master Data Management platform including orchestrations, workflows, transformations and help to design a scalable platform based on streaming data pipelines for structured and semi-structure big data Requirements: 10+ years of total IT experience 5 years of experience developing for Data Warehousing, Data Marts, and/or Master Data Management Experience on Hadoop including Confluent Kafka, Python, Spark, HIVE, HBase and HDFS with an emphasis on strategic future forward architecture Programming experience Python, PySpark, Spark SQL Exposure to Relational Database Management Systems using Oracle, DB2 or SQL Server Possesses and demonstrates deep knowledge of the Hadoop Ecosystem Experienced exposure to Hadoop ecosystem including but not limited to: HDFS, Spark, Sqoop, Oozie, Kafka, Hive (Managed & Iceberg formats), Phoenix, HBase Object oriented programming concepts Expert SQL skills Experience in SDLC and best practices for development Ability to work against mid-level design documentation, take it to a low-level design, and deliver a solution that meets the success criteria Knowledge of packaging and promotion practices for maintaining code in development, test, and production Ability to reengineer traditional database system and stored procedures using Big Data services Expert experience in developing and implementing solutions using Confluent Kafka including connectors, Kafka Streams & KSQL Accomplished development experience using Spark and Spark SQL Expert level skills for evaluating, developing and performance tuning existing HIVE Managed Tables and PySpark implementation Experience with shell scripting (e.g. bash, Python, etc.) Apache Phoenix and HBase design, implementation and tuning Ability to manage multiple priorities Desired skills: Experience with Trino Experience with Apache Flink Only candidates available and ready to work directly as Genesis10 employees will be considered for this position. If you have the described qualifications and are interested in this exciting opportunity, please apply! About Genesis10: Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals. For contract roles, Genesis10 offers the benefits listed below. If this is a perm-placement opportunity, our recruiter can talk you through the unique benefits offered for that particular client. Benefits of Working with Genesis10: * Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years. * The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years. * Access to an experienced, caring recruiting team (more than 7 years of experience, on average.) * Behavioral Health Platform * Medical, Dental, Vision * Health Savings Account * Voluntary Hospital Indemnity (Critical Illness & Accident) * Voluntary Term Life Insurance * 401K * Sick Pay (for applicable states/municipalities) * Commuter Benefits (Dallas, NYC, SF) * Remote opportunities available For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all our available career opportunities, please visit us at our website. Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.