Data Engineer

M9 Solutions is seeking a Data Engineer to work hybrid remote/on-site on a government contract for a client located inWashington, DC.An active TS/SCI clearance is required.

Responsibilities

• Define and communicate a clear product vision for our client's software products, aligning with user needs and business objectives.

• Create and manage product roadmaps that reflect both innovation and growth strategies.

• Partner with a government product owner and a product team of 7-8 FTEs.

• Develop and design data pipelines to support an end-to-end solution.

• Develop and maintain artifacts (e.g. schemas, data dictionaries, and transforms related to ETL processes).

• Integrate data pipelines with AWS cloud services to extract meaningful insights.

• Manage production data within multiple datasets ensuring fault tolerance and redundancy.

• Design and develop robust and functional dataflows to support raw data and expected data.

• Provide Tier 3 technical support for deployed applications and dataflows.

• Collaborate with the rest of the data engineering team to design and launch new features.

• Coordinate and document dataflows, capabilities, etc.

• Occasionally support off-hours deployments such as evenings or weekends.

Required Skills and Qualifications

• Active TS/SCI security clearance.

• Bachelor's degree or equivalent practical experience.

• Understanding of cloud architectures and enabling tools and technologies such as AWS Cloud (GovCloud/C2S).

• Familiarity with Amazon Web Managed Services (AWS).

• Working knowledge with platforms and services such as Docker, Kubernetes, JMS/SQS, SNS, Kafka, AWS Lambda, NiFi, Airflow.

• Proficient in JavaScript, Elasticsearch, JSON, SQL, XML.

• Familiarity with datastores: MongoDB/DynamoDB, PostgreSQL, S3, Redshift, JDBC/ODBC, Redis.

• Familiarity with Linux/Unix server environments.

• Experience with Agile development methodology.

• Experience publishing and presenting design reports.

• Ability to coordinate with other team members to meet milestones and deadlines.

• Working knowledge of collaboration tools such as Jira and Confluence.

Preferred Skills and Qualifications

• Familiarity with secure data environments and data governance processes.

• Experience working in complex data architecture environments.

• Knowledge of cloud-based infrastructures and data integration frameworks.

• Experience with cloud message APIs and push notifications.

• Interest in learning and applying the latest software tools, methods, and technologies to solve complex data problems.

• Knowledge of public keys and digital certificates.

• Experience in DevOps environments.

• Expertise in various COTS, GOTS, and open-source tools that support data integration and visualization applications.

• Specialization in object-oriented programming, scripting, and databases.