Sr. ML Engineer

This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing.

"DDN's A3I solutions are transforming the landscape of AI infrastructure." - IDC

"The real differentiator is DDN. I never hesitate to recommend DDN. DDN is the de facto name for AI Storage in high performance environments" - Marc Hamilton, VP, Solutions Architecture & Engineering | NVIDIA

DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence.

Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management.

Our commitment to innovation, customer success, and market leadership makes this an exciting and rewarding role for a driven professional looking to make a lasting impact in the world of AI and data storage.

We are seeking a talented and experienced Senior ML Engineer to help us deploy AI/ML training and advanced Retrieval-Augmented Generation (RAG) pipelines for high-performance AI applications. You will be responsible for designing, deploying, and optimizing large-scale AI training and inference pipelines. You will work closely with data scientists and software developers to operationalize models using open-source tools like Apache Spark, Airflow, and MLflow. You will collaborate in our efforts to scale Retrieval-Augmented Generation (RAG) pipelines for AI applications, ensuring robust and efficient deployment.

Key Responsibilities:

Design and deploy large-scale AI/ML training pipelines using open-source tools such as Apache Spark and Apache Airflow.
• Integrate MLflow with DDN's Infinia product for tracking and managing machine learning experiments, model versioning, and deployment.
• Implement and scale Retrieval-Augmented Generation (RAG) pipelines to enable efficient retrieval of knowledge for generative models.
• Automate, monitor, and optimize the end-to-end ML workflows and pipelines for production-grade applications.
• Work collaboratively with cross-functional teams including data science, engineering, and product to operationalize AI/ML models.
• Maintain and improve CI/CD pipelines for ML models, ensuring smooth transitions from research to production environments.
• Utilize cloud platforms (AWS, GCP, or Azure) for scalable infrastructure management.
• Monitor and troubleshoot pipeline performance issues, implementing solutions to optimize runtime and resource usage.
• Ensure best practices in version control, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, Ansible).
• Keep up-to-date with the latest developments in MLOps, AI/ML frameworks, and tooling.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Data Science, Machine Learning, or related fields.
• 8+ years of experience in machine learning operations (MLOps) or related roles.
• Extensive experience with Apache Spark, Apache Airflow, and MLflow or equivalent.
• Proven expertise in building and scaling AI/ML pipelines.
• Strong understanding of machine learning frameworks and libraries (TensorFlow, PyTorch, NVIDIA NeMo).
• Experience in deploying open-source vector databases at scale.
• Solid understanding of cloud infrastructure (AWS, GCP, Azure) and distributed computing.
• Proficiency with containerization tools (Docker, Kubernetes) and infrastructure as code.
• Excellent problem-solving and troubleshooting skills, with attention to detail and performance optimization.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with large-scale data processing and storage solutions (Hadoop, Hive, HDFS).
• Knowledge of NLP techniques and tools for model deployment.

Our team is highly motivated and focused on engineering excellence.

We look for individuals who appreciate challenging themselves and thrive on curiosity.

Engineers are encouraged to work across multiple areas of the company.

We operate with a flat organizational structure.

All employees are expected to be hands-on and to contribute directly to the company's mission.

Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important.

All engineers and researchers are expected to have strong communication skills.

They should be able to concisely and accurately share knowledge with their teammates.

Interview Process: After submitting your application, the team reviews your CV and statement of exceptional work. If your application passes this stage, you will be invited to a 30-minute interview ("phone interview") during which a member of our team will ask some basic questions. If you clear the initial phone interview, you will enter the main process, which consists of four technical interviews:

• Coding assessment in a language of your choice.
• Systems design: Translate high-level requirements into a scalable, fault-tolerant service.
• Systems hands-on: Demonstrate practical skills in a live problem-solving session.
• Project deep-dive: Present your past exceptional work to a small audience.
• Meet and greet with the wider team.
• Our goal is to finish the main process within one week.
• We don't rely on recruiters for assessments.
• Every application is reviewed by a member of our technical team.

DDN has a very strong orientation towards these 4 characteristics and any successful employee will demonstrate these capabilities:

Self-Starter - Takes independent action to identify and solve problems. Seeks out relevant information needed to make decisions. Gets involved with new initiatives.

Success/Achievement Orientation - Delivers quality results consistently. Targets, achieves (or exceeds) measurable results. Sets challenging goals, focuses on critical priorities, and is accountable.

Problem Solving - Recognizes problems and responds with a systematic assessment that identifies and addresses cause of issue. Practical, realistic, and resourceful.

Innovative - Builds and improves key business processes that enhance the effectiveness of DDN. Generates new ideas, challenges the status quo, and solves problems creatively.

DataDirect Networks, Inc. is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, gender expression, transgender, sex stereotyping, sexual orientation, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.

#LI-Remote