As a Senior Machine Learning/AI Operations Engineer, you will develop and operate the infrastructure, tooling, and workflows that enable effective use of AI and Machine Learning at CMT. Your responsibilities will span operationalizing AI developer tools and model services, automating model workflows, and improving reliability, observability, and cost efficiency. You will partner with Engineering, Machine Learning, Compliance, Security Engineering, and SRE to translate their requirements into operational capabilities and controls.
You will apply reproducible workflows, evaluation, controlled deployment, monitoring, and incident response to existing developer harnesses and coding tools, large language model (LLM) applications and agents, and machine learning training and inference. Current priorities emphasize AI usage across engineering teams, with shared operational ownership of model development and release infrastructure. The role requires hands-on expertise in scalable AI/ML systems, independent technical judgment, and the ability to turn operational needs into reusable capabilities.
CMT is looking for a collaborative, customer-committed, and creative Senior Machine Learning/AI Operations Engineer who wants to join us in making roads safer by making drivers better!
Responsibilities:
- Develop and optimize scalable AI/ML infrastructure, ensuring a balance between cost-efficiency and speed, with the goal of empowering Engineering and Machine Learning teams
- Operationalize AI developer harnesses, coding tools, and shared model services by tuning model configurations, integrating tools into engineering workflows, evaluating their effectiveness, and implementing usage controls agreed with Compliance and Security Engineering
- Optimize model access and inference through routing, rate limits, retries, timeouts, and fallbacks. Monitor token usage, latency, efficiency, throughput, and costs. Attribute usage and costs by team and workload, and implement budgets, quotas, and capacity controls
- Establish monitoring infrastructure for evaluating the performance of AI applications and machine learning models at scale. Trace model calls, retrieval, tool execution, and training workflows; monitor data quality, model drift, and output quality; and implement safeguards for sensitive data and tool access
- Operationalize evaluation workflows for models, prompts, and AI tool configurations. Work with Engineering and Machine Learning teams to track quality metrics and detect regressions, and with Compliance, Security Engineering, and SRE to incorporate policy, security, and reliability requirements into evaluation workflows and deployment decisions
- Develop and manage reproducible agent, training, and evaluation workflows, including permissions, execution limits, failure recovery, and traceability of inputs and outputs
- Build robust infrastructure to expedite the delivery of new models and AI capabilities, enhancing the efficiency of experimentation, validation, and deployment. Automate versioning, experiment tracking, model registries, release pipelines, retraining, and rollback
- Manage, configure, deploy, and troubleshoot AI/ML workloads on AWS and Kubernetes/EKS. Automate infrastructure with Terraform, Helm, and Argo CD, and optimize CPU/GPU capacity and autoscaling for training and inference
- Partner with SRE, Engineering, and Machine Learning teams to define operational metrics and service-level objectives, investigate production issues, and conduct root-cause analysis for model services, AI tools, and training pipelines. Coordinate with Security Engineering and Compliance when incidents involve security or policy concerns, and implement improvements that prevent recurring failures
- Design and document systems, including writing and reviewing code, to automate operational problems