AI-300 Operationalize Machine Learning and Generative AI Solutions

Learn how to operationalize machine learning models using the complete MLOps lifecycle. This learning path covers experimenting and training models with Azure Machine Learning, automating model training with pipelines and hyperparameter tuning, triggering jobs with GitHub Actions, implementing trunk-based development, managing environments, and deploying models to production.

Module 1: Experiment with Azure Machine Learning

Learn how to find the best machine learning model with automated machine learning (AutoML), MLflow-tracked notebooks, and the Responsible AI dashboard.

• Introduction
• Preprocess data and configure featurization
• Run an automated machine learning experiment
• Evaluate and compare models
• Configure MLflow for model tracking in notebooks
• Train and track models in notebooks
• Evaluate models with the Responsible AI dashboard
• Exercise – Find the best classification model with Azure Machine Learning
• Module assessment
• Summary

Module 2: Perform hyperparameter tuning with Azure Machine Learning

Learn how to perform hyperparameter tuning with a sweep job in Azure Machine Learning.

• Introduction
• Define a search space
• Configure a sampling method
• Configure early termination
• Use a sweep job for hyperparameter tuning
• Exercise – Run a sweep job
• Module assessment
• Summary

Module 3: Run pipelines in Azure Machine Learning

Learn how to create and use components to build pipeline in Azure Machine Learning. Run and schedule Azure Machine Learning pipelines to automate machine learning workflows.

• Introduction
• Create components
• Create a pipeline
• Run a pipeline job
• Exercise – Run a pipeline job
• Module assessment
• Summary

Module 4: Trigger Azure Machine Learning jobs with GitHub Actions

Learn how to automate your machine learning workflows by using GitHub Actions.

• Introduction
• Understand the business problem
• Explore the solution architecture
• Use GitHub Actions for model training
• Exercise
• Module assessment
• Summary

Module 5: Trigger GitHub Actions with feature-based development

Learn how to protect your main branch and how to trigger tasks in the machine learning workflow based on changes to the code.

• Introduction
• Understand the business problem
• Explore the solution architecture
• Trigger a workflow
• Exercise
• Module assessment
• Summary

Module 6: Work with environments in GitHub Actions

Learn how to train, test, and deploy a machine learning model by using environments as part of your machine learning operations (MLOps) strategy.

• Introduction
• Understand the business problem
• Explore the solution architecture
• Set up environments
• Exercise
• Module assessment
• Summary

Module 7: Deploy a model with GitHub Actions

Learn how to automate and test model deployment with GitHub Actions and the Azure Machine Learning CLI (v2).

• Introduction
• Understand the business problem
• Explore the solution architecture
• Model deployment
• Exercise
• Module assessment
• Summary

Learn how to operationalize generative AI applications using the complete GenAIOps lifecycle. This learning path covers planning and preparing GenAIOps solutions, managing prompts for agents with version control, evaluating and optimizing agents through structured experiments, automating evaluations with Microsoft Foundry and GitHub Actions, monitoring application performance and costs, and implementing distributed tracing to debug complex AI workflows.Power Platform: Enables data analysis, process automation, and app creation to streamline business operations.

Module 8: Plan and prepare a GenAIOps solution

Learn how to develop chat applications with language models using a code-first development approach. By developing generative AI apps code-first, you can create robust and reproducible flows that are integral for generative AI Operations, or GenAIOps.

• Introduction
• Explore use cases for GenAIOps
• Select the right generative AI model
• Understand the development lifecycle of a language model application
• Explore available tools and frameworks to implement GenAIOps
• Exercise – Compare language models from the model catalog
• Module assessment
• Summary

Module 9: Manage prompts for agents in Microsoft Foundry with GitHub

Learn how to manage AI prompts as versioned assets using GitHub. Apply software engineering best practices to create, test, and promote prompt versions used in Microsoft Foundry as part of a GenAIOps workflow.

• Introduction
• Apply version control to prompts
• Understand Microsoft Foundry agents and prompt versioning
• Organize prompts in GitHub repositories
• Develop safe prompt deployment workflows
• Exercise – Develop prompt and agent versions
• Knowledge check
• Summary

Module 10: Evaluate and optimize AI agents through structured experiments

Learn how to optimize AI agents through structured evaluation that transforms guesswork into evidence-based engineering decisions. You’ll explore how to design evaluation experiments with clear metrics for quality, cost, and performance; organize experiments using Git-based workflows; create evaluation rubrics for consistent scoring; and compare results to make informed optimization decisions.

• Introduction
• Design evaluation experiments
• Apply Git-based workflows to optimization experiments
• Apply evaluation rubrics for consistent scoring
• Exercise – Evaluate and compare AI agent versions
• Knowledge check
• Summary

Module 11: Automate AI evaluations with Microsoft Foundry and GitHub Actions

Learn how to implement automated evaluations for AI agent responses using Microsoft Foundry evaluators, create evaluation datasets from production data and synthetic generation, run batch evaluations with Python scripts, and integrate evaluation workflows into GitHub Actions for continuous quality assurance.

• Introduction
• Understand why automated evaluations matter
• Align evaluators with human criteria
• Create evaluation datasets
• Implement batch evaluations with Python
• Integrate evaluations into GitHub Actions
• Exercise – Set up automated evaluations
• Knowledge check
• Summary

Module 12: Monitor your generative AI application

Learn how to monitor the performance of your generative AI application using Microsoft Foundry. This module teaches you to track key metrics like latency and token usage to make informed, cost-effective deployment decisions.

• Introduction
• Why do you need to monitor?
• Understand key metrics to monitor
• Explore how to monitor with Azure
• Integrate monitoring into your app
• Interpret monitoring results
• Exercise – Enable monitoring for a generative AI application
• Knowledge check
• Summary

Module 13: Analyze and debug your generative AI app with tracing

Learn how to implement tracing in your generative AI applications using Microsoft Foundry and OpenTelemetry. This module teaches you to capture detailed execution flows, debug complex workflows, and understand application behavior for better reliability and optimization.

• Introduction
• Why do you need to use tracing?
• Identify what to trace in generative AI applications
• Implement tracing in generative AI applications
• Debug complex workflows with advanced tracing patterns
• Make informed decisions with trace data analysis
• Exercise – Enable tracing for a generative AI application
• Knowledge check
• Summary