Amazon SageMaker Unified Studio has introduced a CI/CD command line interface (CLI) that simplifies the deployment of multi-service data and AI applications. This open-source tool allows data teams to define their applications in a single YAML manifest, while DevOps teams can deploy them with just one command. The CLI automatically manages configuration substitutions, resource provisioning, and dependency ordering, making it easier for teams to collaborate.
Why It Matters
Organizations often integrate various AWS services, such as AWS Glue and Amazon Athena, into their applications. Transitioning these applications from development to production involves complex configurations and resource management. The CI/CD CLI addresses these challenges by providing a streamlined process that separates responsibilities between data and DevOps teams.
Key Features of the CI/CD CLI
- Declarative Manifest: Data teams create a YAML manifest that outlines the application’s resources and configurations for each environment.
- Controlled Deployments: DevOps teams maintain control over deployment methods and can integrate the CLI into existing CI/CD workflows.
- Separation of Concerns: The CLI abstracts AWS service interactions, allowing teams to focus on their specific roles without needing deep knowledge of every service involved.
Deployment Workflow
The deployment process begins with the data team defining the application in a manifest file. Each stage of the application corresponds to a distinct SageMaker Unified Studio project, ensuring isolation between development, testing, and production environments. The CLI manages the deployment lifecycle, including:
- Bundling the application from the development stage.
- Deploying it to the testing stage.
- Running validations before promoting the same artifact to production.
Example Use Case: Bureau Veritas
Bureau Veritas, a leader in testing and certification, utilizes the CI/CD CLI to manage their data and AI applications across multiple environments. Their architecture manager highlighted the need for a controlled promotion process that respects team boundaries, which the CLI effectively provides.
Getting Started
To use the CI/CD CLI, users should:
- Install the CLI from PyPI.
- Clone the example repository and configure the manifest file to match their SageMaker projects.
- Run the deployment commands to promote applications through various stages.
Integration with CI/CD Tools
The CLI works seamlessly with popular CI/CD solutions, including GitHub Actions, Jenkins, and GitLab CI. This compatibility allows teams to leverage their existing workflows while benefiting from the CLI's automation capabilities.
Conclusion
The Amazon SageMaker Unified Studio CI/CD CLI enhances the deployment process for data and AI applications, facilitating collaboration between data and DevOps teams. By automating key aspects of deployment, it ensures that applications can be promoted through various stages efficiently and securely.
