Amazon.com Inc.
08/19/2024 | Press release | Distributed by Public on 08/19/2024 15:15
Add macOS to your continuous integration pipelines with AWS CodeBuild
Starting today, you can build applications on macOS with AWS CodeBuild. You can now build artifacts on managed Apple M2 machines that run on macOS 14 Sonoma. AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces ready-to-deploy software packages.
Building, testing, signing, and distributing applications for Apple systems (iOS, iPadOS, watchOS, tvOS, and macOS) requires the use of Xcode, which runs exclusively on macOS. When you build for Apple systems in the AWS Cloud, it is very likely you configured your continuous integration and continuous deployment (CI/CD) pipeline to run on Amazon Elastic Cloud Compute (Amazon EC2) Mac instances.
Since we launched Amazon EC2 Mac in 2020, I have spent a significant amount of time with our customers in various industries and geographies, helping them configure and optimize their pipelines on macOS. In the simplest form, a customer's pipeline might look like the following diagram.
The pipeline starts when there is a new commit or pull request on the source code repository. The repository agent installed on the machine triggers various scripts to configure the environment, build and test the application, and eventually deploy it to App Store Connect.
Amazon EC2 Mac drastically simplifies the management and automation of macOS machines. As I like to describe it, an EC2 Mac instance has all the things I love from Amazon EC2 (Amazon Elastic Block Store (Amazon EBS) volumes, snapshots, virtual private clouds (VPCs), security groups, and more) applied to Mac minis running macOS in the cloud.
However, customers are left with two challenges. The first is to prepare the Amazon Machine Image (AMI) with all the required tools for the build. A minimum build environment requires Xcode, but it is very common to install Fastlane (and Ruby), as well as other build or development tools and libraries. Most organizations require multiple build environments for multiple combinations of macOS and Xcode versions.
The second challenge is to scale your build fleet according to the number and duration of builds. Large organizations typically have hundreds or thousands of builds per day, requiring dozens of build machines. Scaling in and out of that fleet helps to save on costs. EC2 Mac instances are reserved for your dedicated use. One instance is allocated to one dedicated host. Scaling a fleet of dedicated hosts requires a specific configuration.
To address these challenges and simplify the configuration and management of your macOS build machines, today we introduce CodeBuild for macOS.
CodeBuild for macOS is based on the recently introduced reserved capacity fleet, which contains instances powered by Amazon EC2 that are maintained by CodeBuild. With reserved capacity fleets, you configure a set of dedicated instances for your build environment. These machines remain idle, ready to process builds or tests immediately, which reduces build durations. With reserved capacity fleets, your machines are always running and will continue to incur costs as long as they're provisioned.
CodeBuild provides a standard disk image (AMI) to run your builds. It contains preinstalled versions of Xcode, Fastlane, Ruby, Python, Node.js, and other popular tools for a development and build environment. The full list of tools installed is available in the documentation. Over time, we will provide additional disk images with updated versions of these tools. You can also bring your own custom disk image if you desire.
In addition, CodeBuild makes it easy to configure auto scaling. You tell us how much capacity you want, and we manage everything from there.
Let's see CodeBuild for macOS in action
To show you how it works, I create a CI/CD pipeline for my pet project: getting started with AWS Amplify on iOS. This tutorial and its accompanying source code explain how to create a simple iOS app with a cloud-based backend. The app uses a GraphQL API (AWS AppSync), a NoSQL database (Amazon DynamoDB), a file-based storage (Amazon Simple Storage Service (Amazon S3)), and user authentication (Amazon Cognito). AWS Amplify for Swift is the piece that glues all these services together.
The tutorial and the source code of the app are available in a Git repository. It includes scripts to automate the build, test, and deployment of the app.
Configuring a new CI/CD pipeline with CodeBuild for macOS involves the following high-level steps:
- Create the build project.
- Create the dedicated fleet of machines.
- Configure one or more build triggers.
- Add a pipeline definition file (buildspec.yaml) to the project.
To get started, I open the AWS Management Console, select CodeBuild, and select Create project.
I enter a Project name and configure the connection to the Source code repository. I use GitHub in this example. CodeBuild also supports GitLab and BitBucket. The documentation has an up-to-date list of supported source code repositories.
For the Provisioning model, I select Reserved capacity. This is the only model where Amazon EC2 Mac instances are available. I don't have a fleet defined yet, so I decide to create one on the flight while creating the build project. I select Create fleet.
On the Compute fleet configuration page, I enter a Compute fleet name and select macOS as Operating system. Under Compute, I select the amount of memory and the quantity of vCPUs needed for my build project, and the number of instances I want under Capacity.
For this example, I am happy to use the Managed image. It includes Xcode 15.4 and the simulator runtime for iOS 17.5, among other packages. You can read the list of packages preinstalled on this image in the documentation.
When finished, I select Create fleet to return to the CodeBuild project creation page.
As a next step, I tell CodeBuild to create a new service role to define the permissions I want for my build environment. In the context of this project, I must include permissions to pull an Amplify configuration and access AWS Secrets Manager. I'm not sharing step-by-step instructions to do so, but the sample project code contains the list of the permissions I added.
I can choose between providing my set of build commands in the project definition or in a buildspec.yaml file included in my project. I select the latter.
This is optional, but I want to upload the build artifact to an S3 bucket where I can archive each build. In the Artifact 1 - Primary section, I therefore select Amazon S3 as Type, and I enter a Bucket name and artifact Name. The file name to upload is specified in the buildspec.yaml file.
Down on the page, I configure the project trigger to add a GitHub WebHook. This will configure CodeBuild to start the build every time a commit or pull request is sent to my project on GitHub.
Finally, I select the orange Create project button at the bottom of the page to create this project.
Testing my builds
My project already includes build scripts to prepare the build, build the project, run the tests, and deploy it to Apple's TestFlight.
I add a buildspec.yaml file at the root of my project to orchestrate these existing scripts.
version: 0.2
phases:
install:
commands:
- code/ci_actions/00_install_rosetta.sh
pre_build:
commands:
- code/ci_actions/01_keychain.sh
- code/ci_actions/02_amplify.sh
build:
commands:
- code/ci_actions/03_build.sh
- code/ci_actions/04_local_tests.sh
post_build:
commands:
- code/ci_actions/06_deploy_testflight.sh
- code/ci_actions/07_cleanup.sh
artifacts:
name: $(date +%Y-%m-%d)-getting-started.ipa
files:
- 'getting started.ipa'
base-directory: 'code/build-release'
I add this file to my Git repository and push it to GitHub with the following command: git commit -am "add buildpsec" buildpec.yaml
On the console, I can observe that the build has started.
When I select the build, I can see the log files or select Phase details to receive a high-level status of each phase of the build.
When the build is successful, I can see the iOS application IPA file uploaded to my S3 bucket.
The last build script that CodeBuild executes uploads the binary to App Store Connect. I can observe new builds in the TestFlight section of the App Store Connect.
Things to know
It takes 8-10 minutes to prepare an Amazon EC2 Mac instance and to accept the very first build. This is not specific to CodeBuild. The builds you submit during the machine preparation time are queued and will be run in order as soon as the machine is available.
CodeBuild for macOS works with reserved fleets. Contrary to on-demand fleets, where you pay per minute of build, reserved fleets are charged for the time the build machines are reserved for your exclusive usage, even when no builds are running. The capacity reservation follows the Amazon EC2 Mac 24-hour minimum allocation period, as required by the Software License Agreement for macOS (article 3.A.ii).
A fleet of machines can be shared across CodeBuild projects on your AWS account. The machines in the fleet are reserved for your exclusive use. Only CodeBuild can access the machines.
CodeBuild cleans the working directory between builds, but the machines are reused for other builds. It allows you to use the CodeBuild local cache mechanism to quickly restore selected files after a build. If you build different projects on the same fleet, be sure to reset any global state, such as the macOS keychain, and build artifacts, such as the SwiftPM and Xcode package caches, before starting a new build.
When you work with custom build images, be sure they are built for a 64-bit Mac-Arm architecture. You also must install and start the AWS Systems Manager Agent (SSM Agent). CodeBuild uses the SSM Agent to install its own agent and to manage the machine. Finally, make sure the AMI is available to the CodeBuild organization ARN.
CodeBuild for macOS is available in the following AWS Regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Sydney), and Europe (Frankfurt). These are the same Regions that offer Amazon EC2 Mac M2 instances.
Get started today and create your first CodeBuild project on macOS.
-- seb