Scaling iOS CI/CD at Capital One
Capital One is one of the top 10 largest banks in the US, providing a broad spectrum of financial products and services to consumers, small businesses, and commercial clients through a variety of channels.
With more than 70 million customer accounts, providing secure access and excellent customer service via their mobile apps is essential to Capital One’s success. Capital One has an internal team that provides tooling to its mobile engineers across the US, the UK, and Canada. This team aims to provide the best possible digital tools to their internal customers (mobile engineers) so they can, in turn, ship the highest quality application to Capital One’s external customers. In this mobile-first world, happier customers are often the direct result of improvements in the quality of the mobile experience.
Capital One understands that their customers want an awesome mobile experience, so the company had been consistently growing its mobile engineering team. However, it soon realized it did not have the mobile CI capacity to accommodate these engineers. With the mobile team kicking off upwards of 3,000 builds a day, Capital One concluded that it was time to update their in-house mobile CI solution to match their mobile ambitions and better support their developers.
“Largely we want to make sure we have enough capacity and enough speed to keep the developers happy,” said Alex Niderberg, senior manager and lead software engineer at Capital One.
We want to make sure that we have enough capacity and enough speed to keep the developers happy.
Capital One decided a scalable, centralized virtualization platform could provide a consistent environment for developers, allowing them to effectively test results and ensure all mobile features were stable before shipping to customers.
Security Is a Top Priority
As a financial services company, Capital One knew that the bar for security would be extraordinarily high as they figured out how to create scalable infrastructure. The team needed a solution that would operate in an isolated environment they could ultimately control, while also allowing it to connect with other trusted development environments.
Before implementation, Capital One evaluated MacStadium’s VMware private cloud to ensure the solution met its security and compliance standards. “Before we were really able to start working with the MacStadium platform, there was a very thorough security evaluation,” said Niderberg. “In addition, there are continued checks to make sure that it’s ultimately meeting the security and compliance standards of the company.”
Given MacStadium’s continued investment in security and resulting ISO 27001, SOC 1, and SOC 2 certifications, Capital One was confident it found the secure environment to expand its mobile development operations.
Impact on Capital One
When this project kicked off, Capital One’s mobile team had three Mac build machines which were very limited capacity given the large number of internal mobile engineers. As a predominately Mac software engineering shop, purchasing additional Mac hardware would be easy for the internal team and allow them to move quickly. However, in the long run, they wanted a highly scalable, cloud-based Mac infrastructure solution. MacStadium offered the perfect solution to meet their long-term needs for scalable cloud-based CI/CD infrastructure.
To eliminate the environment inconsistency across bare metal hardware, Capital One decided to leverage MacStadium’s VMware private cloud to virtualize their development environments. With VMware vSphere, the Capital One team follows a “baking” process to add new versions of key iOS development tools (Xcode, SwiftLint, RVM, etc.) and then record that state into a golden master VM template. This stable master VM is then deployed using automation workflow tools like Terraform and Ansible across available hardware. This creates a foundational building block, allowing the team to guarantee a consistent, validated build environment.
We could re-provision our whole fleet of VMs in 10 minutes now... which is awesome.
In addition, Capital One was able to speed up development with the Pure Storage all-flash SAN included with their MacStadium private cloud. By combining vSphere with the Pure SAN, Capital One developers are able to clone a VM in around 10 seconds, where it had previously taken 20-30 minutes to clone one VM. “We could re-provision our whole fleet of VMs in ten minutes now,” said Niderberg, “which is awesome since we can accommodate changes a lot easier. We really don’t want to maintain state on these different VMs, so if we wanted to upgrade Xcode for example, we can just blow away all of these VMs, and re-provision them.”
Capital One has improved the quality of its builds, decreased build times, and scaled development capacity by creating a consistent build environment with MacStadium. As the team continues to mature its build approach, developers add validation to the rollout process to ensure infrastructure change does not cause any problems for the mobile engineers.
Based on developer feedback, Capital One and MacStadium continuously refine the system so the infrastructure can continue to scale with the business and provide a secure and consistent build environment for the mobile development team.