Blazing fast continuous integration at Box
Box is an enterprise-grade service that lets users securely manage, share and access content from anywhere.
Box solves simple and complex challenges, from sharing and accessing files on mobile devices to sophisticated business processes like data governance and retention. Box empowers enterprises to revolutionize how they work by securely connecting their people, information and applications. More than 61 million users and 87,000 businesses — including 69% of the Fortune 500 — trust Box to manage their content in the cloud. Founded in 2005, Box is headquartered in Redwood City, CA, with offices across the United States, Europe and Asia. Simply stated, Box makes it easier for people to securely share ideas, collaborate and get work done faster.
Box uses nearly every cloud provider — AWS, Azure, Google Cloud — different teams use them for different purposes. However, those providers do not provide genuine Apple Mac infrastructure. Before they found out about MacStadium, Box’s iOS and Mac development teams started by hosting Macs in-house for automated continuous integration (CI) build and test workloads. However, they quickly realized they did not want to spend time and resources within the organization building, maintaining and troubleshooting Mac infrastructure. The iOS and Mac teams wanted to have the same level of simplicity that major cloud platforms offered for building and testing their apps.
Ray Sennewald, a senior software engineer at Box put it best. “Developers are getting more and more used to having cloud providers. I can’t tell you the number of times somebody has come up to me and said, ‘Hey, how come we can’t use AWS and just deploy a Mac over there?’ and I’d say, ‘Oh man, I would if I could.’”
Box needed a Mac infrastructure solution that was as simple as working with a large cloud provider. At the same time, iOS and Mac development created unique needs for developers and DevOps engineers. Specifically, they needed fast virtual machines that could be fully automated in order to build and test their apps on genuine Apple hardware.
“That went into our decision process as we didn’t want to have Mac hardware on hand,” said Sennewald. “As we were trying to figure out how we wanted to solve this problem, it was nice to have a partner that allowed us as a team to focus a lot more on providing the software and the tools and optimizing our networking considerations as opposed to dealing with the hardware.”
Another consideration was security. “We’re managing customer data, so security is a huge concern,” said Sennewald. “It’s not something that [the development team] wants to worry about, but we can’t use cloud providers unless they maintain all of the security requirements that Box has.” MacStadium maintains ISO 27001, SOC 1 and SOC 2 across all data centers and the company as a whole. MacStadium also provides private clouds with dedicated infrastructure where the security team at Box was able to lock down every aspect of their environment. Together, this allowed MacStadium to meet or exceed Box’s security thresholds.
Ultimately, the Box team decided on MacStadium for their iOS and Mac development needs. They were able to migrate their CI workflow to a secure private cloud built with genuine Mac Pro hardware running on VMware and connected to super-fast SAN storage.
Mac CI in the Cloud
Given the nature of the service they provide, building and releasing quality apps at a rapid pace is essential to Box’s success. Using their MacStadium private cloud, Box engineers created a lightning-fast platform for creating ephemeral virtual machines as the backbone of their CI workflow. Ephemeral — or single-use — VMs are identical for each build to create a consistent, reliable build.
“We use Jenkins as our CI platform, and we use the VMware vSphere Jenkins plug-in, which allows us to do some cool things. Specifically, it gives us one-time-use build VMs,” said Sennewald. “We use Terraform to build out our VMs, so infrastructure is code. With a little bit of scripting we add those as nodes to Jenkins and then using the plug-in, we can allow the VMs to revert back to a snapshot after every build.”
We wanted to have a clean environment for every build, for every developer, because otherwise you’re just going to run into problems.
“It gives us container-like functionality but for macOS, and that’s something we wanted to have at Box,” said Sennewald. “We wanted to have a clean environment for every build, for every developer, because otherwise you’re just going to run into problems.”
Impact on Box
After moving to MacStadium, the Box team realized immediate value by not having Mac hardware under their desks or in the back of a closet. The application engineering team created a CI workflow that allowed their iOS and Mac developers to access clean VMs powering their continuous integration pipeline. Recently, Box upgraded from vSphere 6.0 to 6.5 and replaced an old disk-based SAN with an all-flash Pure Storage SAN array. This is when they began to see radical changes in their performance.
“We have around four templates, and we provision hundreds of VMs from those. On the old SAN and running an older version of vSphere, it would take us 20 to 30 minutes to clone one VM. Now, we were able to get that down to around 10 seconds,” said Sennewald. “We can re-provision our whole fleet of VMs in minutes now, which means we can accommodate changes a lot easier. This is huge for us. Previously we wouldn’t accommodate Xcode upgrades unless we absolutely had to. Developers would be at my door begging me, ‘Hey, can we get Xcode 9.2?’ Previously I’d say, ‘That’s going to take a lot of work.’ Now it’s a lot easier for us to say yes.”
“This is something we didn’t have before MacStadium. We used to manage these VMs manually,” continues Sennewald. “So, it has been integral for us as it allows us to ensure that we get clean builds for every VM.”
Since signing on with MacStadium, Box has been able to leverage a dedicated private cloud built on genuine Mac hardware to its full potential and speed up their CI pipeline. Using MacStadium infrastructure that combines Apple computers, VMware virtualization, and Pure Storage, Box was able to reduce the time it took to clone a virtual machine from 20-30 minutes down to about 10 seconds. This was important as Box wanted one-time-use VMs to ensure a clean environment for every build, for every developer. With VMware, the Box team is able to parallelize their functional tests and get results quickly, while having other VMs run Xcode builds as fast as possible.
Being completely honest, I want the easiest tool for the job. MacStadium can provide that to us utilizing VMware.
“This is pretty powerful. It’s something that we wouldn’t have the capability of doing if we were on bare metal hardware. Having an infrastructure provider like MacStadium is huge to us,” said Sennewald.