Travis CI

How Travis CI Halved Boot Times for OS X and iOS Testing

Travis CI is one of the leading continuous integration platforms in software development today. Originally focused on the Ruby language, Travis CI has grown rapidly from a free for open source CI tool to a major player in the industry, offering both open source and closed source options to single developers, startups, enterprises, and all types of software companies in between.

We had pressure and requests early on like, ‘Hey, we love Travis CI, but we’ve got this need for iOS testing as well’.

Dan Buch
Builder, Travis CI

Before MacStadium

After steady progress building their offering for many language runtimes on Linux, Travis CI made a conscious decision to support multiple operating systems, as many SaaS platforms do when they want to reach the largest possible audience. “We had pressure and requests early on like, ‘Hey, we love Travis CI, but we’ve got this need for iOS testing as well’”, stated Dan Buch, a Builder at Travis CI.

As the company focused on maintaining a bootstrapped budget, Travis CI did not have the capital to invest in a large Mac infrastructure upfront, which would be necessary as Apple’s licensing terms provide for virtualization of OS X only on top of Mac hardware. The lack of a formal Mac OS X cloud hosting provider made it clear that private hosted infrastructure would be required to provide an OS X build environment.

As Travis CI moved towards an OS X offering, they found a provider for testing, but they were working under the radar. “We didn’t really have a direct relationship with someone that was doing support for us,” Buch stated. A public, commercial offering for rapid environment testing and development was not apparent in the market. This limited their ability to perform planned reliability benchmarks and assess the future scalability of an OS X continuous integration ecosystem.

The Solution

Data Center Racks

When Travis CI connected with MacStadium, all worries about growth of the platform on OS X and iOS began to disappear. Conversations evolved rapidly into a solution and strategic plan for Travis CI in one of MacStadium’s tailored private cloud solutions powered by VMWare.

Speaking on developing a solution for multiple operating systems, Buch pointed out, “Our goal is to make it work as close as possible to what people expect from something they have running locally.” Leveraging extremely low latency connections at MacStadium, they’ve been able to offer parity of speed between Linux and OS X builds on servers at multiple cloud providers.

Moving to a private cloud at MacStadium also meant that Travis CI would have full control of dedicated bare metal for the VMware hypervisors. “We used to be a couple layers back with the previous provider and now we’re farther up the management layer which we like a lot better,” Buch shared. On moving to an Apple-centric IaaS provider like MacStadium for Mac hosting, Buch added, “It’s very clear what is under our control and what is expected of MacStadium in the relationship”.

Travis CI now controls their own layer on top of the VMware hypervisor to connect with a custom-built scripting system which optimizes the boot-up and shutdown of OS X and iOS virtual machines as fast as possible. Because of that, Travis CI is able to provide virtual machines for new builds on demand, similar to other popular cloud providers. These virtual machines are designed as a ‘clean room VM’ to provide a strict testing environment. Buch stated, “We have strong guarantees for consistency. We create a fresh clone of a base VM image for every single job”.

Utilizing high performance EMC VMAX SAN storage connected to their private cloud at MacStadium, Travis CI can deploy VM’s via a VMware Linked Clone image in seconds, with an overall time to green for new build VM’s averaging close to 60 seconds. Advanced features like dependency caching and arbitrary directory caching allow consistent testing without wasting resources. “We offer these primarily to our pro customers,” Buch added.

EMC VMAX SAN storage offers Travis CI a huge benefit in the number and age of testing images offered. Henrik Hodne, a lead infrastructure engineer, noted that “most of the people are using the OS X images to test iOS projects with Travis CI. Every time there’s a new iOS SDK out, we immediately get support requests to test against that new version”.

Previously, Travis CI needed external assistance spinning up new VM’s with a new iOS SDK version. All images for virtual machines were stored locally on each VMware host, so hard drive sizes limited the available range of iOS SDK version Travis CI could offer simultaneously.

Now, Travis CI can build and deploy images on demand while offering a larger variety of images to customers. “With shared storage, we have much more space so we can have multiple images at once. Every image since we started at MacStadium is still available,” added Hodne.

iOS Developers using Travis CI’s have varying goals. Many desire to test the most recent versions of iOS as soon as possible while others strive to continue supporting old iOS versions and devices for their application’s user base. The large, scalable SAN storage connected to Travis CI’s private cloud environment meets the requirements of both types of iOS developers.

“It made a big difference at MacStadium being able to better plan how quickly we expand.”

The Results

Travis CI is very happy with the private cloud solution in the 5 months at MacStadium they have been a customer, “It’s been an amazing experience so far — nothing but good things. We’re surprised at every turn with speed to resolving issues and the amount of detail provided along the way”, added Buch.

With bare metal access and full control over VMware vSphere, Travis CI is able to cut boot times in half for many testing images. “We’ve got a wrapper sitting in front of it now. It makes the process of booting up VM’s easier and the wrapper has some special stuff to boot as fast as possible,” stated Hodne. Travis CI has now been able to get boot times close to 60 seconds, depending on load.

Travis CI received assurances early on from MacStadium that adding more capacity to vSphere can be done very quickly. Buch agreed, “It made a big difference at MacStadium being able to better plan how quickly we expand”. Future growth is not a worry as “more aggressive planning is available now,” Buch added when discussing expansion opportunities.

Enterprise customers of Travis CI have benefited dramatically as well. Working together with MacStadium, Travis CI can now reliably construct new private repositories for enterprise customers on MacStadium servers based on an optimized deployment template. Buch stated, “The goal is for those customers to be managed by Travis CI but for them to have a direct support relationship with MacStadium”. This frictionless relationship allows Travis CI to configure and maintain the build environment while MacStadium supports the endeavor at the infrastructure level. This includes managing firewalls and VPNs, load balancing, network storage, disaster recovery, rapid scaling, imaging, and deployment.

Click here to hear Josh Kalderimis, Travis CI's vice president of product, talk more about their infrastructure in a recent panel discussion at VMworld 2018.