Logo

Simulators, Emulators, and Devices -- Which is right for your Mobile CI Builds?

By Jeff Vincent|

January 26, 2021

In the end, the answer to the question we posed in the title will be inextricably linked to the particulars of your operation. However, as MacStadium works with teams facing this very challenge, we have discovered a few insights that may help your team to navigate the ins and outs of this question.

Physical Devices

It is considered best practice to test applications on physical handsets before release. This is largely because of the many nuances inherent in not only the performance of the OS on the phone, but also within the internals of the device itself.

Write some text...

The advantage of testing on physical devices is bulletproof confidence of an application’s performance on a given device.

Write some text...

As enticing as this aforementioned certainty is, this is often an elusive target, as there is an enormous number of phones on the market, and maintaining a working set of devices that is representative of that variety is no small feat.

Emulators

Emulators are software that emulates the operating system that would be running on a given handset, and the code that emulates the internals of the device itself. Basically, it emulates the device and the OS.

Write some text...

The real advantage of an emulator for mobile CI is that an entire bank of real devices can be emulated with a high degree of confidence and relatively little overhead, as compared to builds executed on physical devices.

Write some text...

The challenge inherent in using emulators is that they may not offer an identical emulation of every device your team is targeting.

Additionally, although the overhead for running tests on emulators is significantly less than that it would be on physical devices, there is still significantly greater overhead to run emulator processes as compared to simulator processes, which translates into longer build times – often viewed as the arch-enemy of modern DevOps.  

In the end, we find that most teams who are focused exclusively on iOS CI choose to forego the extended build times that an emulator brings. These teams generally use a simulator instead.

Simulators

Simulators are software that simulates only the OS that would be running on a given device, with no attention paid to the underlying internals of a given device. The majority of teams that use MacStadium run their tests on Apple’s Xcode simulator.

Write some text...

The real advantage of simulators for mobile CI is their lightweight nature, and by extension, their relatively fast execution rates.

Write some text...

The lightweight nature of simulators comes at a cost, of course. Because simulators don’t concern themselves with replicating the underlying device at all, a good number of potential nuances can be whitewashed away, only to be caught at a later testing stage, or even in a beta release.

Android Emulators on a Mac?

Most MacStadium customers use Xcode’s simulator for their iOS builds, but there is a handful of customers doing some very interesting things with Android emulators running on Mac VMs. To read about one, in particular, check out this Medium post from the iFood team.

TL;DR

Where most of our customers are using a simulator, we find that it all depends on your team. If you’d like to find out more about what our customers are doing (not just with simulators, emulators, or physical devices), check out our customers page, and see how companies like Jamf and Dropbox are using MacStadium. Interested in learning more about what MacStadium can help your organization achieve? Reach out and talk to one of our sales engineers today to see how MacStadium can help.

Share this article

Logo

Orka, Orka Workspace and Orka Pulse are trademarks of MacStadium, Inc. Apple, Mac, Mac mini, Mac Pro, Mac Studio, and macOS are trademarks of Apple Inc. The names and logos of third-party products and companies shown on the website are the property of their respective owners and may also be trademarked.

©2023 MacStadium, Inc. is a U.S. corporation headquartered at 3525 Piedmont Road, NE, Building 7, Suite 700, Atlanta, GA 30305. MacStadium, Ltd. is registered in Ireland, company no. 562354.