When iOS CI/CD Config is not Just a File or a Dashboard

2020, Mar 15

A while ago, when I heard the term “CI/CD”, I always thought of a dashboard to drag and drop, upload certificates and input the scheme… blah blah. That’s all! And I was kinda believe that a good CI/CD platform must be like that: convenient, as few setups as possible. Now, I have a different viewpoint. What I very much expect in a CI/CD platform is the ability to customize workflow. This does not mean those drag-and-drop platforms are inadequate. It depends on the scale of your project and the problems you want to solve.

In this post, I will talk about those problems on a general level. Details about how to tackle each will be discussed later in the upcoming posts.

A Large Code Base

In my project, there are many engineers contributing to a fairly large code base. How large is it? - you may wonder. Well, let’s imagine there are a lot of product features in the project. Each feature has its A/B testing logic, making our code base even bigger. Apart from product features, we also have engineering work. Some of them need A/B testing as well, in order to safely roll out to users. And of course, we do cover unit tests and UI tests for both features and engineering work. In addition to A/B testing, we have features toggles (as part of trunk-based development) which means more code than needed is added per feature.

Therefore, our code base has been growing over time. With our project, it takes:

Those are the required steps to run on CI against a change in a merge request. Doing a simple math and you will realize that a developer has to wait for nearly an hour to land his/her change on master (assume other checks such as approvals and code linting are already satisfied).

With such an increasingly large code base, CI/CD configuration is not just to make things work, but to make things work efficiently.

Build Time

iOS build time improvement is a classic problem of large code base projects. There are some tips to improve the project build time including:

Those approaches are not much related to CI/CD configuration. Some tips are suitable for local runs but not for CI/CD runs. For example, changing some optimization flags would reduce build time but it screws up code coverage generation. We have to alternate build settings for CI/CD in such cases.

A Tale of UI Tests

We are pretty proud of our UI tests. Not only do they cover a lot of features in the app but also they are very useful for feature development/bug fixes (especially when we want to simulate complicated workflows without Staging backend). However, the more tests we write, the longer time it takes to execute all test suites. Reducing the overall test execution time is definitely a CI/CD work that drag-and-drop is not capable of.

A simple idea is to split UI tests and run them in parallel jobs. Then you need to answer the following questions:

Another problem with UI tests is that they seem to be more unstable than unit tests. Dealing with unstable tests is not just iOS work, but also a CI/CD work. For example, you need to design your CI/CD pipelines so that it’s less vulnerable to unstable tests and the time it takes to retry (tests or jobs) is as fast as possible. And you need to track those unstable tests (not manually) so that you could revisit to investigate them.

CI Resources

Making good use of CI resources is also a key to make our CI/CD system work at its best. When there are more available runners, try to use them. However, determining CI resources status is not always easy. It usually involves sending api requests to the CI/CD platform (for ex. Gitlab).

Also, when we allocate resources differently, the number of CI jobs are dynamic. How to configure that?

A/B Testing for CI/CD

In our project, we not only have AB testing for features in the app, but also for CI/CD features. Changes related to CI/CD usually affects other engineers. I need to emphasize again that there are many engineers contributing to the project, not just 3-4 engineers. To avoid blocking others, we always think of a safe rollout for important CI/CD changes. If there is any unexpected issue that block others, we can just roll back the change at ease.

A rollout config is just simply a yaml file (hosted somewhere), like this:

name: 'A feature'
description: 'Description of the feature'
rollout:
  - if: 'XCODE_VERSION'
    match: '11'
    then: 2
  - if: 'CI_COMMIT_REF_NAME'
    match: '(master|release)'
    then: 4
  - default: 3  # 👈 use this value in CI/CD code

Automation

In our project, we try to automate tasks/chores as much as possible. Those automated tasks are usually non-standard problems and, of course, are something we need to code on our own.

Conclusion

With some use cases mentioned above, you can imagine that CI/CD work for iOS is not just integrating to CI/CD platform so that we can build and test our project. It’s not that simple, or your project is not complicated enough 🤔. It’s not just about a config dashboard or a config file… To me, it requires more implementation code to get things done in an appropriate way. The advantage of building them instead of relying on a 3rd party platform support is that you have more control and can customize them based on your needs. And it’s sometimes fun.

…

I know this post is a bit general, and lacks details (which you expect more). But I think it would be better if you have an overview first, and then we can dive into details later. So, for those who are interested in, stay tuned for the upcoming posts 😉.

tech
ci