Customizing and Code-Signing a Mobile App
Background
As we saw in “Bring Customers Joy With Automation”, Appian builds custom versions of the mobile Appian application for its customers. To do so, our old process was to apply the customer’s desired settings and images to our mobile code base, run the build, and then sign the application as the customer.
The problem.
Our process worked, but it did take around 20 minutes to build. That required a build machine, which wasted money and time. We don’t like unnecessary waste.
Here is an example of what we had before — a fastlane Fastfile with two lanes:
- customize_build: Configures the background color and welcome message of the application.
- build_custom_app: Takes the arguments provided to fastlane, calls the customize_build lane, and then builds the iOS application. The example doesn’t take 20 minutes, but imagine if it did!
The original build.
platform :ios do desc 'configures the background color of the application, along with the welcome message' lane :customize_build do |options| customize_build(options) end desc 'customize and build the iOS application' lane :build_custom_app do |options| build_custom_app(options) end end # method to configure the iOS application’s welcome message and background color def customize_build(options) welcome_message = options[:welcome_message] || 'Hello World!' background_color = options[:background_color] || '#FFFFFFFF' config_filepath = options[:config_filepath] || '../iOSExample/iOSExample/configurations.plist' update_plist( plist_path: config_filepath, block: proc do |plist| plist[:WelcomeMessage] = welcome_message plist[:BackgroundHexColor] = background_color end ) end # customize and build the iOS application def build_custom_app(options) customer_assets = options[:customer_assets] || ENV['APPIAN_CUSTOMER_ASSETS'] || 'lyndsey' customize_build(options) # get the keychain from Vault so that the app can # be signed as the customer. # # remember, we introduced get_keychain_from_vault in the article # Automate Securing Code Signing Assets # https://bit.ly/32Tiv0U # keychain_data = get_keychain_from_vault( vault_addr: ENV[“VAULT_ADDR”], keychain_name: customer_assets ) unlock_keychain( path: keychain_data[:keychain_path], password: keychain_data[:keychain_password], set_default: true ) # turn off automatic code signing and set up the provisioning # and team id for the application to match the customer's disable_automatic_code_signing(path: './iOSExample/iOSExample.xcodeproj') code_signing_identity = code_signing_identity_from_keychain(keychain_data[:keychain_path]) update_project_provisioning( xcodeproj: './iOSExample/iOSExample.xcodeproj', profile: "./iOSExample/Yillyyally.mobileprovision", build_configuration: "Release", code_signing_identity: code_signing_identity ) update_project_team( path: './iOSExample/iOSExample.xcodeproj', teamid: '57738V598V' ) # now build the application with the customer's customizations # provisioning profiles, and code signing identity build_app( scheme: 'iOSExample', project: './iOSExample/iOSExample.xcodeproj', output_directory: 'test_output', output_name: 'example.ipa', export_options: { method: "app-store", provisioningProfiles: { "com.yilly.yally" => "359e767c-5f71-4b2e-aedd-17645f951e02" } }, export_team_id: '57738V598V', xcargs: "CODE_SIGN_IDENTITY=\"#{code_signing_identity}\"" ) end def code_signing_identity_from_keychain(keychain_filepath) identity_output = Fastlane::Actions.sh('security', 'find-identity', '-v', '-p', 'codesigning', keychain_filepath) UI.user_error!('Keychain does not contain a single valid signing identity') unless identity_output.match(/1 valid identities found/) identity_output.lines.first.chomp.sub(/.*"([^"]+)".*/, '\1') end
Now, 20 minutes may not seem like a lot, but we have a lot of customers, each with a custom version of their application. For every customer, we have to manually start a Jenkins job with Vault token for each of these requests. When you think about it, we really shouldn’t have to build the application again — it was already built when we released the mobile application to Apple’s App Store and Google’s Play Store. The only differences were some settings files, some images, and the fact that it needed to be code signed using the customer’s code signing assets.
The idea.
We initially tried finding the latest released application from the iOS App Store and the Android Google Play Store, downloading the application, changing the settings files to match what the customer wants, swapping out the images in the build, and finally re-signing the application.
That seems pretty simple, right?
The implementation.
First, we had to write the code to download the already built and published application. We quickly found out that there was no programmatic way to download the latest released version of the iOS application. That threw a wrench into our plan.
Instead, for each release of our iOS and Android application, we decided to archive the application along with the image catalog and download that build each time the customer wants a custom version of the application. In the example below, I simulate that by putting the built applications in the GitHub Releases page.
Downloading the latest release.
require 'open-uri' require 'json' require 'tempfile' platform :ios do desc 'download the latest released app for iOS' lane :download_latest_release do |options| download_latest_release(options) end end def download_latest_release(options) # use GitHub's REST endpoint to download the latest release result = github_api( http_method: 'GET', path: '/repos/lyndsey-ferguson/CustomizeExistingAppExample/releases/latest', api_token: File.read(File.absolute_path('../.github_token')) ) body = JSON.parse(result[:body]) # Set up the regex and name for Android or iOS asset_name_regex = // application_bundle_name = '' if ENV["FASTLANE_PLATFORM_NAME"] == "ios" asset_name_regex = %r{example_release_.+\.zip} application_bundle_name = 'package.zip' else asset_name_regex = %r{app-release-unsigned.apk} application_bundle_name = 'android.apk' end # Find the name of the Android or iOS released app platform_specific_assets = body['assets'].find do |asset| asset['name'] =~ asset_name_regex end browser_download_url = platform_specific_assets["browser_download_url"] # download the release to a temporary file zipfile = Tempfile.new(['latest_release', '.zip']) zipfile.binmode open(browser_download_url) do |download| zipfile.write(download.read) end zipfile.close latest_release_dir = File.absolute_path('../latest_release') FileUtils.mkdir_p(latest_release_dir) zip_package_path = File.join(latest_release_dir, application_bundle_name) # copy the downloaded release to a local directory FileUtils.rm_f(zip_package_path) FileUtils.cp(zipfile.path, zip_package_path) zip_package_path end
The fastlane lane download_latest_release.
Once the application is downloaded, it needs to be unarchived, customized, resigned, and archived. Fortunately, both Android and iOS applications are zip archives, so it’s simple to unzip them to access the application contents.
In the case of the iOS application, we are archiving both the iOS “ipa” and the image assets in a parent zip folder. We need to include the same image assets that were associated with a particular release to ensure that we are not missing any images.
Unarchiving
We need to download the latest release and then unarchive the ipa and image assets bundle into a temporary directory. Then, we unarchive the ipa into its own temporary directory so that we can work on it.
require 'yaml' require_relative 'binary_plist' require_relative 'ios_customize_build' platform :ios do desc 'download, customize, and sign the app according to customer\'s needs' lane :customize_built_app do |options| customize_built_app(options) end end def customize_built_app(options) customer_assets = options[:customer_assets] || ENV['APPIAN_CUSTOMER_ASSETS'] || 'puppy' zip_package_path = download_latest_release custom_built_app_path = File.expand_path(File.join('~/Desktop', "#{customer_assets}.ipa")) # unzip the combined ipa & images package into a tmp directory Dir.mktmpdir("latest_release_pkg") do |latest_release_pkg_path| Dir.chdir(latest_release_pkg_path) do sh("unzip -o -q #{zip_package_path}") example_ipa_filepath = File.absolute_path('example.ipa') app_iconset_dirpath = File.absolute_path('AppIcon.appiconset') # now that we have the package unzipped into a temporary folder # lets unzip the ipa so we can operate inside of it Dir.mktmpdir("customize_built_app") do |unzipped_ipa_path| Dir.chdir(unzipped_ipa_path) do sh("unzip -o -q #{example_ipa_filepath}") # TODO: customize the app and code sign it # zip up the customized app to desired file path sh("zip -qr #{custom_built_app_path} .") end end end end puts "Built mobile app: #{custom_built_app_path}" end
Customizing settings.
Now that we have the application downloaded and unarchived, we can start customizing it. First, let’s customize the settings by adding the following code underneath the TODO section above.
# TODO: customize the app and code sign it # Load the customer’s settings from a YAML settings file customer_config_filepath = File.absolute_path("../#{customer_assets}/#{customer_assets}.yaml") if File.exist?(customer_config_filepath) customer_config_file = YAML.load_file(customer_config_filepath) welcome_message = customer_config_file['WelcomeMessage'] || 'Hello World!' background_color = customer_config_file['BackgroundHexColor'] || '#FFFFFFFF' end welcome_message = options[:welcome_message] unless options[:welcome_message].nil? background_color = options[:background_color] unless options[:background_color].nil? example_ipa_payload_dir = File.join(unzipped_ipa_path, "Payload") app_bundle_path = File.join(example_ipa_payload_dir, 'iOSExample.app') configurations_plist_filepath = File.join(app_bundle_path, 'configurations.plist') # Apple stores the xml plist files in a binary format in order to # save space. We need to operate on a plain text file, so we use # the plutil tool to convert it to XML. sh("plutil -convert xml1 #{configurations_plist_filepath}") customize_build( welcome_message: welcome_message, background_color: background_color, config_filepath: configurations_plist_filepath ) # Convert the text plist XML file back to a binary XML file sh("plutil -convert binary1 #{configurations_plist_filepath}")
This code loads a YAML file that has been configured with the customer’s settings. It then calls the method we declared earlier to customize the background color and the welcome message. It’s worth noting that plist files inside of an iOS application are stored as binaries to save space, so they have to be converted to text files, updated, and then converted back to binaries.
Customizing images.
Next, we need to write the code that customizes the icons using what the customer gave us. Let’s append the following to the code snippet above:
# ... # Convert the text plist XML file back to a binary XML file sh("plutil -convert binary1 #{configurations_plist_filepath}") copy_customer_images(customer_appiconset_dirpath, latest_release_pkg_path) compile_images(latest_release_pkg_path, example_ipa_payload_dir) # zip up the customized app to desired file path sh("zip -qr #{custom_built_app_path} .") end end end end end # of the customize_built_app method APP_ICON_ASSET_DIR = 'AppIcon.appiconset' def app_icon_asset_path(latest_release_pkg_path) File.join(latest_release_pkg_path, APP_ICON_ASSET_DIR) end def copy_customer_images(customer_appiconset_dirpath, latest_release_pkg_path) remove_app_icon_assets(latest_release_pkg_path) FileUtils.cp_r("#{customer_appiconset_dirpath}/.", app_icon_asset_path(latest_release_pkg_path)) end # remove pre-existing application icons just in case the # customer did not provide an icon for one of the many # resolutions that iOS applications support. We do not want # the standard icon to appear for those resolutions def remove_app_icon_assets(latest_release_pkg_path) FileUtils.rm_r(Dir.glob("#{app_icon_asset_path(latest_release_pkg_path)}/*.png")) end # the `xcrun actool` requires the minimum deployment target # as one of its command line options. Rather than hard-coding # it, we get that value from the application Info.plist. def minimum_deployment_target plist_filepath = 'Payload/iOSExample.app/Info.plist' temporary_plist_file = Tempfile.new FileUtils.cp(plist_filepath, temporary_plist_file.path) info_plist = Plist.parse_binary_xml(temporary_plist_file.path) info_plist.fetch('MinimumOSVersion', '10.0') end def compile_images(latest_release_pkg_path, example_ipa_payload_dir) FileUtils.mkdir_p("#{latest_release_pkg_path}/build") command = "xcrun actool #{latest_release_pkg_path} " command += "--compile #{example_ipa_payload_dir}/iOSExample.app " command += "--minimum-deployment-target #{minimum_deployment_target} " command += '--app-icon AppIcon --platform iphoneos ' command += "--output-partial-info-plist #{latest_release_pkg_path}/build/partial.plist " sh(command) end
Note how we have to first put the customer’s images into the iconset bundle and then use the xcrun actool to compile that iconset into an Assets.car file to put into the application bundle.
Code-signing.
Now that we have the customized application, we need to code-sign it with the customer’s certificate. Continuing in the customize_built_app method where we left off with compile_images:
compile_images(latest_release_pkg_path, example_ipa_payload_dir) keychain_data = get_keychain_from_vault( vault_addr: 'http://127.0.0.1:8200', keychain_name: customer_assets ) keychain_password = keychain_data[:keychain_password] keychain_path = keychain_data[:keychain_path] unlock_keychain(path: keychain_path, password: keychain_password, set_default: true) cert = certificate_id_from_keychain(keychain_path) sign_frameworks(cert, keychain_path) prepare_app_bundle(app_bundle_path, customer_profile_pathname) prepare_entitlements(cert, app_bundle_path, keychain_path, customer_profile_pathname) # zip up the customized app to desired file path sh("zip -qr #{custom_built_app_path} .") end end end end end # of the customize_built_app method # find the certificate in the keychain so that we can get the # id and use it for code signing. def certificate_id_from_keychain(keychain_filepath) identity_output = Fastlane::Actions.sh('security', 'find-identity', '-v', '-p', 'codesigning', keychain_filepath) UI.user_error!('Keychain does not contain a single valid signing identity') unless identity_output.match(/1 valid identities found/) identity_output.lines.first.split[1].downcase end # sign any frameworks inside of the application def sign_frameworks(cert, keychain_filepath) frameworks = Dir.glob('Payload/iOSExample.app/Frameworks/*.dylib').map { |s| "'#{s}'" }.join(' ') unless frameworks.empty? sh("/usr/bin/codesign -f --keychain \"#{keychain_filepath.shellescape}\" -s \"#{cert}\" #{frameworks}") end end # remove any existing signature and put the provisioning profile # inside of the app bundle def prepare_app_bundle(app_bundle_path, profile_pathname) # Remove existing signature file FileUtils.rm_rf("#{app_bundle_path}/_CodeSignature") # Copy associated provisioning profile to app FileUtils.cp(profile_pathname, "#{app_bundle_path}/embedded.mobileprovision") end # create a plist from the provisioning profile so that we # can get the team id and the application bundle id def plist_for_profile(profile) output_plist_file = Tempfile.new `security cms -D -i \"#{profile}\" > \"#{output_plist_file.path}\"` if File.exists?(profile) Plist.parse_xml(output_plist_file) || {} end # customize the entitlements file with values from the customer's # mobileprovision file. def customize_entitlements_xml(xml_path, customer_app_id, customer_team_id) xml_hash = Plist.parse_xml(xml_path) xml_hash['application-identifier'] = customer_app_id xml_hash['com.apple.developer.team-identifier'] = customer_team_id if xml_hash.key?('com.apple.developer.team-identifier') xml_hash.save_plist(xml_path) end # pull out the application's existing entitlements and customize the values that # are associated with the customer, and then re-embed them into the application binary def prepare_entitlements(cert, app_bundle_path, keychain_filepath, mobileprovisioning_filepath) # File exists in builds that were created with xcode versions < 10. # Entitlements are now embedded in the app binary which are read and manipulated before signing them back into the binary. FileUtils.rm_f("#{app_bundle_path}/#{File.basename(app_bundle_path, '.*')}.entitlements") entitlements = Tempfile.new() sh("/usr/bin/codesign -d --entitlements :\"#{entitlements.path}\" \"#{app_bundle_path}\"") mobileprovisioning_data = plist_for_profile(mobileprovisioning_filepath) team_id = mobileprovisioning_data.dig('Entitlements', 'com.apple.developer.team-identifier') app_id = mobileprovisioning_data.dig('Entitlements', 'application-identifier').sub("#{team_id}.", '') customize_entitlements_xml(entitlements.path, app_id, team_id) sh("/usr/bin/codesign -f --keychain \"#{keychain_filepath.shellescape}\" -s \"#{cert}\" --entitlements \"#{entitlements.path}\" \"#{app_bundle_path}\"") end
This code uses the Vault token role custom-mobile-apps-read-policy (as seen in “Automatically Secure Code Signing Assets”) to retrieve the keychain and password in preparation for code signing. After unlocking the keychain, it proceeds to get the keychain’s certificate ID to sign any frameworks inside the app bundle as well as the app bundle itself.
Take note that before code signing the application itself, we first have to retrieve the entitlements that were embedded into the application binary, update them to contain the bundle ID and application ID that were in the customer’s mobileprovision file, and re-embed those entitlements back into the application binary. This happens in the prepare_entitlements method and its various helper methods.
Validations
Finally, we also added code to ensure that what we built was correct (not shown here). This code
- Removes transparencies from any launch icons, as apps that have them will be rejected from App Stores.
- Fails and alerts us if images were not of the correct size.
- Fails and alerts us when certificates or provisioning profiles were expired or missing.
- Fails and alerts us if the customizations were invalid.
The result.
With these changes, we no longer have to build the application each time. Instead, we can apply some basic customizations in a few minutes rather than waiting 20 minutes for the build.
I’ve written up the example code for both an Android and an iOS application, and you can find it at this GitHub repo. If you have other ideas of how to make this process faster, or if this helps you with your own automation, I’d love to hear from you!
Afterword
This leads me to think: what other ways can we all benefit from this knowledge? There may be other instances where we need to make a change to an application, but the code or project doesn’t actually have to change. For example, a React Native application that loads a bundled JavaScript file could take advantage of this for code changes in only JS files.
If you find this kind of work interesting and want to work with creative, passionate, and smart people, there are plenty of other projects to work on like this at Appian. Apply today!