Skip to main content

Customizing and Code-Signing a Mobile App

Lyndseyferguson Lyndsey Ferguson

Screen Shot 2021 07 29 At 2.39.37 PM

Background

As we saw in “Bring Customers Joy With Automation”, Appian builds custom versions of the mobile Appian application for its customers. To do so, our old process was to apply the customer’s desired settings and images to our mobile code base, run the build, and then sign the application as the customer.

The problem.

Our process worked, but it did take around 20 minutes to build. That required a build machine, which wasted money and time. We don’t like unnecessary waste.

Here is an example of what we had before — a fastlane Fastfile with two lanes:

  1. customize_build: Configures the background color and welcome message of the application.
  2. build_custom_app: Takes the arguments provided to fastlane, calls the customize_build lane, and then builds the iOS application. The example doesn’t take 20 minutes, but imagine if it did! 

The original build.

platform :ios do
  desc 'configures the background color of the application, along with the welcome message'
  lane :customize_build do |options|
    customize_build(options)
  end

  desc 'customize and build the iOS application'
  lane :build_custom_app do |options|
    build_custom_app(options)
  end
end

# method to configure the iOS application’s welcome message and background color
def customize_build(options)
  welcome_message = options[:welcome_message] || 'Hello World!'
  background_color = options[:background_color] || '#FFFFFFFF'
  config_filepath = options[:config_filepath] || '../iOSExample/iOSExample/configurations.plist'

  update_plist(
    plist_path: config_filepath,
    block: proc do |plist|
      plist[:WelcomeMessage] = welcome_message
      plist[:BackgroundHexColor] = background_color
    end
  )
end

# customize and build the iOS application
def build_custom_app(options)
  customer_assets = options[:customer_assets] || ENV['APPIAN_CUSTOMER_ASSETS'] || 'lyndsey'
  customize_build(options)
  
  # get the keychain from Vault so that the app can
  # be signed as the customer.
  #
  # remember, we introduced get_keychain_from_vault in the article
  # Automate Securing Code Signing Assets
  # https://bit.ly/32Tiv0U
  #
  keychain_data = get_keychain_from_vault(
    vault_addr: ENV[“VAULT_ADDR”],
    keychain_name: customer_assets
  )
  unlock_keychain(
    path: keychain_data[:keychain_path],
    password: keychain_data[:keychain_password],
    set_default: true
  )

  # turn off automatic code signing and set up the provisioning
  # and team id for the application to match the customer's
  disable_automatic_code_signing(path: './iOSExample/iOSExample.xcodeproj')
  code_signing_identity = code_signing_identity_from_keychain(keychain_data[:keychain_path])
  update_project_provisioning(
    xcodeproj: './iOSExample/iOSExample.xcodeproj',
    profile: "./iOSExample/Yillyyally.mobileprovision",
    build_configuration: "Release",
    code_signing_identity: code_signing_identity
  )
  update_project_team(
    path: './iOSExample/iOSExample.xcodeproj',
    teamid: '57738V598V'
  )

  # now build the application with the customer's customizations
  # provisioning profiles, and code signing identity
  build_app(
      scheme: 'iOSExample',
      project: './iOSExample/iOSExample.xcodeproj',
      output_directory: 'test_output',
      output_name: 'example.ipa',
      export_options: {
        method: "app-store",
        provisioningProfiles: {
          "com.yilly.yally" => "359e767c-5f71-4b2e-aedd-17645f951e02"
        }
      },
      export_team_id: '57738V598V',
      xcargs: "CODE_SIGN_IDENTITY=\"#{code_signing_identity}\""
  )
end

def code_signing_identity_from_keychain(keychain_filepath)
  identity_output = Fastlane::Actions.sh('security', 'find-identity', '-v', '-p', 'codesigning', keychain_filepath)
  UI.user_error!('Keychain does not contain a single valid signing identity') unless identity_output.match(/1 valid identities found/)
  identity_output.lines.first.chomp.sub(/.*"([^"]+)".*/, '\1')
end


Now, 20 minutes may not seem like a lot, but we have a lot of customers, each with a custom version of their application. For every customer, we have to manually start a Jenkins job with Vault token for each of these requests. When you think about it, we really shouldn’t have to build the application again — it was already built when we released the mobile application to Apple’s App Store and Google’s Play Store. The only differences were some settings files, some images, and the fact that it needed to be code signed using the customer’s code signing assets.

The idea.

We initially tried finding the latest released application from the iOS App Store and the Android Google Play Store, downloading the application, changing the settings files to match what the customer wants, swapping out the images in the build, and finally re-signing the application.

That seems pretty simple, right?

The implementation.

First, we had to write the code to download the already built and published application. We quickly found out that there was no programmatic way to download the latest released version of the iOS application. That threw a wrench into our plan.

Instead, for each release of our iOS and Android application, we decided to archive the application along with the image catalog and download that build each time the customer wants a custom version of the application. In the example below, I simulate that by putting the built applications in the GitHub Releases page.

Downloading the latest release.

require 'open-uri'
require 'json'
require 'tempfile'

platform :ios do
  desc 'download the latest released app for iOS'
  lane :download_latest_release do |options|
    download_latest_release(options)
  end
end

def download_latest_release(options)
  # use GitHub's REST endpoint to download the latest release
  result = github_api(
    http_method: 'GET',
    path: '/repos/lyndsey-ferguson/CustomizeExistingAppExample/releases/latest',
    api_token: File.read(File.absolute_path('../.github_token'))
  )
  body = JSON.parse(result[:body])

  # Set up the regex and name for Android or iOS
  asset_name_regex = //
  application_bundle_name = ''
  if ENV["FASTLANE_PLATFORM_NAME"] == "ios"
    asset_name_regex = %r{example_release_.+\.zip}
    application_bundle_name = 'package.zip'
  else
    asset_name_regex = %r{app-release-unsigned.apk}
    application_bundle_name = 'android.apk'
  end
  # Find the name of the Android or iOS released app
  platform_specific_assets = body['assets'].find do |asset|
    asset['name'] =~ asset_name_regex
  end
  browser_download_url = platform_specific_assets["browser_download_url"]

  # download the release to a temporary file
  zipfile = Tempfile.new(['latest_release', '.zip'])
  zipfile.binmode
  open(browser_download_url) do |download|
    zipfile.write(download.read)
  end
  zipfile.close
  latest_release_dir = File.absolute_path('../latest_release')
  FileUtils.mkdir_p(latest_release_dir)
  zip_package_path = File.join(latest_release_dir, application_bundle_name)

  # copy the downloaded release to a local directory
  FileUtils.rm_f(zip_package_path)
  FileUtils.cp(zipfile.path, zip_package_path)
  zip_package_path
end

The fastlane lane download_latest_release.

Once the application is downloaded, it needs to be unarchived, customized, resigned, and archived. Fortunately, both Android and iOS applications are zip archives, so it’s simple to unzip them to access the application contents.

In the case of the iOS application, we are archiving both the iOS “ipa” and the image assets in a parent zip folder. We need to include the same image assets that were associated with a particular release to ensure that we are not missing any images.

Unarchiving

We need to download the latest release and then unarchive the ipa and image assets bundle into a temporary directory. Then, we unarchive the ipa into its own temporary directory so that we can work on it.

 

require 'yaml'
require_relative 'binary_plist'
require_relative 'ios_customize_build'

platform :ios do
  desc 'download, customize, and sign the app according to customer\'s needs'
  lane :customize_built_app do |options|
    customize_built_app(options)
  end
end

def customize_built_app(options)
  customer_assets = options[:customer_assets] || ENV['APPIAN_CUSTOMER_ASSETS'] || 'puppy'
  
  zip_package_path = download_latest_release
  
  custom_built_app_path = File.expand_path(File.join('~/Desktop', "#{customer_assets}.ipa"))

  # unzip the combined ipa & images package into a tmp directory
  Dir.mktmpdir("latest_release_pkg") do |latest_release_pkg_path|
    Dir.chdir(latest_release_pkg_path) do
      sh("unzip -o -q #{zip_package_path}")

      example_ipa_filepath = File.absolute_path('example.ipa')
      app_iconset_dirpath = File.absolute_path('AppIcon.appiconset')

      # now that we have the package unzipped into a temporary folder
      # lets unzip the ipa so we can operate inside of it
      Dir.mktmpdir("customize_built_app") do |unzipped_ipa_path|
        Dir.chdir(unzipped_ipa_path) do
          sh("unzip -o -q #{example_ipa_filepath}")
          
          # TODO: customize the app and code sign it

          # zip up the customized app to desired file path
          sh("zip -qr #{custom_built_app_path} .")
        end
      end
    end
  end
  puts "Built mobile app: #{custom_built_app_path}"
end


Customizing settings.

Now that we have the application downloaded and unarchived, we can start customizing it. First, let’s customize the settings by adding the following code underneath the TODO section above.

 

# TODO: customize the app and code sign it
# Load the customer’s settings from a YAML settings file
customer_config_filepath = File.absolute_path("../#{customer_assets}/#{customer_assets}.yaml")
if File.exist?(customer_config_filepath)
  customer_config_file = YAML.load_file(customer_config_filepath)
  welcome_message = customer_config_file['WelcomeMessage'] || 'Hello World!'
  background_color = customer_config_file['BackgroundHexColor'] || '#FFFFFFFF'
end
welcome_message = options[:welcome_message] unless options[:welcome_message].nil?

background_color = options[:background_color] unless options[:background_color].nil?

example_ipa_payload_dir = File.join(unzipped_ipa_path, "Payload")
app_bundle_path = File.join(example_ipa_payload_dir, 'iOSExample.app')
configurations_plist_filepath = File.join(app_bundle_path, 'configurations.plist')

# Apple stores the xml plist files in a binary format in order to
# save space. We need to operate on a plain text file, so we use
# the plutil tool to convert it to XML.
sh("plutil -convert xml1 #{configurations_plist_filepath}")
customize_build(
  welcome_message: welcome_message,
  background_color: background_color,
  config_filepath: configurations_plist_filepath
)
# Convert the text plist XML file back to a binary XML file
sh("plutil -convert binary1 #{configurations_plist_filepath}")

This code loads a YAML file that has been configured with the customer’s settings. It then calls the method we declared earlier to customize the background color and the welcome message. It’s worth noting that plist files inside of an iOS application are stored as binaries to save space, so they have to be converted to text files, updated, and then converted back to binaries.

Customizing images.

Next, we need to write the code that customizes the icons using what the customer gave us. Let’s append the following to the code snippet above:

         # ...
          # Convert the text plist XML file back to a binary XML file
          sh("plutil -convert binary1 #{configurations_plist_filepath}")
          copy_customer_images(customer_appiconset_dirpath, latest_release_pkg_path)
          compile_images(latest_release_pkg_path, example_ipa_payload_dir)

          # zip up the customized app to desired file path
          sh("zip -qr #{custom_built_app_path} .")
        end
      end
    end
  end
end # of the customize_built_app method

APP_ICON_ASSET_DIR = 'AppIcon.appiconset'

def app_icon_asset_path(latest_release_pkg_path)
  File.join(latest_release_pkg_path, APP_ICON_ASSET_DIR)
end

def copy_customer_images(customer_appiconset_dirpath, latest_release_pkg_path)
  remove_app_icon_assets(latest_release_pkg_path)
  FileUtils.cp_r("#{customer_appiconset_dirpath}/.", app_icon_asset_path(latest_release_pkg_path))
end

# remove pre-existing application icons just in case the
# customer did not provide an icon for one of the many
# resolutions that iOS applications support. We do not want
# the standard icon to appear for those resolutions
def remove_app_icon_assets(latest_release_pkg_path)
   FileUtils.rm_r(Dir.glob("#{app_icon_asset_path(latest_release_pkg_path)}/*.png"))
end

# the `xcrun actool` requires the minimum deployment target
# as one of its command line options. Rather than hard-coding
# it, we get that value from the application Info.plist.
def minimum_deployment_target
  plist_filepath = 'Payload/iOSExample.app/Info.plist'
  temporary_plist_file = Tempfile.new
  FileUtils.cp(plist_filepath, temporary_plist_file.path)
  info_plist = Plist.parse_binary_xml(temporary_plist_file.path)
  info_plist.fetch('MinimumOSVersion', '10.0')
end

def compile_images(latest_release_pkg_path, example_ipa_payload_dir)
  FileUtils.mkdir_p("#{latest_release_pkg_path}/build")
  command = "xcrun actool #{latest_release_pkg_path} "
  command += "--compile #{example_ipa_payload_dir}/iOSExample.app "
  command += "--minimum-deployment-target #{minimum_deployment_target} "
  command += '--app-icon AppIcon --platform iphoneos '
  command += "--output-partial-info-plist #{latest_release_pkg_path}/build/partial.plist "
  sh(command)
end


Note how we have to first put the customer’s images into the iconset bundle and then use the xcrun actool to compile that iconset into an Assets.car file to put into the application bundle.

Code-signing.

Now that we have the customized application, we need to code-sign it with the customer’s certificate. Continuing in the customize_built_app method where we left off with compile_images:

 

          compile_images(latest_release_pkg_path, example_ipa_payload_dir)

          keychain_data = get_keychain_from_vault(
            vault_addr: 'http://127.0.0.1:8200',
            keychain_name: customer_assets
          )
          keychain_password = keychain_data[:keychain_password]
          keychain_path = keychain_data[:keychain_path]

          unlock_keychain(path: keychain_path, password: keychain_password, set_default: true)
          cert = certificate_id_from_keychain(keychain_path)
          sign_frameworks(cert, keychain_path)
          prepare_app_bundle(app_bundle_path, customer_profile_pathname)
          prepare_entitlements(cert, app_bundle_path, keychain_path, customer_profile_pathname)

          # zip up the customized app to desired file path
          sh("zip -qr #{custom_built_app_path} .")
        end
      end
    end
  end
end # of the customize_built_app method
# find the certificate in the keychain so that we can get the
# id and use it for code signing.
def certificate_id_from_keychain(keychain_filepath)
  identity_output = Fastlane::Actions.sh('security', 'find-identity', '-v', '-p', 'codesigning', keychain_filepath)
  UI.user_error!('Keychain does not contain a single valid signing identity') unless identity_output.match(/1 valid identities found/)
  identity_output.lines.first.split[1].downcase
end

# sign any frameworks inside of the application
def sign_frameworks(cert, keychain_filepath)
  frameworks = Dir.glob('Payload/iOSExample.app/Frameworks/*.dylib').map { |s| "'#{s}'" }.join(' ')
  unless frameworks.empty?
    sh("/usr/bin/codesign -f --keychain \"#{keychain_filepath.shellescape}\"  -s \"#{cert}\" #{frameworks}")
  end
end

# remove any existing signature and put the provisioning profile
# inside of the app bundle
def prepare_app_bundle(app_bundle_path, profile_pathname)
  # Remove existing signature file
  FileUtils.rm_rf("#{app_bundle_path}/_CodeSignature")

  # Copy associated provisioning profile to app
  FileUtils.cp(profile_pathname, "#{app_bundle_path}/embedded.mobileprovision")
end

# create a plist from the provisioning profile so that we
# can get the team id and the application bundle id
def plist_for_profile(profile)
  output_plist_file = Tempfile.new
  `security cms -D -i \"#{profile}\" > \"#{output_plist_file.path}\"` if File.exists?(profile)
  Plist.parse_xml(output_plist_file) || {}
end

# customize the entitlements file with values from the customer's
# mobileprovision file.
def customize_entitlements_xml(xml_path, customer_app_id, customer_team_id)
  xml_hash = Plist.parse_xml(xml_path)

  xml_hash['application-identifier'] = customer_app_id
  xml_hash['com.apple.developer.team-identifier'] = customer_team_id if xml_hash.key?('com.apple.developer.team-identifier')

  xml_hash.save_plist(xml_path)
end

# pull out the application's existing entitlements and customize the values that
# are associated with the customer, and then re-embed them into the application binary
def prepare_entitlements(cert, app_bundle_path, keychain_filepath, mobileprovisioning_filepath)
  # File exists in builds that were created with xcode versions < 10.
  # Entitlements are now embedded in the app binary which are read and manipulated before signing them back into the binary.
  FileUtils.rm_f("#{app_bundle_path}/#{File.basename(app_bundle_path, '.*')}.entitlements")
  entitlements = Tempfile.new()
  sh("/usr/bin/codesign -d --entitlements :\"#{entitlements.path}\" \"#{app_bundle_path}\"")
  mobileprovisioning_data = plist_for_profile(mobileprovisioning_filepath)
  team_id = mobileprovisioning_data.dig('Entitlements', 'com.apple.developer.team-identifier')
  app_id = mobileprovisioning_data.dig('Entitlements', 'application-identifier').sub("#{team_id}.", '')
  customize_entitlements_xml(entitlements.path, app_id, team_id)
  sh("/usr/bin/codesign -f --keychain \"#{keychain_filepath.shellescape}\" -s \"#{cert}\" --entitlements \"#{entitlements.path}\" \"#{app_bundle_path}\"")
end

This code uses the Vault token role custom-mobile-apps-read-policy (as seen in “Automatically Secure Code Signing Assets”) to retrieve the keychain and password in preparation for code signing. After unlocking the keychain, it proceeds to get the keychain’s certificate ID to sign any frameworks inside the app bundle as well as the app bundle itself.

Take note that before code signing the application itself, we first have to retrieve the entitlements that were embedded into the application binary, update them to contain the bundle ID and application ID that were in the customer’s mobileprovision file, and re-embed those entitlements back into the application binary. This happens in the prepare_entitlements method and its various helper methods.

Validations

Finally, we also added code to ensure that what we built was correct (not shown here). This code

  • Removes transparencies from any launch icons, as apps that have them will be rejected from App Stores.
  • Fails and alerts us if images were not of the correct size.
  • Fails and alerts us when certificates or provisioning profiles were expired or missing.
  • Fails and alerts us if the customizations were invalid.

The result.

With these changes, we no longer have to build the application each time. Instead, we can apply some basic customizations in a few minutes rather than waiting 20 minutes for the build.

I’ve written up the example code for both an Android and an iOS application, and you can find it at this GitHub repo. If you have other ideas of how to make this process faster, or if this helps you with your own automation, I’d love to hear from you!

Afterword

This leads me to think: what other ways can we all benefit from this knowledge? There may be other instances where we need to make a change to an application, but the code or project doesn’t actually have to change. For example, a React Native application that loads a bundled JavaScript file could take advantage of this for code changes in only JS files.

If you find this kind of work interesting and want to work with creative, passionate, and smart people, there are plenty of other projects to work on like this at Appian. Apply today!






 



Lyndseyferguson

Written by

Lyndsey Ferguson

Lyndsey is a Principal Software Engineer at Appian.