AutoPkg recipes automate and codify the often tedious tasks involved in packaging and distributing Mac software. Central to AutoPkg’s greatness are the many built-in security measures that verify you’re getting the software you intend — including code signature verification, embedded trust information in overrides, and the autopkg audit command.
AutoPkg recipe authors should also follow another important security practice: use HTTPS URLs instead of HTTP whenever possible. Whether downloading actual software or downloading metadata about the software, using an HTTPS URL helps prevent person-in-the-middle attacks and keep your organization’s software pipeline secure.
In particular, the arguments and input variables used by the URLDownloader, URLTextSearcher, and SparkleUpdateInfoProvider processors should use HTTPS if the option is available, and recipe authors should perform periodic checks to detect when software developers (or their CDNs) begin offering HTTPS downloads.
The security benefits aren’t just theoretical; a few years ago, security researchers demonstrated an attack targeting Mac apps using insecure Sparkle feeds. Ben Toms wrote a good article detailing the Mac admin community’s response to the vulnerability.
HTTPS Spotter
Checking for the existence of HTTPS URLs can be tedious if you manage more than a handful of AutoPkg recipes, so I’ve written a Python tool called HTTPS Spotter that will automate the process for you. The source code is on GitHub and embedded below.
Requirements
To use the script, you’ll need Git and AutoPkg installed.
Steps
-
Clone the script to your Mac (substitute the path to your source, if not ~/Developer).
git clone https://gist.github.com/66d1c8772baf5f731bb8ddf263f33401.git ~/Developer/https_spotter -
Run the script with
--helpto see usage information./usr/local/autopkg/python ~/Developer/https_spotter/https_spotter.py --help -
Now run the script again, pointing it to your repository of AutoPkg recipes:
/usr/local/autopkg/python ~/Developer/https_spotter/https_spotter.py ~/Developer/your-autopkg-recipesYou’ll see output that might look like this:
../homebysix-recipes/NeoFinder/NeoFinder.download.recipe Replace: http://www.cdfinder.de/en/downloads.html With: https://www.cdfinder.de/en/downloads.html ../homebysix-recipes/FontFinagler/FontFinagler.download.recipe Replace: http://www.markdouma.com/fontfinagler/version.xml With: https://www.markdouma.com/fontfinagler/version.xml 2 suggested changes. To apply, run again with --auto. -
Run the script again with the
--autoflag in order to automatically apply the changes, or apply the changes manually in your preferred text editor. -
Test the modified recipes prior to committing/pushing the changes to your public repo on GitHub.
tip
Here’s a one-liner that will run recently-modified recipes in “check only” mode:
find * -iname "*.recipe" -mtime -1 -exec autopkg run -vvcq "{}" '+'
Source code
The script is below. Suggestions or improvements are welcome!
| #!/usr/local/autopkg/python | |
| # encoding: utf-8 | |
| # HTTPS Spotter | |
| # Copyright 2016-2024 Elliot Jordan | |
| # | |
| # Licensed under the Apache License, Version 2.0 (the "License"); | |
| # you may not use this file except in compliance with the License. | |
| # You may obtain a copy of the License at | |
| # | |
| # http://www.apache.org/licenses/LICENSE-2.0 | |
| # | |
| # Unless required by applicable law or agreed to in writing, software | |
| # distributed under the License is distributed on an "AS IS" BASIS, | |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| # See the License for the specific language governing permissions and | |
| # limitations under the License. | |
| """https_spotter.py. | |
| Scans for HTTP URLs in a specified repository of AutoPkg recipes and suggests | |
| HTTPS alternatives wherever possible. | |
| """ | |
| import argparse | |
| import os | |
| import plistlib | |
| import subprocess | |
| import yaml | |
| # Path to curl. | |
| CURL_PATH = "/usr/bin/curl" | |
| def build_argument_parser(): | |
| """Build and return the argument parser.""" | |
| parser = argparse.ArgumentParser( | |
| description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter | |
| ) | |
| parser.add_argument( | |
| "repo_path", nargs="?", help="Path to search for AutoPkg recipes." | |
| ) | |
| parser.add_argument( | |
| "--auto", | |
| action="store_true", | |
| default=False, | |
| help="Automatically apply suggested changes to recipes. Only recommended " | |
| "for repos you manage or are submitting a pull request to. (Applying " | |
| "changes to repos added using `autopkg repo-add` may result in failure " | |
| "to update the repo in the future.)", | |
| ) | |
| parser.add_argument( | |
| "-v", | |
| "--verbose", | |
| action="count", | |
| default=0, | |
| help="Print additional output useful for " | |
| "troubleshooting. (Can be specified multiple times.)", | |
| ) | |
| return parser | |
| def check_url(url, verbose): | |
| """Checks a given HTTPS URL to see whether an HTTPS equivalent is | |
| available.""" | |
| https_url = url.replace("http:", "https:") | |
| try: | |
| cmd = [ | |
| CURL_PATH, | |
| "--head", | |
| "--silent", | |
| "--location", | |
| "--max-time", | |
| "5", | |
| "--max-redirs", | |
| "10", | |
| "--url", | |
| https_url, | |
| ] | |
| if verbose > 1: | |
| print(" Curl command: %s" % " ".join(cmd)) | |
| proc = subprocess.run(cmd, check=False, capture_output=True) | |
| http_response = [x for x in proc.stdout.splitlines() if x.startswith(b"HTTP/")] | |
| http_status = int(http_response[0].split()[1]) | |
| if verbose > 2: | |
| print(" Exit code: %s" % proc.returncode) | |
| print(" HTTP status: %s" % http_status) | |
| if proc.returncode == 0: | |
| # Ignoring HTTP status because 404 with SSL is better than 404 without. | |
| return https_url | |
| except Exception as err: | |
| if verbose > 3: | |
| print(" %s" % err) | |
| return False | |
| def process_recipe(recipe_path, args): | |
| """Processes one recipe.""" | |
| try: | |
| # Read recipe. | |
| if not os.path.isfile(recipe_path): | |
| return | |
| if recipe_path.endswith(".recipe.yaml"): | |
| with open(recipe_path, "r") as openfile: | |
| recipe = yaml.safe_load(openfile) | |
| else: | |
| with open(recipe_path, "rb") as openfile: | |
| recipe = plistlib.load(openfile) | |
| if "Process" in recipe: | |
| processors = [x.get("Processor") for x in recipe["Process"]] | |
| if "DeprecationWarning" in processors: | |
| if args.verbose > 0: | |
| print("Skipping: %s (contains DeprecationWarning)" % (recipe_path)) | |
| return | |
| if args.verbose > 0: | |
| print("Processing: %s" % (recipe_path)) | |
| urls_to_check = [] | |
| # Gather HTTP URLs from Input. | |
| for key in recipe.get("Input", {}): | |
| if isinstance(recipe["Input"][key], str) and recipe["Input"][ | |
| key | |
| ].startswith("http:"): | |
| urls_to_check.append(recipe["Input"][key]) | |
| # Gather HTTP URLs from Processors. | |
| for proc in recipe.get("Process", []): | |
| if not proc.get("Arguments"): | |
| continue | |
| for _, value in proc.get("Arguments", {}).items(): | |
| if not isinstance(value, str): | |
| # Only looking at string arguments, not diving | |
| # deeper into lists/dicts. | |
| continue | |
| if value.startswith("http:"): | |
| urls_to_check.append(value) | |
| # Check HTTP URLs for HTTPS equivalents. | |
| for http_url in urls_to_check: | |
| https_url = check_url(http_url, args.verbose) | |
| if not https_url: | |
| continue | |
| if args.auto: | |
| # Read/write as text instead of using plistlib in order | |
| # to prevent unrelated changes (e.g. whitespace) in diff. | |
| with open(recipe_path, "r") as openfile: | |
| recipe_contents = openfile.read() | |
| with open(recipe_path, "w") as openfile: | |
| openfile.write(recipe_contents.replace(http_url, https_url)) | |
| print(recipe_path) | |
| print(" Replaced: %s" % (http_url)) | |
| print(" With: %s" % (https_url)) | |
| else: | |
| print(recipe_path) | |
| print(" Replace: %s" % (http_url)) | |
| print(" With: %s" % (https_url)) | |
| return True | |
| except Exception as err: | |
| print("[ERROR] %s (%s)" % (recipe_path, err)) | |
| raise err | |
| def main(): | |
| """Scans all recipes in the specified folder for HTTP URLs.""" | |
| # Parse command line arguments. | |
| argparser = build_argument_parser() | |
| args = argparser.parse_args() | |
| if args.repo_path: | |
| path = args.repo_path | |
| else: | |
| path = "~/Library/AutoPkg/Recipes" | |
| # Process all recipes looking for HTTP URLs. | |
| suggestion_count = 0 | |
| for dirpath, dirnames, filenames in os.walk(os.path.expanduser(path)): | |
| for dirname in dirnames: | |
| dirnames = [x for x in dirnames if not x.startswith(".")] | |
| filenames = [ | |
| x for x in filenames if x.endswith(".recipe") or x.endswith(".recipe.yaml") | |
| ] | |
| for filename in filenames: | |
| if process_recipe(os.path.join(dirpath, filename), args): | |
| suggestion_count += 1 | |
| if not args.auto and suggestion_count > 0: | |
| changes = "change" if suggestion_count == 1 else "changes" | |
| print( | |
| "\n%d suggested %s. To apply, run again with --auto." | |
| % (suggestion_count, changes) | |
| ) | |
| if __name__ == "__main__": | |
| main() |
