From fcf7babe420eac89dcce461cb6d34afd547657d6 Mon Sep 17 00:00:00 2001 From: stephenreading Date: Mon, 13 Feb 2023 04:13:39 -0800 Subject: [PATCH 1/8] Create MakeMSIX-spec.md API spec document for msix packaging in winappsdk --- specs/WinRT/MakeMSIX-spec.md | 640 +++++++++++++++++++++++++++++++++++ 1 file changed, 640 insertions(+) create mode 100644 specs/WinRT/MakeMSIX-spec.md diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md new file mode 100644 index 0000000000..d6382b9ac8 --- /dev/null +++ b/specs/WinRT/MakeMSIX-spec.md @@ -0,0 +1,640 @@ +# Make MSIX API in Windows App SDK + +# Background + +The Make MSIX api allows developers to create app packages for distribution. The api +provides similar functionality as multiple existing command line tools. + +For more details see: + +- [The makeappx command line tool](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) + Shipped as part of the Windows Development Kit. + Exposes pack, unpack, bundle, and unbundle commands, as well as encryption support + via a custom key-file list file, and mapping . + +- [The makemsix command line tool and dll](https://github.com/Microsoft/msix-packaging) + Available through the github project. + Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command is not implemented). + +- [The msixmgr command line tool](https://github.com/Microsoft/msix-packaging and + https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr) + Available through github and as a direct download from the learn.microsoft.com site. + Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. + +## The problems today + +None of the existing tools provide a good programmatic interface for all aspects of packaging. + +The makeappx.exe built into windows provides the only complete implementation of packaging +and bundling but requires CreateProcess and does not return any specific errors from process exit. +It also does not support any of the azure app attach scenarios. + +The makemsix tool is designed as a cross-platform implementation of packaging, but does not support +the bundle command. The functionality is exposed as a command line as well as flat dll exports. + +The msixmgr tool is mostly for specific azure app attach packaging scenarios that are not covered by +the other tools. To use it programmatically would require either using CreateProcess or building +the msix-packaging github project and linking the static lib it produces into a project. + +# Existing interfaces and options + +## Makeappx + +### Pack +Usage: +------ + MakeAppx pack [options] /d /p + MakeAppx pack [options] /f /p + MakeAppx pack [options] /m /f /p + MakeAppx pack [options] /r /m /f /p + MakeAppx pack [options] /d /ep /kf + MakeAppx pack [options] /d /ep /kt + +Options: +-------- + /h, /hashAlgorithm: Specifies a hash algorithm to use for creating the + block map. Valid algorithm IDs are SHA256, SHA384, and SHA512. The + default is SHA256. + /nv, /noValidation: Skips validation that ensures the package will be + installable on Windows. The validation include: existence of files + referenced in manifest, ContentGroupMap correctness, and additional + manifest validation on Protocols and FileTypeAssociation. By default, + all semantic validation is performed. + /nfv, /noFileValidation: Skips validation that ensures that the files used + to create the package (from a folder through /d or layout file through + /l) exist and are accessible. By default, all files must exist and not + be read locked. If a file is inaccessible, it will not be added to the + package. + /nc, /noCompress: Prevents MakeAppx from compressing files in the package. + By default, files in the package are compressed based on detected file + type. + /m: Specifies the path to an input app manifest which will be used as the + basis for generating the output app package or resource package's + manifest. When you use this option, you must also use /f and include a + [ResourceMetadata] section in the mapping file to specify the resource + dimensions to be included in the generated manifest. + /r: Builds a resource package. You must use the /m option with this. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /cgm: The input content group map (CGM) file path used to create a + stream-able package. Providing a content group map through this option + will disable the check that all files in the content group map exist in + the package. + /pri, /makepriExeFullPath: You can use /pri to override the default + MakePri.exe path with the custom fullpath from which makeappx.exe will + launch the tool from when needed. + /mp: Specifies the path to a main package in the same package family as the + resource package being built. You should provide this when encrypting + a resource package or when the resource package contains a content + group map. + /pb, /publisherBridging: Use this option to add publisher bridging entries + to the package or bundle. Publisher bridging is useful when the new + issued cert subject name changed but the old publisher name is still + desired to maintain package identity continuity. + /np, /noParallel: Use this option to disable parallel execution of this + command. + /ml, /memoryLimit: Specify maximum memory in bytes that MakeAppx should + consume while executing in parallel. By default, this value is set to + half of total physical memory. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. + +### Bundle + +Usage: +------ + MakeAppx bundle [options] /d /p + MakeAppx bundle [options] /f /p + MakeAppx bundle [options] /d /ep /kf MyKeyFile.txt + MakeAppx bundle [options] /f /ep /kt + +Options: +-------- + /bv: Specifies the version number of the bundle being created. The version + must be in dotted-quad notation of four integers + ... ranging from 0 to 65535 each. If the + /bv option is not specified or is set to 0.0.0.0, the bundle is created + using the current date-time formatted as the version: + .... + /mo: Generates a bundle manifest only, instead of a full bundle. Input + files must all be package manifests in XML format if this option is + specified. + /fb: Generates a fully sparse bundle where all packages are references to + packages that exist outside of the bundle file + /pri, /makepriExeFullPath: You can use /pri to override the default + MakePri.exe path with the custom fullpath from which makeappx.exe will + launch the tool from when needed. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. + +### Unpack +Usage: +------ + MakeAppx unpack [options] /p /d + MakeAppx unpack [options] /ep /d /kf + MakeAppx unpack [options] /ep /d /kt + +Options: +-------- + /pfn: Unpacks all files to a subdirectory under the specified output path, + named after the package full name. + /nv, /noValidation: Skips validation that ensures the package will be + installable on Windows. The validation include: existence of files + referenced in manifest, ContentGroupMap correctness, and additional + manifest validation on Protocols and FileTypeAssociation. By default, + all semantic validation is performed. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /nd: Skips decryption when unpacking an encrypted package or bundle. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. + +### Unbundle + +Usage: +------ + MakeAppx unbundle [options] /p /d + MakeAppx unbundle [options] /ep /d /kf + MakeAppx unbundle [options] /ep /d /kt + +Options: +-------- + /pfn: Unpacks all files to a subdirectory under the specified output path, + named after the package full name. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /nd: Skips decryption when unpacking an encrypted package or bundle. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. + +## MakeMsix + +MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for +encryption, content groups, mapping files, and bundle support is limited to +only allowing creation of flat-bundles (bundles where the appxbundle file itself +only references but does not contain the main and resource appxpackages) + +It does add the ability to optionally unpack all packages inside the bundle +in one command using the -pfn-flat option, as in: +makemsix.exe unbundle -p -d -pfn-flat + +There is, however, no good option for reversing this command when recreating the +packages and bundle. Callers need to call makemsix pack on each individual +package folder and then follow it with a makemsix bundle call to generate a +flat bundle. + +### Pack +Usage: +--------------- + makemsix.exe pack -d -p [options] + +Options: +--------------- + -d {Required} + Input directory path. + -p {Required} + Output package file path. + -? [Flag] + Displays this help text. + +### Bundle +Usage: +--------------- + makemsix.exe bundle -p [options] + +Options: +--------------- + -d + Input directory path. + -p {Required} + Output bundle file path. + -f + Mapping file path. + -bv + Specifies the version number of the bundle being created. The version must be in dotted - quad notation of four integers ... ranging from 0 to 65535 each. If the -bv option is not specified or is set to 0.0.0.0, the bundle is created using the current date - time formatted as the version: .... + -mo [Flag] + Generates a bundle manifest only, instead of a full bundle. Input files must all be package manifests in XML format if this option is specified. + -fb [Flag] + Generates a fully sparse bundle where all packages are references to packages that exist outside of the bundle file. + -o [Flag] + Forces the output to overwrite any existing files with the same name.By default, the user is asked whether to overwrite existing files with the same name.You can't use this option with -no. + -no [Flag] + Prevents the output from overwriting any existing files with the same name. By default, the user is asked whether to overwrite existing files with the same name.You can't use this option with -o. + -v [Flag] + Enables verbose output of messages to the console. + -? [Flag] + Displays this help text. + +### Unpack +Usage: +--------------- + makemsix.exe unpack -p -d [options] + +Options: +--------------- + -p {Required} + Input package file path. + -d {Required} + Output directory path. + -pfn [Flag] + Unpacks all files to a subdirectory under the output path, named after the package full name. + -ac [Flag] + Allows any certificate. By default the signature origin must be known. + -ss [Flag] + Skips enforcement of signed packages. By default packages must be signed. + -pfn-flat [Flag] + Same behavior as -pfn for packages. + -? [Flag] + Displays this help text. + +### Unbundle + +Usage: +--------------- + makemsix.exe unbundle -p -d [options] + +Options: +--------------- + -p {Required} + Input bundle file path. + -d {Required} + Output directory path. + -pfn [Flag] + Unpacks all files to a subdirectory under the output path, named after the package full name. + -ac [Flag] + Allows any certificate. By default the signature origin must be known. + -ss [Flag] + Skips enforcement of signed packages. By default packages must be signed. + -sl [Flag] + Skips matching packages with the language of the system. By default unpacked resources packages will match the system languages. + -sp [Flag] + Skips matching packages with of the same system. By default unpacked application packages will only match the platform. + -extract-all [Flag] + Extracts all packages from the bundle. + -pfn-flat [Flag] + Unpacks bundle's files to a subdirectory under the specified output path, named after the package full name. Unpacks packages to subdirectories also under the specified output path, named after the package full name. By default unpacked packages will be nested inside the bundle folder. + -? [Flag] + Displays this help text. + +## MSIXMgr + +### Unpack +msixmgr.exe -Unpack -packagePath -destination [-applyacls] [-create] [-vhdSize ] [-filetype ] [-rootDirectory ] + +-Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. + -applyACLs: optional parameter that applies ACLs to the resulting package folder(s) and their parent folder + -create: optional parameter that creates a new image with the specified -filetype and unpacks the packages to that image + -destination: the directory to place the resulting package folder(s) in + -fileType: the type of file to unpack packages to. Valid file types include {VHD, VHDX, CIM}. This is a required parameter when unpacking to CIM files + -packagePath: the path to the package to unpack OR the path to a directory containing multiple packages to unpack + -rootDirectory: root directory on an image to unpack packages to. Required parameter for unpacking to new and existing CIM files + -validateSignature: optional parameter that validates a package's signature file before unpacking the package. This will require that the package's certificate is installed on the machine. + -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. Use only for VHD or VHDX files + +# Summary of existing options + +Existing options all treat pack and bundle as separate commands. Though they are similar, each command does have +some options that are irrelevant to the other command. For bundling that means specifying the Bundle Version, which +nearly all callers will specify (the alternative is the tool setting the version based on a timestamp), and allowing +creation of sparse bundles. For packaging that means hash algorithm, and creation of streamable packages and resource +packages. + +In the long term it's likely that these differences in the information and flags required to support creating a bundle +and creating a package will only continue to grow. + +Unpack and Unbundle is a somewhat different story. MSIXMgr defines an unpack command which in some ways looks more like a +"convert" than an "unpack" (i.e. "unpack" msix to folder, "pack" folder to vhdx). This means that the msixmgr unpack command +needs to take many more options to define the "unpack" destination than the makeappx unpack command which always unpacks +to a folder. With the set of options to describe the output it looks more like the "pack\bundle" commands of the other tools. +In an api that supported creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers might +find it strange to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just +creating the vhdx directly. However, signing is not integrated into any of these tools, and is a separate step that +is done after packaging is complete. This may prevent direct creation of VHDX files from the layout directory if +signing tools don't support this scenario (TODO: sreading, looking into this) + +If defined as an atomic operation where a package is unpacked to a folder, it is less likely that differences +will arise between unbundle and unpack. However, as seen with the existing makemsix tool, as long as pack and bundle +are separate operations it may not be especially useful for unbundle and unpack to be joined. Unpacking of +all bundled packages might save time if someone wanted to examine the manifest of a main package inside the bundle, +as might be an expected use case for a command line tool. +But in cases where the api is actually being used programattically, to do something like extract all packages and update +every manifest, say with a different version number, there is no avoiding use of filesystem apis to find each +unpacked package. In such a case merging the unbundle and unpack command into one api call saves very little complexity for +the caller, especially if pack and bundle have not also been merged to allow callers to use one call to package all subfolders +with appxmanifests and then bundle all of them together. + +For these reasons pack, bundle, unpack, unbundle have been left as separate commands. The MSIXMgr "unpack" command +is implemented here as another form of pack. + + +# Examples + +## Packing a package + +```cpp + TEST_METHOD(TestPack) + { + winrt::init_apartment(); + auto packOptions = PackOptions(); + packOptions.OverwriteFiles(true); + //PackOptions.KozaniPackageFilePath(L"E:\\vm\\apps\\kozaniPackage.msix"); + packOptions.PackageFilePath(L"E:\\test\\packagedOutput.msix"); + + PackagingResult result{ MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", packOptions).get()}; + VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + + auto kozaniPackOptions = PackOptions(); + kozaniPackOptions.OverwriteFiles(true); + kozaniPackOptions.CreateAsKozaniPackage(true); + kozaniPackOptions.PackageFilePath(L"E:\\test\\kozaniPackagedOutput.msix"); + + PackagingResult kozaniResult{ MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", kozaniPackOptions).get() }; + VERIFY_IS_TRUE(SUCCEEDED(kozaniResult.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(kozaniResult.ExtendedErrorCode())); + } +``` + +## Bundling a package + +```cpp + TEST_METHOD(TestBundle) + { + winrt::init_apartment(); + auto bundleOptions = BundleOptions(); + bundleOptions.OverwriteFiles(true); + bundleOptions.BundleFilePath(L"E:\\test\\bundledOutput.msixbundle"); + + PackagingResult result{ MakeMSIXManager::Bundle(L"E:\\test\\unbundledPackageInput", bundleOptions).get() }; + VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + } +``` + +## Unpacking a package + +```cpp + TEST_METHOD(TestUnpack) + { + winrt::init_apartment(); + auto unpackOptions = UnpackOptions(); + unpackOptions.OverwriteFiles(true); + unpackOptions.UnpackedPackageRootDirectory(L"E:\\test\\unpackedPackageOutput"); + + PackagingResult result{ MakeMSIXManager::Unpack(L"E:\\test\\package.msix", unpackOptions).get() }; + VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + } +``` + +## Unbundling a package + +```cpp + TEST_METHOD(TestUnbundle) + { + winrt::init_apartment(); + std::wstring outputDir{ L"E:\\test\\unpackedBundleOutput" }; + auto unbundleOptions = UnbundleOptions(); + unbundleOptions.OverwriteFiles(true); + unbundleOptions.UnbundledPackageRootDirectory(outputDir); + + PackagingResult result{ MakeMSIXManager::Unbundle(L"E:\\test\\bundle.msixbundle", unbundleOptions).get() }; + VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + + // Iterate through packages and unpack each one. + for (const auto& file : std::filesystem::directory_iterator(outputDir)) + { + auto unpackOptions = UnpackOptions(); + unpackOptions.OverwriteFiles(true); + std::filesystem::path outputDirForPackage{ outputDir }; + outputDirForPackage /= file.path().filename(); + unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); + + PackagingResult unpackResult{ MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get()}; + VERIFY_IS_TRUE(SUCCEEDED(unpackResult.Status() == PackagingResultStatus::Ok)); + VERIFY_IS_TRUE(SUCCEEDED(unpackResult.ExtendedErrorCode())); + } + } +``` + +# API Details + +```c# +// Copyright (c) Microsoft Corporation and Contributors. +// Licensed under the MIT License. + +namespace Microsoft.Kozani.MakeMSIX +{ + [contractversion(1)] + apicontract MakeMSIXContract{}; + + /// + /// Result of the packaging operation + /// + [contract(MakeMSIXContract, 1)] + enum PackagingResultStatus + { + Ok, + InvalidOptions, + FileAlreadyExistsError, + InternalError, + }; + + /// + /// Result of the packaging operation + /// + [contract(MakeMSIXContract, 1)] + runtimeclass PackagingResult + { + /// + /// Status of the overall operation. + /// + PackagingResultStatus Status { get; }; + /// + /// Error code of the overall operation. + /// + HRESULT ExtendedErrorCode { get; }; + }; + + /// Options for creating a package + [contract(MakeMSIXContract, 1)] + runtimeclass PackOptions + { + /// + /// Created by callers to pass options into CreatePackage + /// + PackOptions(); + + /// + /// Output path for the full packaged file. + /// + String PackageFilePath{ get; set; }; + + // API REVIEW NOTE: Two options here. Needing to create a traditional package + // and a Kozani package at the same time may be fairly common for certain users + // but most users will not ever need to create Kozani packages. + // Either require the caller to make two separate calls, or allow creation of both + // packages in one call. My preference is separate calls, which makes it clearer + // that Kozani packages are optional. + + /// + /// If true, writes a Kozani package to PackageFilePath. + /// Defaults to false. + /// + Boolean CreateAsKozaniPackage{ get; set; }; + /// + /// Output path for the Kozani package file. + /// At least one of PackageFilePath and KozaniPackageFilePath must be set. + /// + String KozaniPackageFilePath{ get; set; }; + + + /// + /// Overwrite PackageFilePath and KozaniPackageFilePath if they already exist. + /// Defaults to true. + /// + Boolean OverwriteFiles{ get; set; }; + /// + /// Validates elements of the package during creation. + /// Defaults to false. + /// + Boolean ValidateFiles{ get; set; }; + }; + + /// Options for creating a bundle + [contract(MakeMSIXContract, 1)] + runtimeclass BundleOptions + { + /// + /// Created by callers to pass options into Bundle + /// + BundleOptions(); + + /// + /// Output path for the full bundled file. + /// + String BundleFilePath{ get; set; }; + + /// + /// Overwrite BundleFilePath if they already exist. + /// Defaults to true. + /// + Boolean OverwriteFiles{ get; set; }; + + /// + /// Sets the version in the AppxBundleManifest.xml during packaging if + /// the source layout is a bundle. + /// + Windows.ApplicationModel.PackageVersion BundleVersion{ get; set; }; + }; + + /// + /// Options for unpacking a package + /// + [contract(MakeMSIXContract, 1)] + runtimeclass UnpackOptions + { + /// + /// Created by callers to pass options into UnpackPackage + /// + UnpackOptions(); + + /// + /// The output folder to unpack the package into. + /// + String UnpackedPackageRootDirectory; + /// + /// If true, the operation will overwrite existing files in the UnpackedPackageRootDirectory + /// when unpacking. + /// If false, the operation will fail if a file already exists. + /// + Boolean OverwriteFiles; + }; + + /// + /// Options for unpacking a package + /// + [contract(MakeMSIXContract, 1)] + runtimeclass UnbundleOptions + { + /// + /// Created by callers to pass options into UnpackPackage + /// + UnbundleOptions(); + + /// + /// The output folder to unpack the bundle package into. + /// + String UnbundledPackageRootDirectory; + /// + /// If true, the operation will overwrite existing files in the UnbundledPackageRootDirectory + /// when unpacking. + /// If false, the operation will fail if a file already exists. + /// + Boolean OverwriteFiles; + }; + + /// + /// Static methods for creating and unpacking packages. + /// + [contract(MakeMSIXContract, 1)] + runtimeclass MakeMSIXManager + { + /// + /// Creates packages from the directoryPathToPack as specified by the packOptions. + /// + static Windows.Foundation.IAsyncOperation Pack(String directoryPathToPack, PackOptions packOptions); + /// + /// Creates bundles from the directoryPathToBundle as specified by the bundleOptions. + /// + static Windows.Foundation.IAsyncOperation Bundle(String directoryPathToBundle, BundleOptions bundleOptions); + /// + /// Unpacks a package at packageFilePathToUnpack as specified by the unpackOptions. + /// + static Windows.Foundation.IAsyncOperation Unpack(String packageFilePathToUnpack, UnpackOptions unpackOptions); + /// + /// Unbundles a package at bundleFilePathToUnbundle as specified by the unbundleOptions. + /// + static Windows.Foundation.IAsyncOperation Unbundle(String bundleFilePathToUnbundle, UnbundleOptions unbundleOptions); + }; +} + + +``` + +# Appendix From 35afc2aac1fea43c3d643e7c04d0a926c9801260 Mon Sep 17 00:00:00 2001 From: stephenreading Date: Wed, 1 Mar 2023 02:37:19 -0800 Subject: [PATCH 2/8] Update MakeMSIX-spec.md Add msixmgr appattach support to api spec. --- specs/WinRT/MakeMSIX-spec.md | 353 ++++++++++++++++++++++++----------- 1 file changed, 240 insertions(+), 113 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index d6382b9ac8..ef17b3cd28 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -336,19 +336,9 @@ packages. In the long term it's likely that these differences in the information and flags required to support creating a bundle and creating a package will only continue to grow. -Unpack and Unbundle is a somewhat different story. MSIXMgr defines an unpack command which in some ways looks more like a -"convert" than an "unpack" (i.e. "unpack" msix to folder, "pack" folder to vhdx). This means that the msixmgr unpack command -needs to take many more options to define the "unpack" destination than the makeappx unpack command which always unpacks -to a folder. With the set of options to describe the output it looks more like the "pack\bundle" commands of the other tools. -In an api that supported creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers might -find it strange to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just -creating the vhdx directly. However, signing is not integrated into any of these tools, and is a separate step that -is done after packaging is complete. This may prevent direct creation of VHDX files from the layout directory if -signing tools don't support this scenario (TODO: sreading, looking into this) - -If defined as an atomic operation where a package is unpacked to a folder, it is less likely that differences -will arise between unbundle and unpack. However, as seen with the existing makemsix tool, as long as pack and bundle -are separate operations it may not be especially useful for unbundle and unpack to be joined. Unpacking of +Unpack and Unbundle is a somewhat different story. If defined as an atomic operation where a package is unpacked to a folder, +it is less likely that differences will arise between unbundle and unpack. However, as seen with the existing makemsix tool, +as long as pack and bundle are separate operations it may not be especially useful for unbundle and unpack to be joined. Unpacking of all bundled packages might save time if someone wanted to examine the manifest of a main package inside the bundle, as might be an expected use case for a command line tool. But in cases where the api is actually being used programattically, to do something like extract all packages and update @@ -357,9 +347,21 @@ unpacked package. In such a case merging the unbundle and unpack command into on the caller, especially if pack and bundle have not also been merged to allow callers to use one call to package all subfolders with appxmanifests and then bundle all of them together. -For these reasons pack, bundle, unpack, unbundle have been left as separate commands. The MSIXMgr "unpack" command -is implemented here as another form of pack. +For these reasons pack, bundle, unpack, unbundle have been left as separate commands. + +MSIXMgr defines an unpack command which has fairly different usage from the makeappx unpack command. +The msixmgr unpack command mostly takes options that affect the phase of the operation that creates the vhdx\cim. +With the set of options to describe the output it looks more like the "pack\bundle" commands of the other tools. +In an api that supports creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers may +find it strange at first to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just +creating the vhdx directly. However, signing is not integrated into any of these tools, and is a separate step that +is done after packaging is complete. The Appx SIP knows how to sign packages, but not vhdx and other mounted directories. The result +is that creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, then signing it, then +creating the vhdx. +The msixmgr command line supports adding all packages in a folder to an image at once, but does not currently support expandable vhdx files. +CIM files meanwhile are always expandable. This means that the caller needs to know up front the size of all unpacked packages only when +creating vhds. This api proposes adding support for dynamic vhds\vhdx files in order to ease that burden . # Examples @@ -369,23 +371,55 @@ is implemented here as another form of pack. TEST_METHOD(TestPack) { winrt::init_apartment(); - auto packOptions = PackOptions(); - packOptions.OverwriteFiles(true); - //PackOptions.KozaniPackageFilePath(L"E:\\vm\\apps\\kozaniPackage.msix"); - packOptions.PackageFilePath(L"E:\\test\\packagedOutput.msix"); - - PackagingResult result{ MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", packOptions).get()}; - VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); - - auto kozaniPackOptions = PackOptions(); - kozaniPackOptions.OverwriteFiles(true); - kozaniPackOptions.CreateAsKozaniPackage(true); - kozaniPackOptions.PackageFilePath(L"E:\\test\\kozaniPackagedOutput.msix"); - - PackagingResult kozaniResult{ MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", kozaniPackOptions).get() }; - VERIFY_IS_TRUE(SUCCEEDED(kozaniResult.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(kozaniResult.ExtendedErrorCode())); + + // Create package from folder. + std::wstring packageOutputFilePath{ L"E:\\test\\packagedOutput.msix" }; + try + { + PackOptions packOptions = PackOptions(); + packOptions.OverwriteFiles(true); + packOptions.PackageFilePath(packageOutputFilePath); + + MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", packOptions).get(); + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); + } + + // Create kozani package from package. + try + { + CreateKozaniPackageOptions kozaniPackOptions = CreateKozaniPackageOptions(); + kozaniPackOptions.OverwriteFiles(true); + kozaniPackOptions.PackageFilePath(L"E:\\test\\kozaniPackagedOutput.msix"); + MakeMSIXManager::CreateKozaniPackage(packageOutputFilePath, kozaniPackOptions).get(); + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); + } + + // Create app attach vhd from package. + try + { + std::wstring appAttachImageFilePath{ L"E:\\test\\appAttachOutput.vhdx" }; + CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); + mountableImageOptions.DynamicallyExpandable(true); + mountableImageOptions.MaximumExpandableImageSizeMegabytes(100); + MakeMSIXManager::CreateMountableImage(appAttachImageFilePath, mountableImageOptions).get(); + + AddPackageToImageOptions addToImageOptions = AddPackageToImageOptions(); + addToImageOptions.PackageRootDirectoryInImage(L"WindowsApps"); + MakeMSIXManager::AddPackageToImage(packageOutputFilePath, appAttachImageFilePath, addToImageOptions).get(); + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); + } } ``` @@ -395,13 +429,20 @@ is implemented here as another form of pack. TEST_METHOD(TestBundle) { winrt::init_apartment(); - auto bundleOptions = BundleOptions(); - bundleOptions.OverwriteFiles(true); - bundleOptions.BundleFilePath(L"E:\\test\\bundledOutput.msixbundle"); + try + { + auto bundleOptions = BundleOptions(); + bundleOptions.OverwriteFiles(true); + bundleOptions.BundleFilePath(L"E:\\test\\bundledOutput.msixbundle"); + bundleOptions.BundleVersion(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); - PackagingResult result{ MakeMSIXManager::Bundle(L"E:\\test\\unbundledPackageInput", bundleOptions).get() }; - VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + MakeMSIXManager::Bundle(L"E:\\test\\unbundledPackageInput", bundleOptions).get(); + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); + } } ``` @@ -411,13 +452,19 @@ is implemented here as another form of pack. TEST_METHOD(TestUnpack) { winrt::init_apartment(); - auto unpackOptions = UnpackOptions(); - unpackOptions.OverwriteFiles(true); - unpackOptions.UnpackedPackageRootDirectory(L"E:\\test\\unpackedPackageOutput"); + try + { + auto unpackOptions = UnpackOptions(); + unpackOptions.OverwriteFiles(true); + unpackOptions.UnpackedPackageRootDirectory(L"E:\\test\\unpackedPackageOutput"); - PackagingResult result{ MakeMSIXManager::Unpack(L"E:\\test\\package.msix", unpackOptions).get() }; - VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); + MakeMSIXManager::Unpack(L"E:\\test\\package.msix", unpackOptions).get(); + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); + } } ``` @@ -427,27 +474,31 @@ is implemented here as another form of pack. TEST_METHOD(TestUnbundle) { winrt::init_apartment(); - std::wstring outputDir{ L"E:\\test\\unpackedBundleOutput" }; - auto unbundleOptions = UnbundleOptions(); - unbundleOptions.OverwriteFiles(true); - unbundleOptions.UnbundledPackageRootDirectory(outputDir); - - PackagingResult result{ MakeMSIXManager::Unbundle(L"E:\\test\\bundle.msixbundle", unbundleOptions).get() }; - VERIFY_IS_TRUE(SUCCEEDED(result.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(result.ExtendedErrorCode())); - - // Iterate through packages and unpack each one. - for (const auto& file : std::filesystem::directory_iterator(outputDir)) + try { - auto unpackOptions = UnpackOptions(); - unpackOptions.OverwriteFiles(true); - std::filesystem::path outputDirForPackage{ outputDir }; - outputDirForPackage /= file.path().filename(); - unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); - - PackagingResult unpackResult{ MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get()}; - VERIFY_IS_TRUE(SUCCEEDED(unpackResult.Status() == PackagingResultStatus::Ok)); - VERIFY_IS_TRUE(SUCCEEDED(unpackResult.ExtendedErrorCode())); + std::wstring outputDir{ L"E:\\test\\unpackedBundleOutput" }; + auto unbundleOptions = UnbundleOptions(); + unbundleOptions.OverwriteFiles(true); + unbundleOptions.UnbundledPackageRootDirectory(outputDir); + + MakeMSIXManager::Unbundle(L"E:\\test\\bundle.msixbundle", unbundleOptions).get(); + + // Iterate through packages and unpack each one. + for (const auto& file : std::filesystem::directory_iterator(outputDir)) + { + auto unpackOptions = UnpackOptions(); + unpackOptions.OverwriteFiles(true); + std::filesystem::path outputDirForPackage{ outputDir }; + outputDirForPackage /= file.path().stem(); + unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); + + MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get(); + } + } + catch (winrt::hresult_error const& ex) + { + OutputDebugString(ex.message().c_str()); + winrt::check_hresult(ex.code()); } } ``` @@ -463,69 +514,62 @@ namespace Microsoft.Kozani.MakeMSIX [contractversion(1)] apicontract MakeMSIXContract{}; - /// - /// Result of the packaging operation - /// + /// Options for creating a package [contract(MakeMSIXContract, 1)] - enum PackagingResultStatus + runtimeclass PackOptions { - Ok, - InvalidOptions, - FileAlreadyExistsError, - InternalError, - }; + /// + /// Created by callers to pass options into CreatePackage + /// + PackOptions(); + + /// + /// Output path for the full packaged file. + /// + String PackageFilePath{ get; set; }; - /// - /// Result of the packaging operation - /// - [contract(MakeMSIXContract, 1)] - runtimeclass PackagingResult - { /// - /// Status of the overall operation. + /// Overwrite PackageFilePath if it already exists. + /// Defaults to true. /// - PackagingResultStatus Status { get; }; + Boolean OverwriteFiles{ get; set; }; /// - /// Error code of the overall operation. + /// Validates elements of the package during creation. + /// Defaults to false. /// - HRESULT ExtendedErrorCode { get; }; + Boolean ValidateFiles{ get; set; }; }; - - /// Options for creating a package + + /// Options for creating a Kozani package [contract(MakeMSIXContract, 1)] - runtimeclass PackOptions + runtimeclass CreateKozaniPackageOptions { /// - /// Created by callers to pass options into CreatePackage + /// /// - PackOptions(); + CreateKozaniPackageOptions(); /// /// Output path for the full packaged file. + /// Format must match input file. If packageFilePathToConvert is an appx or msix, then PackageFilePath must be an appx or msix. + /// If packageFilePathToConvert is an appxbundle or msixbundle, then PackageFilePath must be an appxbundle or msixbundle. /// String PackageFilePath{ get; set; }; - // API REVIEW NOTE: Two options here. Needing to create a traditional package - // and a Kozani package at the same time may be fairly common for certain users - // but most users will not ever need to create Kozani packages. - // Either require the caller to make two separate calls, or allow creation of both - // packages in one call. My preference is separate calls, which makes it clearer - // that Kozani packages are optional. - /// - /// If true, writes a Kozani package to PackageFilePath. - /// Defaults to false. + /// Optional replacement package publisher. + /// If not set, publisher from appxmanifest.xml is unchaged. /// - Boolean CreateAsKozaniPackage{ get; set; }; + String PackagePublisher{ get; set; }; + /// - /// Output path for the Kozani package file. - /// At least one of PackageFilePath and KozaniPackageFilePath must be set. + /// Optional replacement package name. + /// If not set, name from appxmanifest.xml is unchaged. /// - String KozaniPackageFilePath{ get; set; }; - + String PackageName{ get; set; }; /// - /// Overwrite PackageFilePath and KozaniPackageFilePath if they already exist. + /// Overwrite PackageFilePath if it already exists. /// Defaults to true. /// Boolean OverwriteFiles{ get; set; }; @@ -557,8 +601,15 @@ namespace Microsoft.Kozani.MakeMSIX Boolean OverwriteFiles{ get; set; }; /// - /// Sets the version in the AppxBundleManifest.xml during packaging if - /// the source layout is a bundle. + /// Create as a flat bundle. Package locations will be stored as external path references + /// rather than being stored inside the msixbundle file itself. + /// Defaults to false. + /// + Boolean FlatBundle{ get; set; }; + + /// + /// Optional. Sets the version in the AppxBundleManifest.xml + /// If not set the version is created based on the current time. /// Windows.ApplicationModel.PackageVersion BundleVersion{ get; set; }; }; @@ -577,13 +628,13 @@ namespace Microsoft.Kozani.MakeMSIX /// /// The output folder to unpack the package into. /// - String UnpackedPackageRootDirectory; + String UnpackedPackageRootDirectory{ get; set; }; /// /// If true, the operation will overwrite existing files in the UnpackedPackageRootDirectory /// when unpacking. /// If false, the operation will fail if a file already exists. /// - Boolean OverwriteFiles; + Boolean OverwriteFiles{ get; set; }; }; /// @@ -600,13 +651,59 @@ namespace Microsoft.Kozani.MakeMSIX /// /// The output folder to unpack the bundle package into. /// - String UnbundledPackageRootDirectory; + String UnbundledPackageRootDirectory{ get; set; }; /// /// If true, the operation will overwrite existing files in the UnbundledPackageRootDirectory /// when unpacking. /// If false, the operation will fail if a file already exists. /// - Boolean OverwriteFiles; + Boolean OverwriteFiles{ get; set; }; + }; + + /// + /// + /// + [contract(MakeMSIXContract, 1)] + runtimeclass CreateMountableImageOptions + { + /// + /// Created by callers to pass options into CreateMountableImage + /// + CreateMountableImageOptions(); + /// + /// Fixed image size in GB. Valid for vhds and vhdx. CIM images automatically expand. + /// Default is 0. + /// + UInt32 FixedImageSize{ get; set; }; + + /// + /// Make image file expandable. + /// Default is true + /// + Boolean DynamicallyExpandable{ get; set; }; + /// + /// Fixed image size in GB. Valid for vhds and vhdx. CIM images have no limit. + /// Default is 1. + /// + UInt32 MaximumExpandableImageSize{ get; set; }; + }; + + /// + /// + /// + [contract(MakeMSIXContract, 1)] + runtimeclass AddPackageToImageOptions + { + /// + /// Created by callers to pass options into AddPackageToImage + /// + AddPackageToImageOptions(); + + /// + /// Root directory on an image to unpack packages to. + /// Required parameter for adding to CIM files. Optional for vhd\vhdx. + /// + String PackageRootDirectoryInImage{ get; set; }; }; /// @@ -618,19 +715,49 @@ namespace Microsoft.Kozani.MakeMSIX /// /// Creates packages from the directoryPathToPack as specified by the packOptions. /// - static Windows.Foundation.IAsyncOperation Pack(String directoryPathToPack, PackOptions packOptions); + static Windows.Foundation.IAsyncAction Pack(String directoryPathToPack, PackOptions packOptions); /// /// Creates bundles from the directoryPathToBundle as specified by the bundleOptions. /// - static Windows.Foundation.IAsyncOperation Bundle(String directoryPathToBundle, BundleOptions bundleOptions); + static Windows.Foundation.IAsyncAction Bundle(String directoryPathToBundle, BundleOptions bundleOptions); /// /// Unpacks a package at packageFilePathToUnpack as specified by the unpackOptions. /// - static Windows.Foundation.IAsyncOperation Unpack(String packageFilePathToUnpack, UnpackOptions unpackOptions); + static Windows.Foundation.IAsyncAction Unpack(String packageFilePathToUnpack, UnpackOptions unpackOptions); /// /// Unbundles a package at bundleFilePathToUnbundle as specified by the unbundleOptions. /// - static Windows.Foundation.IAsyncOperation Unbundle(String bundleFilePathToUnbundle, UnbundleOptions unbundleOptions); + static Windows.Foundation.IAsyncAction Unbundle(String bundleFilePathToUnbundle, + UnbundleOptions unbundleOptions); + /// + /// Creates a Kozani package from an existing package. + /// packageFilePathToConvert can be an appx, appxbundle, msix, or msixbundle + /// + static Windows.Foundation.IAsyncAction CreateKozaniPackage(String packageFilePathToConvert, + CreateKozaniPackageOptions createKozaniPackageOptions); + + + /// + /// Mount the image at imageFilePathToMount + /// + static Windows.Foundation.IAsyncAction Mount(String imageFilePathToMount, Boolean readOnly); + + /// + /// Unmount the image that was mounted from imageFilePathToUnmount + /// + static Windows.Foundation.IAsyncAction Unmount(String imageFilePathToUnmount); + + /// + /// Create mountable image at imageFilePath. Valid file extensions are vhd, vhdx, and CIM. + /// + static Windows.Foundation.IAsyncAction CreateMountableImage(String imageFilePath, + CreateMountableImageOptions createMountableImageOptions); + + /// + /// Add the package at packageFilePath to the image at imageFilePath. + /// + static Windows.Foundation.IAsyncAction AddPackageToImage(String packageFilePath, String imageFilePath, + AddPackageToImageOptions addPackageToImageOptions); }; } From 783b68c5ded5bd0b96ce8648a94a780b21ba1114 Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Tue, 7 Mar 2023 10:43:00 -0800 Subject: [PATCH 3/8] Update spec typos and add sections --- specs/WinRT/MakeMSIX-spec.md | 570 +++++++++++++++++++++++------------ 1 file changed, 381 insertions(+), 189 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index ef17b3cd28..b36a788186 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -1,28 +1,55 @@ -# Make MSIX API in Windows App SDK - -# Background - -The Make MSIX api allows developers to create app packages for distribution. The api -provides similar functionality as multiple existing command line tools. +# MakeMSIX API in Windows App SDK +- [1. Background](#1-background) +- [2. Existing interfaces and options](#2-existing-interfaces-and-options) + - [2.1. Makeappx](#21-makeappx) + - [2.1.1. pack](#211-pack) + - [2.1.2. bundle](#212-bundle) + - [2.1.3. unpack](#213-unpack) + - [2.1.4. unbundle](#214-unbundle) + - [2.2. Makemsix](#21-makemsix) + - [2.2.1. pack](#221-pack) + - [2.2.2. bundle](#222-bundle) + - [2.2.3. unpack](#223-unpack) + - [2.2.4. unbundle](#224-unbundle) + - [2.3. MSIXMgr](#23-msixmgr) + - [2.3.1. unpack](#231-unpack) +- [3. Summary of existing options in relation to proposed api](#3-summary-of-existing-options-in-relation-to-proposed-api) +- [4. Examples](#4-examples) + - [4.1. Packing a package](#41-packing-a-package) + - [4.2. Unbundling and unpacking a package to change information](#42-unbundling-and-unpacking-a-package-to-change-information) + - [4.3. Helper methods for examples](#43-helper-methods-for-examples) +- [5. API Details](#5-api-details) + +# 1. Background + +The MakeMSIX api allows developers to create app packages for distribution. The api +provides a WinRT interface that joins the functionality from multiple existing command line tools in +one developer friendly location to allow creation of msix\appx packages and msixbundle\appxbundle packages, +unpacking of those packages, and creation of package images for Azure App Attach. +In addition to functionality from those existing tools, this api adds support for creation of Kozani +packages. -For more details see: +# 2. Existing interfaces and options +The existing tools are: - [The makeappx command line tool](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) Shipped as part of the Windows Development Kit. Exposes pack, unpack, bundle, and unbundle commands, as well as encryption support - via a custom key-file list file, and mapping . + via a custom key-file list file, and creation of packages via a custom mapping file + to allow creation of packages with different payloads from the same folder. - [The makemsix command line tool and dll](https://github.com/Microsoft/msix-packaging) Available through the github project. - Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command is not implemented). + Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command only supports creation of + flat bundles. Flat bundles are bundles where package locations are stored as external path references + rather than being stored inside the msixbundle file itself. Flat bundles referred to as + sparse bundles in the makeappx documentation). - [The msixmgr command line tool](https://github.com/Microsoft/msix-packaging and https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr) Available through github and as a direct download from the learn.microsoft.com site. Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. -## The problems today - None of the existing tools provide a good programmatic interface for all aspects of packaging. The makeappx.exe built into windows provides the only complete implementation of packaging @@ -30,17 +57,15 @@ and bundling but requires CreateProcess and does not return any specific errors It also does not support any of the azure app attach scenarios. The makemsix tool is designed as a cross-platform implementation of packaging, but does not support -the bundle command. The functionality is exposed as a command line as well as flat dll exports. +a complete set of bundling features. The functionality is exposed as a command line as well as flat dll exports. The msixmgr tool is mostly for specific azure app attach packaging scenarios that are not covered by the other tools. To use it programmatically would require either using CreateProcess or building the msix-packaging github project and linking the static lib it produces into a project. -# Existing interfaces and options +## 2.1. Makeappx -## Makeappx - -### Pack +### 2.1.1. Pack Usage: ------ MakeAppx pack [options] /d /p @@ -107,7 +132,7 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -### Bundle +### 2.1.2. Bundle Usage: ------ @@ -145,7 +170,7 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -### Unpack +### 2.1.3. Unpack Usage: ------ MakeAppx unpack [options] /p /d @@ -175,7 +200,7 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -### Unbundle +### 2.1.4. Unbundle Usage: ------ @@ -201,11 +226,11 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -## MakeMsix +## 2.2. MakeMsix MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for encryption, content groups, mapping files, and bundle support is limited to -only allowing creation of flat-bundles (bundles where the appxbundle file itself +only allowing creation of flat bundles (bundles where the appxbundle file itself only references but does not contain the main and resource appxpackages) It does add the ability to optionally unpack all packages inside the bundle @@ -217,7 +242,7 @@ packages and bundle. Callers need to call makemsix pack on each individual package folder and then follow it with a makemsix bundle call to generate a flat bundle. -### Pack +### 2.2.1. Pack Usage: --------------- makemsix.exe pack -d -p [options] @@ -231,7 +256,7 @@ Options: -? [Flag] Displays this help text. -### Bundle +### 2.2.2. Bundle Usage: --------------- makemsix.exe bundle -p [options] @@ -259,7 +284,7 @@ Options: -? [Flag] Displays this help text. -### Unpack +### 2.2.3. Unpack Usage: --------------- makemsix.exe unpack -p -d [options] @@ -281,7 +306,7 @@ Options: -? [Flag] Displays this help text. -### Unbundle +### 2.2.4. Unbundle Usage: --------------- @@ -310,9 +335,9 @@ Options: -? [Flag] Displays this help text. -## MSIXMgr +## 2.3. MSIXMgr -### Unpack +### 2.3.1. Unpack msixmgr.exe -Unpack -packagePath -destination [-applyacls] [-create] [-vhdSize ] [-filetype ] [-rootDirectory ] -Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. @@ -325,118 +350,197 @@ msixmgr.exe -Unpack -packagePath -destination -validateSignature: optional parameter that validates a package's signature file before unpacking the package. This will require that the package's certificate is installed on the machine. -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. Use only for VHD or VHDX files -# Summary of existing options +# 3. Summary of existing options in relation to proposed api Existing options all treat pack and bundle as separate commands. Though they are similar, each command does have -some options that are irrelevant to the other command. For bundling that means specifying the Bundle Version, which -nearly all callers will specify (the alternative is the tool setting the version based on a timestamp), and allowing -creation of sparse bundles. For packaging that means hash algorithm, and creation of streamable packages and resource -packages. +some options that are irrelevant to the other command. Options that are relevant to bundling but not packaging are +the Bundle Version, which nearly all callers will specify (the alternative is the tool setting the version based on a timestamp), +and allowing creation of sparse bundles. Options that are relevant for packaging but not bundling are hash algorithm, and creation +of streamable packages and resource packages. In the long term it's likely that these differences in the information and flags required to support creating a bundle and creating a package will only continue to grow. -Unpack and Unbundle is a somewhat different story. If defined as an atomic operation where a package is unpacked to a folder, -it is less likely that differences will arise between unbundle and unpack. However, as seen with the existing makemsix tool, +It is less likely that differences will arise between unbundle and unpack. However, as seen with the existing makemsix tool, as long as pack and bundle are separate operations it may not be especially useful for unbundle and unpack to be joined. Unpacking of -all bundled packages might save time if someone wanted to examine the manifest of a main package inside the bundle, -as might be an expected use case for a command line tool. -But in cases where the api is actually being used programattically, to do something like extract all packages and update -every manifest, say with a different version number, there is no avoiding use of filesystem apis to find each -unpacked package. In such a case merging the unbundle and unpack command into one api call saves very little complexity for -the caller, especially if pack and bundle have not also been merged to allow callers to use one call to package all subfolders -with appxmanifests and then bundle all of them together. +all bundled packages makes sense for a command line tool where an IT admin may be typing in individual commands each time and so +there's a good reason to avoid having to run multiple commands. In cases where the api is actually being used programattically, +to do something like extract all packages and update every manifest with a different version number, there is no avoiding use of filesystem +apis to find each unpacked package. In such a case merging the unbundle and unpack command into one api just adds hidden complexity into the api +in the form of either requiring more configuration options or hiding decisions about default choices like folder naming decisions in the api documentation. For these reasons pack, bundle, unpack, unbundle have been left as separate commands. MSIXMgr defines an unpack command which has fairly different usage from the makeappx unpack command. -The msixmgr unpack command mostly takes options that affect the phase of the operation that creates the vhdx\cim. -With the set of options to describe the output it looks more like the "pack\bundle" commands of the other tools. -In an api that supports creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers may -find it strange at first to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just +The msixmgr unpack command merges creation of an image file and adding packages to that image file into one command. +The msixmgr command line supports adding all packages in a folder to an image at once. The CIM file format it supports always expands to fit those apps, +but msixmgr does not currently support expandable vhdx files. This means that the caller needs to know up front the size of all unpacked packages when +creating vhds, which will be difficult to know without either unpacking them first or reading each package's block map to get file sizes. +This api proposes getting rid of that requirement and having the api implementation itself determine the size of all packages being added in +order to create an image of the correct size. This behavior is more consistent with other types of package creation and eliminates the api differences +between CIM files and VHD\VHDX files. + +Since the api surface supports creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers may +find it strange to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just creating the vhdx directly. However, signing is not integrated into any of these tools, and is a separate step that is done after packaging is complete. The Appx SIP knows how to sign packages, but not vhdx and other mounted directories. The result is that creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, then signing it, then -creating the vhdx. +creating the vhdx. Allowing creation of a CIM, VHD, or VHDX file directly from a directory seems likely to cause confusion for +developers who may not realize that they won't be able to sign the package in the image. This api also proposes always applying acls +for the package to the folder, and never validating the signature. If signature validation of a package (checking that the root cert of a package +is trusted on the current machine) is an important api for callers it can be done as a separate operation from creation of an image file. -The msixmgr command line supports adding all packages in a folder to an image at once, but does not currently support expandable vhdx files. -CIM files meanwhile are always expandable. This means that the caller needs to know up front the size of all unpacked packages only when -creating vhds. This api proposes adding support for dynamic vhds\vhdx files in order to ease that burden . +# 4. Examples -# Examples - -## Packing a package +## 4.1. Packing a package ```cpp - TEST_METHOD(TestPack) + void CreatePackagesFromFolderExample() { winrt::init_apartment(); - // Create package from folder. - std::wstring packageOutputFilePath{ L"E:\\test\\packagedOutput.msix" }; - try + // Scenario: A developer is creating their own app. A build tool wants to create every type of distributable package for them. + + // Initial Conditions: The build has already produced a directory that contains the exact layout of each individual package + // that it wants to bundle. + // Each folder has an appxmanifest.xml in the root directory. + + // A full non-relative path to those layout folder is needed, as below. + // Layout directories are inside this root, for example: packageLayoutRootPath\\ContosoApp1_2023.302.1739.686_x64__8wekyb3d8bbwe. + std::wstring packageLayoutRootPath{ L"D:\\test\\ContosoApp1\\BuildOutput\\PackageLayouts" }; + + // The build tool packages each folder and puts the resulting packages at this location + std::wstring packageOutputRootDirectoryPath{ L"D:\\test\\ContosoApp1\\BuildOutput\\packagedOutput\\" }; + // The build tool wants to create the full bundle at this location + std::wstring bundleOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle" }; + // The build tool wants to create the kozani package at this location + std::wstring kozaniPackageOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\kozaniPackagedOutput.msix" }; + // The build tool wants to create the app attach image at this location + std::wstring appAttachImageFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\appAttachOutput.vhdx" }; + // The certificate to sign the package is at + std::wstring developerCertPfxFile{ L"D:\\test\\ContosoApp1\\developerCert.pfx" }; + + // Step 1. Create the full bundle. + + // Iterate through package layout folders and pack each one. + for (const std::filesystem::directory_entry& directoryEntry : std::filesystem::directory_iterator(packageLayoutRootPath)) { + if (!directoryEntry.is_directory()) + { + continue; + } PackOptions packOptions = PackOptions(); packOptions.OverwriteFiles(true); + std::wstring packageFolderName = directoryEntry.path().filename().wstring(); + std::wstring packageOutputFilePath{ packageOutputRootDirectoryPath + packageFolderName + L".appx" }; packOptions.PackageFilePath(packageOutputFilePath); - MakeMSIXManager::Pack(L"E:\\test\\unpackedPackageInput", packOptions).get(); - } - catch (winrt::hresult_error const& ex) - { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); - } - - // Create kozani package from package. - try - { - CreateKozaniPackageOptions kozaniPackOptions = CreateKozaniPackageOptions(); - kozaniPackOptions.OverwriteFiles(true); - kozaniPackOptions.PackageFilePath(L"E:\\test\\kozaniPackagedOutput.msix"); - MakeMSIXManager::CreateKozaniPackage(packageOutputFilePath, kozaniPackOptions).get(); - } - catch (winrt::hresult_error const& ex) - { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); - } - - // Create app attach vhd from package. - try - { - std::wstring appAttachImageFilePath{ L"E:\\test\\appAttachOutput.vhdx" }; - CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); - mountableImageOptions.DynamicallyExpandable(true); - mountableImageOptions.MaximumExpandableImageSizeMegabytes(100); - MakeMSIXManager::CreateMountableImage(appAttachImageFilePath, mountableImageOptions).get(); - - AddPackageToImageOptions addToImageOptions = AddPackageToImageOptions(); - addToImageOptions.PackageRootDirectoryInImage(L"WindowsApps"); - MakeMSIXManager::AddPackageToImage(packageOutputFilePath, appAttachImageFilePath, addToImageOptions).get(); - } - catch (winrt::hresult_error const& ex) - { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); + MakeMSIXManager::Pack(directoryEntry.path().c_str(), packOptions).get(); } + // Bundle all the packages together. + auto bundleOptions = BundleOptions(); + bundleOptions.OverwriteFiles(true); + bundleOptions.BundleFilePath(bundleOutputFilePath); + bundleOptions.BundleVersion(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); + MakeMSIXManager::Bundle(packageOutputRootDirectoryPath, bundleOptions).get(); + + // Sign the bundle + Example_SignPackage(developerCertPfxFile, bundleOutputFilePath); + + // Step 2. Create the kozani package from the bundle created in Step 1. + CreateKozaniPackageOptions kozaniPackOptions = CreateKozaniPackageOptions(); + kozaniPackOptions.OverwriteFiles(true); + kozaniPackOptions.PackageFilePath(kozaniPackageOutputFilePath); + MakeMSIXManager::CreateKozaniPackage(bundleOutputFilePath, kozaniPackOptions).get(); + + // Step 3. Create app attach vhd from the bundle created in Step 1. + CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); + mountableImageOptions.ImageFilePath(appAttachImageFilePath); + mountableImageOptions.OverwriteFiles(true); + winrt::Windows::Foundation::Collections::IVector packagesToAddToImage{ winrt::single_threaded_vector() }; + packagesToAddToImage.Append(bundleOutputFilePath); + + MakeMSIXManager::CreateMountableImage(packagesToAddToImage, mountableImageOptions).get(); } ``` -## Bundling a package +## 4.2. Unbundling and unpacking a package to change information ```cpp - TEST_METHOD(TestBundle) + void ChangeVersionOfAllPackagesInBundle() { winrt::init_apartment(); try { + // Scenario: A developer is writing code to update the version of all packages in a bundle. + + std::filesystem::path bundleFilePath{ L"D:\\test\\bundle.msixbundle" }; + std::wstring outputDirRoot{ L"D:\\test\\unpackedBundleOutput" }; + winrt::Windows::ApplicationModel::PackageVersion newVersion = winrt::Windows::ApplicationModel::PackageVersion(1, 0, 0, 0); + + std::filesystem::path outputDirForBundle{ outputDirRoot }; + outputDirForBundle /= L"bundle"; + std::filesystem::path outputDirRootForPackages{ outputDirRoot }; + outputDirRootForPackages /= L"unpackedPackages"; + + // Unbundle the bundle to its own folder. After the operation the folder will contain the + // bundled packages as appx\msix packages, as well as the bundle metadata. + auto unbundleOptions = UnbundleOptions(); + unbundleOptions.OverwriteFiles(true); + unbundleOptions.UnbundledPackageRootDirectory(outputDirForBundle.c_str()); + MakeMSIXManager::Unbundle(bundleFilePath.c_str(), unbundleOptions).get(); + + // Iterate through the bundled packages and unpack each one. + for (const auto& file : std::filesystem::directory_iterator(outputDirForBundle)) + { + // Skip the metadata files. + std::wstring fileExtension{ file.path().extension() }; + std::transform(fileExtension.begin(), fileExtension.end(), fileExtension.begin(), tolower); + if ((fileExtension.compare(L".appx") != 0) && + (fileExtension.compare(L".msix") != 0)) + { + continue; + } + + // Name the unpacked folders based on the package file name. + std::filesystem::path outputDirForPackage{ outputDirRootForPackages }; + outputDirForPackage /= file.path().stem(); + + auto unpackOptions = UnpackOptions(); + unpackOptions.OverwriteFiles(true); + unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); + MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get(); + } + + std::filesystem::path outputDirRootForPackedPackages{ outputDirRoot }; + outputDirRootForPackedPackages /= L"packedPackages"; + std::filesystem::path outputPathForBundle{ outputDirRoot }; + outputPathForBundle /= bundleFilePath.filename(); + + // Iterate through packages and repack each one. + for (const auto& packageDir : std::filesystem::directory_iterator(outputDirRootForPackages)) + { + // Open the manifest and change the version before re-packing + std::filesystem::path appxManifestPath{ packageDir }; + appxManifestPath /= L"AppxManifest.xml"; + Example_ChangeManifestVersion(appxManifestPath, newVersion); + + std::filesystem::path outputPackagePath{ outputDirRootForPackedPackages }; + std::wstring outputPackageFileName = packageDir.path().filename().wstring() + L".appx"; + outputPackagePath /= outputPackageFileName; + + auto packOptions = PackOptions(); + packOptions.OverwriteFiles(true); + packOptions.PackageFilePath(outputPackagePath.c_str()); + MakeMSIXManager::Pack(packageDir.path().c_str(), packOptions).get(); + } + + // Re-bundle with the new version auto bundleOptions = BundleOptions(); bundleOptions.OverwriteFiles(true); - bundleOptions.BundleFilePath(L"E:\\test\\bundledOutput.msixbundle"); - bundleOptions.BundleVersion(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); - - MakeMSIXManager::Bundle(L"E:\\test\\unbundledPackageInput", bundleOptions).get(); + bundleOptions.BundleFilePath(outputPathForBundle.c_str()); + bundleOptions.BundleVersion(newVersion); + MakeMSIXManager::Bundle(outputDirRootForPackedPackages.c_str(), bundleOptions).get(); } catch (winrt::hresult_error const& ex) { @@ -446,64 +550,178 @@ creating vhds. This api proposes adding support for dynamic vhds\vhdx files in o } ``` -## Unpacking a package +## 4.3. Helper methods for examples +Windows has existing non-WinRT apis for signing packages available in the CryptoAPI +https://learn.microsoft.com/en-us/windows/win32/seccrypto/using-cryptography +For completeness purposes some helper methods to instead find and call signtool.exe +from the Windows SDK are included here. ```cpp - TEST_METHOD(TestUnpack) + void Example_ChangeManifestVersion(std::wstring appxManifestPath, winrt::Windows::ApplicationModel::PackageVersion newVersion) { - winrt::init_apartment(); - try + winrt::Windows::Storage::StorageFile manifestFile = winrt::Windows::Storage::StorageFile::GetFileFromPathAsync(appxManifestPath).get(); + winrt::Windows::Data::Xml::Dom::XmlDocument docElement = winrt::Windows::Data::Xml::Dom::XmlDocument::LoadFromFileAsync(manifestFile).get(); + + std::wstring packageIdentityVersionQuery{ L"/*[local-name()='Package']/*[local-name()='Identity']/@Version" }; + winrt::Windows::Data::Xml::Dom::IXmlNode identityVersionAttributeNode = docElement.SelectSingleNode(packageIdentityVersionQuery.c_str()); + + std::wstring newVersionString = std::to_wstring(newVersion.Major) + L"." + + std::to_wstring(newVersion.Minor) + L"." + + std::to_wstring(newVersion.Build) + L"." + + std::to_wstring(newVersion.Revision); + identityVersionAttributeNode.InnerText(newVersionString); + + docElement.SaveToFileAsync(manifestFile).get(); + } + + /// + /// Create a process and synchronously wait for it to exit. + /// + /// CommandLine argument for CreateProcess + /// Exit code of the process + /// Success if process is created and exit code is returned. + HRESULT CreateProcessAndWaitForExitCode(std::wstring commandLine, DWORD& exitCode) + { + STARTUPINFO startupInfo; + ZeroMemory(&startupInfo, sizeof(startupInfo)); + startupInfo.cb = sizeof(startupInfo); + + PROCESS_INFORMATION processInformation; + ZeroMemory(&processInformation, sizeof(processInformation)); + + if (!CreateProcess( + nullptr, + const_cast(commandLine.c_str()), + nullptr, + nullptr, + FALSE, + 0, + nullptr, + nullptr, + &startupInfo, + &processInformation + )) { - auto unpackOptions = UnpackOptions(); - unpackOptions.OverwriteFiles(true); - unpackOptions.UnpackedPackageRootDirectory(L"E:\\test\\unpackedPackageOutput"); + return HRESULT_FROM_WIN32(GetLastError()); + } - MakeMSIXManager::Unpack(L"E:\\test\\package.msix", unpackOptions).get(); + DWORD waitResult = WaitForSingleObject(processInformation.hProcess, INFINITE); + switch (waitResult) + { + case WAIT_OBJECT_0: + break; + case WAIT_FAILED: + return HRESULT_FROM_WIN32(GetLastError()); + case WAIT_ABANDONED: + case WAIT_TIMEOUT: + default: + return E_UNEXPECTED; } - catch (winrt::hresult_error const& ex) + if (!GetExitCodeProcess(processInformation.hProcess, &exitCode)) { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); + return HRESULT_FROM_WIN32(GetLastError()); } - } -``` -## Unbundling a package + return S_OK; + } -```cpp - TEST_METHOD(TestUnbundle) + std::filesystem::path GetPlatformSDKPath() { - winrt::init_apartment(); - try + wil::unique_hkey hKey; + std::wstring sdkRegPath{ LR"(SOFTWARE\WOW6432Node\Microsoft\Microsoft SDKs\Windows\v10.0)" }; + winrt::check_hresult(RegOpenKeyEx(HKEY_LOCAL_MACHINE, sdkRegPath.c_str(), 0, KEY_READ, &hKey)); + + std::wstring installPathKeyName{ L"InstallationFolder" }; + WCHAR installPathBuffer[MAX_PATH]; + DWORD installPathBufferLength = sizeof(installPathBuffer); + winrt::check_hresult(RegGetValueW( + hKey.get(), + nullptr, + installPathKeyName.c_str(), + RRF_RT_REG_SZ, + nullptr /* pdwType */, + &installPathBuffer, + &installPathBufferLength)); + std::wstring installPathString{ installPathBuffer }; + + std::wstring productVersionKeyName{ L"ProductVersion" }; + WCHAR productVersionBuffer[MAX_PATH]; + DWORD productVersionBufferLength = sizeof(productVersionBuffer); + winrt::check_hresult(RegGetValueW( + hKey.get(), + nullptr, + productVersionKeyName.c_str(), + RRF_RT_REG_SZ, + nullptr /* pdwType */, + &productVersionBuffer, + &productVersionBufferLength)); + std::wstring productVersionString{ productVersionBuffer }; + // The install path of the platform sdk in the filesystem always uses version format x.x.x.x + // even though the version string in the registry may be stored as x.x if the full version is + // x.x.0.0 + auto versionPartsFound = std::count(productVersionString.begin(), productVersionString.end(), '.') + 1; + INT64 versionPartsRequired = 4; + INT64 versionDigitsToAdd = versionPartsRequired - versionPartsFound; + for (auto i = 0; i < versionDigitsToAdd; i++) { - std::wstring outputDir{ L"E:\\test\\unpackedBundleOutput" }; - auto unbundleOptions = UnbundleOptions(); - unbundleOptions.OverwriteFiles(true); - unbundleOptions.UnbundledPackageRootDirectory(outputDir); + productVersionString += L".0"; + } + std::wstring processorArchitecture{}; + SYSTEM_INFO systemInfo; + GetNativeSystemInfo(&systemInfo); + switch (systemInfo.wProcessorArchitecture) + { + case PROCESSOR_ARCHITECTURE_AMD64: + processorArchitecture += L"x64"; + break; + case PROCESSOR_ARCHITECTURE_ARM: + processorArchitecture += L"arm"; + break; + case PROCESSOR_ARCHITECTURE_ARM64: + processorArchitecture += L"arm64"; + break; + case PROCESSOR_ARCHITECTURE_INTEL: + processorArchitecture += L"x86"; + break; + case PROCESSOR_ARCHITECTURE_IA64: + case PROCESSOR_ARCHITECTURE_UNKNOWN: + winrt::throw_hresult(E_UNEXPECTED); + } - MakeMSIXManager::Unbundle(L"E:\\test\\bundle.msixbundle", unbundleOptions).get(); + std::filesystem::path platformSDKExecutablePath = installPathString; + platformSDKExecutablePath /= L"bin"; + platformSDKExecutablePath /= productVersionString; + platformSDKExecutablePath /= processorArchitecture; - // Iterate through packages and unpack each one. - for (const auto& file : std::filesystem::directory_iterator(outputDir)) - { - auto unpackOptions = UnpackOptions(); - unpackOptions.OverwriteFiles(true); - std::filesystem::path outputDirForPackage{ outputDir }; - outputDirForPackage /= file.path().stem(); - unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); + return platformSDKExecutablePath; + } - MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get(); - } - } - catch (winrt::hresult_error const& ex) + void LaunchSignToolWithArguments(std::wstring arguments) + { + std::filesystem::path signToolPath = GetPlatformSDKPath(); + signToolPath /= L"signtool.exe"; + + std::wstring commandLine{ signToolPath }; + commandLine += L" " + arguments; + DWORD exitCode{}; + winrt::check_hresult(CreateProcessAndWaitForExitCode(commandLine, exitCode)); + if (exitCode == EXIT_FAILURE) { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); + winrt::throw_hresult(E_FAIL); } } + + void Example_SignPackage(std::wstring pfxFile, std::wstring packageFile) + { + std::wstring signToolArguments; + signToolArguments.append(L"sign /fd SHA256 /f "); + signToolArguments.append(pfxFile); + signToolArguments.append(L" " + packageFile); + LaunchSignToolWithArguments(signToolArguments); + } ``` -# API Details +# 5. API Details ```c# // Copyright (c) Microsoft Corporation and Contributors. @@ -539,7 +757,7 @@ namespace Microsoft.Kozani.MakeMSIX /// Boolean ValidateFiles{ get; set; }; }; - + /// Options for creating a Kozani package [contract(MakeMSIXContract, 1)] runtimeclass CreateKozaniPackageOptions @@ -550,7 +768,7 @@ namespace Microsoft.Kozani.MakeMSIX CreateKozaniPackageOptions(); /// - /// Output path for the full packaged file. + /// Output path for the kozani packaged file. /// Format must match input file. If packageFilePathToConvert is an appx or msix, then PackageFilePath must be an appx or msix. /// If packageFilePathToConvert is an appxbundle or msixbundle, then PackageFilePath must be an appxbundle or msixbundle. /// @@ -573,6 +791,7 @@ namespace Microsoft.Kozani.MakeMSIX /// Defaults to true. /// Boolean OverwriteFiles{ get; set; }; + /// /// Validates elements of the package during creation. /// Defaults to false. @@ -624,7 +843,7 @@ namespace Microsoft.Kozani.MakeMSIX /// Created by callers to pass options into UnpackPackage /// UnpackOptions(); - + /// /// The output folder to unpack the package into. /// @@ -659,7 +878,7 @@ namespace Microsoft.Kozani.MakeMSIX /// Boolean OverwriteFiles{ get; set; }; }; - + /// /// /// @@ -670,40 +889,25 @@ namespace Microsoft.Kozani.MakeMSIX /// Created by callers to pass options into CreateMountableImage /// CreateMountableImageOptions(); - /// - /// Fixed image size in GB. Valid for vhds and vhdx. CIM images automatically expand. - /// Default is 0. - /// - UInt32 FixedImageSize{ get; set; }; /// - /// Make image file expandable. - /// Default is true - /// - Boolean DynamicallyExpandable{ get; set; }; - /// - /// Fixed image size in GB. Valid for vhds and vhdx. CIM images have no limit. - /// Default is 1. + /// Output path for the image file. + /// Valid file extensions are vhd, vhdx, and CIM. /// - UInt32 MaximumExpandableImageSize{ get; set; }; - }; + String ImageFilePath{ get; set; }; - /// - /// - /// - [contract(MakeMSIXContract, 1)] - runtimeclass AddPackageToImageOptions - { /// - /// Created by callers to pass options into AddPackageToImage + /// Root directory path for packages inside the image. + /// Defaults to "WindowsApps" /// - AddPackageToImageOptions(); + String PackageRootDirectoryInImage{ get; set; }; /// - /// Root directory on an image to unpack packages to. - /// Required parameter for adding to CIM files. Optional for vhd\vhdx. + /// If true, the operation will overwrite an existing file in the ImageFilePath + /// when unpacking. + /// If false, the operation will fail if the file already exists. /// - String PackageRootDirectoryInImage{ get; set; }; + Boolean OverwriteFiles{ get; set; }; }; /// @@ -727,37 +931,25 @@ namespace Microsoft.Kozani.MakeMSIX /// /// Unbundles a package at bundleFilePathToUnbundle as specified by the unbundleOptions. /// - static Windows.Foundation.IAsyncAction Unbundle(String bundleFilePathToUnbundle, + static Windows.Foundation.IAsyncAction Unbundle( + String bundleFilePathToUnbundle, UnbundleOptions unbundleOptions); + /// /// Creates a Kozani package from an existing package. /// packageFilePathToConvert can be an appx, appxbundle, msix, or msixbundle /// - static Windows.Foundation.IAsyncAction CreateKozaniPackage(String packageFilePathToConvert, + static Windows.Foundation.IAsyncAction CreateKozaniPackage( + String packageFilePathToConvert, CreateKozaniPackageOptions createKozaniPackageOptions); - - - /// - /// Mount the image at imageFilePathToMount - /// - static Windows.Foundation.IAsyncAction Mount(String imageFilePathToMount, Boolean readOnly); - - /// - /// Unmount the image that was mounted from imageFilePathToUnmount - /// - static Windows.Foundation.IAsyncAction Unmount(String imageFilePathToUnmount); /// - /// Create mountable image at imageFilePath. Valid file extensions are vhd, vhdx, and CIM. + /// Creates a mountable image that contains the packages at packageFilePathsToAdd. + /// Can be used for Azure App Attach. /// - static Windows.Foundation.IAsyncAction CreateMountableImage(String imageFilePath, + static Windows.Foundation.IAsyncAction CreateMountableImage( + Windows.Foundation.Collections.IVector packageFilePathsToAdd, CreateMountableImageOptions createMountableImageOptions); - - /// - /// Add the package at packageFilePath to the image at imageFilePath. - /// - static Windows.Foundation.IAsyncAction AddPackageToImage(String packageFilePath, String imageFilePath, - AddPackageToImageOptions addPackageToImageOptions); }; } From 5ba22c4c189a420a6611b5daf08805c0dab5bc23 Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Thu, 1 Jun 2023 12:47:54 -0700 Subject: [PATCH 4/8] Update from pr comments. --- specs/WinRT/MakeMSIX-spec.md | 1267 +++++++++++++++++----------------- 1 file changed, 630 insertions(+), 637 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index b36a788186..ac700ee331 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -1,398 +1,87 @@ # MakeMSIX API in Windows App SDK - [1. Background](#1-background) -- [2. Existing interfaces and options](#2-existing-interfaces-and-options) - - [2.1. Makeappx](#21-makeappx) - - [2.1.1. pack](#211-pack) - - [2.1.2. bundle](#212-bundle) - - [2.1.3. unpack](#213-unpack) - - [2.1.4. unbundle](#214-unbundle) - - [2.2. Makemsix](#21-makemsix) - - [2.2.1. pack](#221-pack) - - [2.2.2. bundle](#222-bundle) - - [2.2.3. unpack](#223-unpack) - - [2.2.4. unbundle](#224-unbundle) - - [2.3. MSIXMgr](#23-msixmgr) - - [2.3.1. unpack](#231-unpack) -- [3. Summary of existing options in relation to proposed api](#3-summary-of-existing-options-in-relation-to-proposed-api) -- [4. Examples](#4-examples) - - [4.1. Packing a package](#41-packing-a-package) - - [4.2. Unbundling and unpacking a package to change information](#42-unbundling-and-unpacking-a-package-to-change-information) - - [4.3. Helper methods for examples](#43-helper-methods-for-examples) -- [5. API Details](#5-api-details) +- [2. Summary of existing options in relation to proposed API](#2-summary-of-existing-options-in-relation-to-proposed-API) +- [3. Examples](#4-examples) + - [3.1. Packing a package](#41-packing-a-package) + - [3.2. Unbundling and unpacking a package to change information](#32-unbundling-and-unpacking-a-package-to-change-information) + - [3.3. Helper methods for examples](#33-helper-methods-for-examples) +- [4. API Details](#4-API-details) +- [5. Appendix](#5-Appendix) +- [6. Existing interfaces and options](#6-existing-interfaces-and-options) + - [6.1. Makeappx](#61-makeappx) + - [6.1.1. pack](#611-pack) + - [6.1.2. bundle](#612-bundle) + - [6.1.3. unpack](#613-unpack) + - [6.1.4. unbundle](#614-unbundle) + - [6.2. Makemsix](#62-makemsix) + - [6.2.1. pack](#621-pack) + - [6.2.2. bundle](#622-bundle) + - [6.2.3. unpack](#623-unpack) + - [6.2.4. unbundle](#624-unbundle) + - [6.3. MSIXMgr](#63-msixmgr) + - [6.3.1. unpack](#631-unpack) # 1. Background -The MakeMSIX api allows developers to create app packages for distribution. The api -provides a WinRT interface that joins the functionality from multiple existing command line tools in -one developer friendly location to allow creation of msix\appx packages and msixbundle\appxbundle packages, -unpacking of those packages, and creation of package images for Azure App Attach. -In addition to functionality from those existing tools, this api adds support for creation of Kozani -packages. +The MakeMSIX API allows developers to create app packages for distribution. The API provides a WinRT +interface that joins the functionality from multiple existing command line tools in one developer +friendly location to allow creation of msix\appx packages and msixbundle\appxbundle packages, +unpacking of those packages, and creation of package images for Azure App Attach. In addition to +functionality from those existing tools, this API adds support for creation of Kozani packages. -# 2. Existing interfaces and options +# 2. Summary of existing options in relation to proposed API -The existing tools are: -- [The makeappx command line tool](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) - Shipped as part of the Windows Development Kit. - Exposes pack, unpack, bundle, and unbundle commands, as well as encryption support - via a custom key-file list file, and creation of packages via a custom mapping file - to allow creation of packages with different payloads from the same folder. - -- [The makemsix command line tool and dll](https://github.com/Microsoft/msix-packaging) - Available through the github project. - Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command only supports creation of - flat bundles. Flat bundles are bundles where package locations are stored as external path references - rather than being stored inside the msixbundle file itself. Flat bundles referred to as - sparse bundles in the makeappx documentation). - -- [The msixmgr command line tool](https://github.com/Microsoft/msix-packaging and - https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr) - Available through github and as a direct download from the learn.microsoft.com site. - Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. - -None of the existing tools provide a good programmatic interface for all aspects of packaging. - -The makeappx.exe built into windows provides the only complete implementation of packaging -and bundling but requires CreateProcess and does not return any specific errors from process exit. -It also does not support any of the azure app attach scenarios. - -The makemsix tool is designed as a cross-platform implementation of packaging, but does not support -a complete set of bundling features. The functionality is exposed as a command line as well as flat dll exports. - -The msixmgr tool is mostly for specific azure app attach packaging scenarios that are not covered by -the other tools. To use it programmatically would require either using CreateProcess or building -the msix-packaging github project and linking the static lib it produces into a project. - -## 2.1. Makeappx - -### 2.1.1. Pack -Usage: ------- - MakeAppx pack [options] /d /p - MakeAppx pack [options] /f /p - MakeAppx pack [options] /m /f /p - MakeAppx pack [options] /r /m /f /p - MakeAppx pack [options] /d /ep /kf - MakeAppx pack [options] /d /ep /kt - -Options: --------- - /h, /hashAlgorithm: Specifies a hash algorithm to use for creating the - block map. Valid algorithm IDs are SHA256, SHA384, and SHA512. The - default is SHA256. - /nv, /noValidation: Skips validation that ensures the package will be - installable on Windows. The validation include: existence of files - referenced in manifest, ContentGroupMap correctness, and additional - manifest validation on Protocols and FileTypeAssociation. By default, - all semantic validation is performed. - /nfv, /noFileValidation: Skips validation that ensures that the files used - to create the package (from a folder through /d or layout file through - /l) exist and are accessible. By default, all files must exist and not - be read locked. If a file is inaccessible, it will not be added to the - package. - /nc, /noCompress: Prevents MakeAppx from compressing files in the package. - By default, files in the package are compressed based on detected file - type. - /m: Specifies the path to an input app manifest which will be used as the - basis for generating the output app package or resource package's - manifest. When you use this option, you must also use /f and include a - [ResourceMetadata] section in the mapping file to specify the resource - dimensions to be included in the generated manifest. - /r: Builds a resource package. You must use the /m option with this. - /kf: Use this option to encrypt or decrypt the package or bundle using a - key file. This option cannot be combined with /kt. - /kt: Use this option to encrypt or decrypt the package or bundle using the - global test key. This option cannot be combined with /kf. - /cgm: The input content group map (CGM) file path used to create a - stream-able package. Providing a content group map through this option - will disable the check that all files in the content group map exist in - the package. - /pri, /makepriExeFullPath: You can use /pri to override the default - MakePri.exe path with the custom fullpath from which makeappx.exe will - launch the tool from when needed. - /mp: Specifies the path to a main package in the same package family as the - resource package being built. You should provide this when encrypting - a resource package or when the resource package contains a content - group map. - /pb, /publisherBridging: Use this option to add publisher bridging entries - to the package or bundle. Publisher bridging is useful when the new - issued cert subject name changed but the old publisher name is still - desired to maintain package identity continuity. - /np, /noParallel: Use this option to disable parallel execution of this - command. - /ml, /memoryLimit: Specify maximum memory in bytes that MakeAppx should - consume while executing in parallel. By default, this value is set to - half of total physical memory. - /o, /overwrite: Forces the output to overwrite any existing files with the - same name. By default, the user is asked whether to overwrite existing - files with the same name. You can't use this option with /no. - /no, /noOverwrite: Prevents the output from overwriting any existing files - with the same name. By default, the user is asked whether to overwrite - existing files with the same name. You can't use this option with /o. - /v, /verbose: Enables verbose output of messages to the console. - /?, /help: Displays this help text. - -### 2.1.2. Bundle - -Usage: ------- - MakeAppx bundle [options] /d /p - MakeAppx bundle [options] /f /p - MakeAppx bundle [options] /d /ep /kf MyKeyFile.txt - MakeAppx bundle [options] /f /ep /kt - -Options: --------- - /bv: Specifies the version number of the bundle being created. The version - must be in dotted-quad notation of four integers - ... ranging from 0 to 65535 each. If the - /bv option is not specified or is set to 0.0.0.0, the bundle is created - using the current date-time formatted as the version: - .... - /mo: Generates a bundle manifest only, instead of a full bundle. Input - files must all be package manifests in XML format if this option is - specified. - /fb: Generates a fully sparse bundle where all packages are references to - packages that exist outside of the bundle file - /pri, /makepriExeFullPath: You can use /pri to override the default - MakePri.exe path with the custom fullpath from which makeappx.exe will - launch the tool from when needed. - /kf: Use this option to encrypt or decrypt the package or bundle using a - key file. This option cannot be combined with /kt. - /kt: Use this option to encrypt or decrypt the package or bundle using the - global test key. This option cannot be combined with /kf. - /o, /overwrite: Forces the output to overwrite any existing files with the - same name. By default, the user is asked whether to overwrite existing - files with the same name. You can't use this option with /no. - /no, /noOverwrite: Prevents the output from overwriting any existing files - with the same name. By default, the user is asked whether to overwrite - existing files with the same name. You can't use this option with /o. - /v, /verbose: Enables verbose output of messages to the console. - /?, /help: Displays this help text. - -### 2.1.3. Unpack -Usage: ------- - MakeAppx unpack [options] /p /d - MakeAppx unpack [options] /ep /d /kf - MakeAppx unpack [options] /ep /d /kt - -Options: --------- - /pfn: Unpacks all files to a subdirectory under the specified output path, - named after the package full name. - /nv, /noValidation: Skips validation that ensures the package will be - installable on Windows. The validation include: existence of files - referenced in manifest, ContentGroupMap correctness, and additional - manifest validation on Protocols and FileTypeAssociation. By default, - all semantic validation is performed. - /kf: Use this option to encrypt or decrypt the package or bundle using a - key file. This option cannot be combined with /kt. - /kt: Use this option to encrypt or decrypt the package or bundle using the - global test key. This option cannot be combined with /kf. - /nd: Skips decryption when unpacking an encrypted package or bundle. - /o, /overwrite: Forces the output to overwrite any existing files with the - same name. By default, the user is asked whether to overwrite existing - files with the same name. You can't use this option with /no. - /no, /noOverwrite: Prevents the output from overwriting any existing files - with the same name. By default, the user is asked whether to overwrite - existing files with the same name. You can't use this option with /o. - /v, /verbose: Enables verbose output of messages to the console. - /?, /help: Displays this help text. - -### 2.1.4. Unbundle - -Usage: ------- - MakeAppx unbundle [options] /p /d - MakeAppx unbundle [options] /ep /d /kf - MakeAppx unbundle [options] /ep /d /kt - -Options: --------- - /pfn: Unpacks all files to a subdirectory under the specified output path, - named after the package full name. - /kf: Use this option to encrypt or decrypt the package or bundle using a - key file. This option cannot be combined with /kt. - /kt: Use this option to encrypt or decrypt the package or bundle using the - global test key. This option cannot be combined with /kf. - /nd: Skips decryption when unpacking an encrypted package or bundle. - /o, /overwrite: Forces the output to overwrite any existing files with the - same name. By default, the user is asked whether to overwrite existing - files with the same name. You can't use this option with /no. - /no, /noOverwrite: Prevents the output from overwriting any existing files - with the same name. By default, the user is asked whether to overwrite - existing files with the same name. You can't use this option with /o. - /v, /verbose: Enables verbose output of messages to the console. - /?, /help: Displays this help text. - -## 2.2. MakeMsix - -MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for -encryption, content groups, mapping files, and bundle support is limited to -only allowing creation of flat bundles (bundles where the appxbundle file itself -only references but does not contain the main and resource appxpackages) - -It does add the ability to optionally unpack all packages inside the bundle -in one command using the -pfn-flat option, as in: -makemsix.exe unbundle -p -d -pfn-flat - -There is, however, no good option for reversing this command when recreating the -packages and bundle. Callers need to call makemsix pack on each individual -package folder and then follow it with a makemsix bundle call to generate a -flat bundle. - -### 2.2.1. Pack -Usage: ---------------- - makemsix.exe pack -d -p [options] - -Options: ---------------- - -d {Required} - Input directory path. - -p {Required} - Output package file path. - -? [Flag] - Displays this help text. - -### 2.2.2. Bundle -Usage: ---------------- - makemsix.exe bundle -p [options] - -Options: ---------------- - -d - Input directory path. - -p {Required} - Output bundle file path. - -f - Mapping file path. - -bv - Specifies the version number of the bundle being created. The version must be in dotted - quad notation of four integers ... ranging from 0 to 65535 each. If the -bv option is not specified or is set to 0.0.0.0, the bundle is created using the current date - time formatted as the version: .... - -mo [Flag] - Generates a bundle manifest only, instead of a full bundle. Input files must all be package manifests in XML format if this option is specified. - -fb [Flag] - Generates a fully sparse bundle where all packages are references to packages that exist outside of the bundle file. - -o [Flag] - Forces the output to overwrite any existing files with the same name.By default, the user is asked whether to overwrite existing files with the same name.You can't use this option with -no. - -no [Flag] - Prevents the output from overwriting any existing files with the same name. By default, the user is asked whether to overwrite existing files with the same name.You can't use this option with -o. - -v [Flag] - Enables verbose output of messages to the console. - -? [Flag] - Displays this help text. - -### 2.2.3. Unpack -Usage: ---------------- - makemsix.exe unpack -p -d [options] - -Options: ---------------- - -p {Required} - Input package file path. - -d {Required} - Output directory path. - -pfn [Flag] - Unpacks all files to a subdirectory under the output path, named after the package full name. - -ac [Flag] - Allows any certificate. By default the signature origin must be known. - -ss [Flag] - Skips enforcement of signed packages. By default packages must be signed. - -pfn-flat [Flag] - Same behavior as -pfn for packages. - -? [Flag] - Displays this help text. - -### 2.2.4. Unbundle - -Usage: ---------------- - makemsix.exe unbundle -p -d [options] - -Options: ---------------- - -p {Required} - Input bundle file path. - -d {Required} - Output directory path. - -pfn [Flag] - Unpacks all files to a subdirectory under the output path, named after the package full name. - -ac [Flag] - Allows any certificate. By default the signature origin must be known. - -ss [Flag] - Skips enforcement of signed packages. By default packages must be signed. - -sl [Flag] - Skips matching packages with the language of the system. By default unpacked resources packages will match the system languages. - -sp [Flag] - Skips matching packages with of the same system. By default unpacked application packages will only match the platform. - -extract-all [Flag] - Extracts all packages from the bundle. - -pfn-flat [Flag] - Unpacks bundle's files to a subdirectory under the specified output path, named after the package full name. Unpacks packages to subdirectories also under the specified output path, named after the package full name. By default unpacked packages will be nested inside the bundle folder. - -? [Flag] - Displays this help text. - -## 2.3. MSIXMgr - -### 2.3.1. Unpack -msixmgr.exe -Unpack -packagePath -destination [-applyacls] [-create] [-vhdSize ] [-filetype ] [-rootDirectory ] - --Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. - -applyACLs: optional parameter that applies ACLs to the resulting package folder(s) and their parent folder - -create: optional parameter that creates a new image with the specified -filetype and unpacks the packages to that image - -destination: the directory to place the resulting package folder(s) in - -fileType: the type of file to unpack packages to. Valid file types include {VHD, VHDX, CIM}. This is a required parameter when unpacking to CIM files - -packagePath: the path to the package to unpack OR the path to a directory containing multiple packages to unpack - -rootDirectory: root directory on an image to unpack packages to. Required parameter for unpacking to new and existing CIM files - -validateSignature: optional parameter that validates a package's signature file before unpacking the package. This will require that the package's certificate is installed on the machine. - -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. Use only for VHD or VHDX files - -# 3. Summary of existing options in relation to proposed api - -Existing options all treat pack and bundle as separate commands. Though they are similar, each command does have -some options that are irrelevant to the other command. Options that are relevant to bundling but not packaging are -the Bundle Version, which nearly all callers will specify (the alternative is the tool setting the version based on a timestamp), -and allowing creation of sparse bundles. Options that are relevant for packaging but not bundling are hash algorithm, and creation +Existing options all treat pack and bundle as separate commands. Though they are similar, each +command does have some options that are irrelevant to the other command. Options that are relevant +to bundling but not packaging are the Bundle Version, which nearly all callers will specify (the +alternative is the tool setting the version based on a timestamp), and allowing creation of flat +bundles. Options that are relevant for packaging but not bundling are hash algorithm, and creation of streamable packages and resource packages. -In the long term it's likely that these differences in the information and flags required to support creating a bundle -and creating a package will only continue to grow. +In the long term it's likely that these differences in the information and flags required to support +creating a bundle and creating a package will only continue to grow. -It is less likely that differences will arise between unbundle and unpack. However, as seen with the existing makemsix tool, -as long as pack and bundle are separate operations it may not be especially useful for unbundle and unpack to be joined. Unpacking of -all bundled packages makes sense for a command line tool where an IT admin may be typing in individual commands each time and so -there's a good reason to avoid having to run multiple commands. In cases where the api is actually being used programattically, -to do something like extract all packages and update every manifest with a different version number, there is no avoiding use of filesystem -apis to find each unpacked package. In such a case merging the unbundle and unpack command into one api just adds hidden complexity into the api -in the form of either requiring more configuration options or hiding decisions about default choices like folder naming decisions in the api documentation. +It is less likely that differences will arise between unbundle and unpack. However, as seen with the +existing makemsix tool, as long as pack and bundle are separate operations it may not be especially +useful for unbundle and unpack to be joined. Unpacking of all bundled packages makes sense for a +command line tool where an IT admin may be typing in individual commands each time and so there's a +good reason to avoid having to run multiple commands. In cases where the API is actually being used +programattically, to do something like extract all packages and update every manifest with a +different version number, there is no avoiding use of filesystem APIs to find each unpacked package. +In such a case merging the unbundle and unpack command into one API just adds hidden complexity into +the API in the form of either requiring more configuration options or hiding decisions about default +choices like folder naming decisions in the API documentation. For these reasons pack, bundle, unpack, unbundle have been left as separate commands. MSIXMgr defines an unpack command which has fairly different usage from the makeappx unpack command. -The msixmgr unpack command merges creation of an image file and adding packages to that image file into one command. -The msixmgr command line supports adding all packages in a folder to an image at once. The CIM file format it supports always expands to fit those apps, -but msixmgr does not currently support expandable vhdx files. This means that the caller needs to know up front the size of all unpacked packages when -creating vhds, which will be difficult to know without either unpacking them first or reading each package's block map to get file sizes. -This api proposes getting rid of that requirement and having the api implementation itself determine the size of all packages being added in -order to create an image of the correct size. This behavior is more consistent with other types of package creation and eliminates the api differences -between CIM files and VHD\VHDX files. - -Since the api surface supports creation of the CIM, VHD, VHDX file as well as creation of msix and msixbundle files, callers may -find it strange to need to first pack their directory to an msix, and then create the vhdx from the msix, rather than just -creating the vhdx directly. However, signing is not integrated into any of these tools, and is a separate step that -is done after packaging is complete. The Appx SIP knows how to sign packages, but not vhdx and other mounted directories. The result -is that creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, then signing it, then -creating the vhdx. Allowing creation of a CIM, VHD, or VHDX file directly from a directory seems likely to cause confusion for -developers who may not realize that they won't be able to sign the package in the image. This api also proposes always applying acls -for the package to the folder, and never validating the signature. If signature validation of a package (checking that the root cert of a package -is trusted on the current machine) is an important api for callers it can be done as a separate operation from creation of an image file. - -# 4. Examples - -## 4.1. Packing a package +The msixmgr unpack command merges creation of an image file and adding packages to that image file +into one command. The msixmgr command line supports adding all packages in a folder to an image at +once. The CIM file format it supports always expands to fit those apps, but msixmgr does not +currently support expandable vhdx files. This means that the caller needs to know up front the size +of all unpacked packages when creating vhds, which will be difficult to know without either +unpacking them first or reading each package's block map to get file sizes. This API proposes +getting rid of that requirement and having the API implementation itself determine the size of all +packages being added in order to create an image of the correct size. This behavior is more +consistent with other types of package creation and eliminates the API differences between CIM files +and VHD\VHDX files. + +As the API supports creation of .cim, .vhd, and .vhdx file as well as .msix and .msixbundle files, +callers may find it strange to need to first pack their directory to an msix, and then create the +vhdx from the msix, rather than just creating the vhdx directly. However, signing is not integrated +into any of these tools, and is a separate step that is done after packaging is complete. The Appx +SIP knows how to sign packages, but not vhdx and other mounted directories. The result is that +creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, +then signing it, then creating the vhdx. Allowing creation of a cim, vhd, or vhdx file directly from +a directory seems likely to cause confusion for developers who may not realize that they won't be +able to sign the package in the image. This API always applies ACLs for the package +to the folder, and never validates the signature. If signature validation of a package (checking +that the root cert of a package is trusted on the current machine) is an important API for callers +it can be done as a separate operation from creation of an image file. + +# 3. Examples + +## 3.1. Packing a package ```cpp void CreatePackagesFromFolderExample() @@ -429,42 +118,40 @@ is trusted on the current machine) is an important api for callers it can be don { continue; } - PackOptions packOptions = PackOptions(); - packOptions.OverwriteFiles(true); + CreatePackageOptions createPackageOptions = CreatePackageOptions(); + createPackageOptions.OverwriteOutputFileIfExists(true); std::wstring packageFolderName = directoryEntry.path().filename().wstring(); std::wstring packageOutputFilePath{ packageOutputRootDirectoryPath + packageFolderName + L".appx" }; - packOptions.PackageFilePath(packageOutputFilePath); - MakeMSIXManager::Pack(directoryEntry.path().c_str(), packOptions).get(); + MakeMSIXManager::CreatePackage(directoryEntry.path().c_str(), packageOutputFilePath.c_str(), createPackageOptions).get(); } // Bundle all the packages together. - auto bundleOptions = BundleOptions(); - bundleOptions.OverwriteFiles(true); - bundleOptions.BundleFilePath(bundleOutputFilePath); - bundleOptions.BundleVersion(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); - MakeMSIXManager::Bundle(packageOutputRootDirectoryPath, bundleOptions).get(); + CreateBundleOptions createbundleOptions = CreateBundleOptions(); + createbundleOptions.OverwriteOutputFileIfExists(true); + createbundleOptions.FlatBundle(true); + createbundleOptions.Version(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); + MakeMSIXManager::CreateBundle(packageOutputRootDirectoryPath, bundleOutputFilePath.c_str(), createbundleOptions).get(); // Sign the bundle Example_SignPackage(developerCertPfxFile, bundleOutputFilePath); // Step 2. Create the kozani package from the bundle created in Step 1. - CreateKozaniPackageOptions kozaniPackOptions = CreateKozaniPackageOptions(); - kozaniPackOptions.OverwriteFiles(true); - kozaniPackOptions.PackageFilePath(kozaniPackageOutputFilePath); - MakeMSIXManager::CreateKozaniPackage(bundleOutputFilePath, kozaniPackOptions).get(); + CreateKozaniPackageOptions createKozaniPackageOptions = CreateKozaniPackageOptions(); + createKozaniPackageOptions.OverwriteOutputFileIfExists(true); + createKozaniPackageOptions.RemoveExtensions(true); + MakeMSIXManager::CreateKozaniPackage(bundleOutputFilePath, kozaniPackageOutputFilePath.c_str(), createKozaniPackageOptions).get(); // Step 3. Create app attach vhd from the bundle created in Step 1. CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); - mountableImageOptions.ImageFilePath(appAttachImageFilePath); - mountableImageOptions.OverwriteFiles(true); + mountableImageOptions.OverwriteOutputFileIfExists(true); winrt::Windows::Foundation::Collections::IVector packagesToAddToImage{ winrt::single_threaded_vector() }; packagesToAddToImage.Append(bundleOutputFilePath); - MakeMSIXManager::CreateMountableImage(packagesToAddToImage, mountableImageOptions).get(); + MakeMSIXManager::CreateMountableImage(packagesToAddToImage, appAttachImageFilePath.c_str(), mountableImageOptions).get(); } ``` -## 4.2. Unbundling and unpacking a package to change information +## 3.2. Unbundling and unpacking a package to change information ```cpp void ChangeVersionOfAllPackagesInBundle() @@ -485,17 +172,16 @@ is trusted on the current machine) is an important api for callers it can be don // Unbundle the bundle to its own folder. After the operation the folder will contain the // bundled packages as appx\msix packages, as well as the bundle metadata. - auto unbundleOptions = UnbundleOptions(); - unbundleOptions.OverwriteFiles(true); - unbundleOptions.UnbundledPackageRootDirectory(outputDirForBundle.c_str()); - MakeMSIXManager::Unbundle(bundleFilePath.c_str(), unbundleOptions).get(); + auto extractBundleOptions = ExtractBundleOptions(); + extractBundleOptions.OverwriteOutputFilesIfExists(true); + MakeMSIXManager::ExtractBundle(bundleFilePath.c_str(), outputDirForBundle.c_str(), extractBundleOptions).get(); // Iterate through the bundled packages and unpack each one. for (const auto& file : std::filesystem::directory_iterator(outputDirForBundle)) { // Skip the metadata files. std::wstring fileExtension{ file.path().extension() }; - std::transform(fileExtension.begin(), fileExtension.end(), fileExtension.begin(), tolower); + std::transform(fileExtension.begin(), fileExtension.end(), fileExtension.begin(), towlower); if ((fileExtension.compare(L".appx") != 0) && (fileExtension.compare(L".msix") != 0)) { @@ -506,10 +192,9 @@ is trusted on the current machine) is an important api for callers it can be don std::filesystem::path outputDirForPackage{ outputDirRootForPackages }; outputDirForPackage /= file.path().stem(); - auto unpackOptions = UnpackOptions(); - unpackOptions.OverwriteFiles(true); - unpackOptions.UnpackedPackageRootDirectory(outputDirForPackage.c_str()); - MakeMSIXManager::Unpack(file.path().c_str(), unpackOptions).get(); + auto extractPackageOptions = ExtractPackageOptions(); + extractPackageOptions.OverwriteOutputFilesIfExists(true); + MakeMSIXManager::ExtractPackage(file.path().c_str(), outputDirForPackage.c_str(), extractPackageOptions).get(); } std::filesystem::path outputDirRootForPackedPackages{ outputDirRoot }; @@ -520,27 +205,22 @@ is trusted on the current machine) is an important api for callers it can be don // Iterate through packages and repack each one. for (const auto& packageDir : std::filesystem::directory_iterator(outputDirRootForPackages)) { - // Open the manifest and change the version before re-packing - std::filesystem::path appxManifestPath{ packageDir }; - appxManifestPath /= L"AppxManifest.xml"; - Example_ChangeManifestVersion(appxManifestPath, newVersion); - std::filesystem::path outputPackagePath{ outputDirRootForPackedPackages }; std::wstring outputPackageFileName = packageDir.path().filename().wstring() + L".appx"; outputPackagePath /= outputPackageFileName; - auto packOptions = PackOptions(); - packOptions.OverwriteFiles(true); - packOptions.PackageFilePath(outputPackagePath.c_str()); - MakeMSIXManager::Pack(packageDir.path().c_str(), packOptions).get(); + // Change the version when re-packing + auto createPackageOptions = CreatePackageOptions(); + createPackageOptions.OverwriteOutputFileIfExists(true); + createPackageOptions.Version(newVersion); + MakeMSIXManager::CreatePackage(packageDir.path().c_str(), outputPackagePath.c_str(), createPackageOptions).get(); } // Re-bundle with the new version - auto bundleOptions = BundleOptions(); - bundleOptions.OverwriteFiles(true); - bundleOptions.BundleFilePath(outputPathForBundle.c_str()); - bundleOptions.BundleVersion(newVersion); - MakeMSIXManager::Bundle(outputDirRootForPackedPackages.c_str(), bundleOptions).get(); + auto createBundleOptions = CreateBundleOptions(); + createBundleOptions.OverwriteOutputFileIfExists(true); + createBundleOptions.Version(newVersion); + MakeMSIXManager::CreateBundle(outputDirRootForPackedPackages.c_str(), outputPathForBundle.c_str(), createBundleOptions).get(); } catch (winrt::hresult_error const& ex) { @@ -550,30 +230,13 @@ is trusted on the current machine) is an important api for callers it can be don } ``` -## 4.3. Helper methods for examples +## 3.3. Helper methods for examples -Windows has existing non-WinRT apis for signing packages available in the CryptoAPI +Windows has existing non-WinRT APIs for signing packages available in the CryptoAPI https://learn.microsoft.com/en-us/windows/win32/seccrypto/using-cryptography For completeness purposes some helper methods to instead find and call signtool.exe from the Windows SDK are included here. ```cpp - void Example_ChangeManifestVersion(std::wstring appxManifestPath, winrt::Windows::ApplicationModel::PackageVersion newVersion) - { - winrt::Windows::Storage::StorageFile manifestFile = winrt::Windows::Storage::StorageFile::GetFileFromPathAsync(appxManifestPath).get(); - winrt::Windows::Data::Xml::Dom::XmlDocument docElement = winrt::Windows::Data::Xml::Dom::XmlDocument::LoadFromFileAsync(manifestFile).get(); - - std::wstring packageIdentityVersionQuery{ L"/*[local-name()='Package']/*[local-name()='Identity']/@Version" }; - winrt::Windows::Data::Xml::Dom::IXmlNode identityVersionAttributeNode = docElement.SelectSingleNode(packageIdentityVersionQuery.c_str()); - - std::wstring newVersionString = std::to_wstring(newVersion.Major) + L"." - + std::to_wstring(newVersion.Minor) + L"." - + std::to_wstring(newVersion.Build) + L"." - + std::to_wstring(newVersion.Revision); - identityVersionAttributeNode.InnerText(newVersionString); - - docElement.SaveToFileAsync(manifestFile).get(); - } - /// /// Create a process and synchronously wait for it to exit. /// @@ -721,7 +384,7 @@ from the Windows SDK are included here. } ``` -# 5. API Details +# 4. API Details ```c# // Copyright (c) Microsoft Corporation and Contributors. @@ -734,226 +397,556 @@ namespace Microsoft.Kozani.MakeMSIX /// Options for creating a package [contract(MakeMSIXContract, 1)] - runtimeclass PackOptions + runtimeclass CreatePackageOptions + { + /// Created by callers to pass options into CreatePackage() + CreatePackageOptions(); + + /// Optional replacement package name. + /// If field is empty, values from AppxManifest.xml are unchanged. + String Name; + + /// Optional replacement package publisher. + /// If field is empty, values from AppxManifest.xml are unchanged. + String Publisher; + + /// Optional replacement package version + /// If version is 0, version is created based on the current time. + Windows.ApplicationModel.PackageVersion Version; + + /// Overwrite output file if it already exists. + /// Defaults to true. + Boolean OverwriteOutputFileIfExists; + + /// Validates elements of the package during creation. + /// Defaults to false. + Boolean ValidateFiles; + }; + + /// Options for creating a Kozani package + [contract(MakeMSIXContract, 1)] + runtimeclass CreateKozaniPackageOptions { - /// - /// Created by callers to pass options into CreatePackage - /// - PackOptions(); + /// Created by callers to pass options into CreateKozaniPackage() + CreateKozaniPackageOptions(); + + /// Remove extensions from the manifest. + /// Defaults to false. + Boolean RemoveExtensions; + + /// Optional replacement package name. + /// If field is empty, values from AppxManifest.xml are unchanged. + String Name; + + /// Optional replacement package publisher. + /// If field is empty, values from AppxManifest.xml are unchanged. + String Publisher; + + /// Optional replacement package version + /// If version is 0, version is created based on the current time. + Windows.ApplicationModel.PackageVersion Version; - /// + /// Overwrite output file if it already exists. + /// Defaults to true. + Boolean OverwriteOutputFileIfExists; + + /// Validates elements of the package during creation. + /// Defaults to false. + Boolean ValidateFiles; + }; + + /// Options for creating a bundle + [contract(MakeMSIXContract, 1)] + runtimeclass CreateBundleOptions + { + /// Created by callers to pass options into CreateBundle() + CreateBundleOptions(); + + /// Create as a flat bundle. Package locations will be stored as external path references + /// rather than being stored inside the msixbundle file itself. + /// Defaults to false. + Boolean FlatBundle; + + /// Optional. Sets the version in the AppxBundleManifest.xml + /// If not set the version is created based on the current time. + Windows.ApplicationModel.PackageVersion Version; + + /// Overwrite output file if it already exists. + /// Defaults to true. + Boolean OverwriteOutputFileIfExists; + }; + + /// Options for unpacking a package + [contract(MakeMSIXContract, 1)] + runtimeclass ExtractPackageOptions + { + /// Created by callers to pass options into ExtractPackage() + ExtractPackageOptions(); + + /// If true, the operation will overwrite existing files in the output path + /// when unpacking. + /// If false, the operation will fail if a file already exists. + Boolean OverwriteOutputFilesIfExists; + }; + + /// Options for unpacking a package + [contract(MakeMSIXContract, 1)] + runtimeclass ExtractBundleOptions + { + /// Created by callers to pass options into ExtractBundle() + ExtractBundleOptions(); + + /// If true, the operation will overwrite existing files in the output path + /// when unpacking. + /// If false, the operation will fail if a file already exists. + Boolean OverwriteOutputFilesIfExists; + }; + + /// + [contract(MakeMSIXContract, 1)] + runtimeclass CreateMountableImageOptions + { + /// Created by callers to pass options into CreateMountableImage() + CreateMountableImageOptions(); + + /// Root directory path for packages inside the image. + /// Defaults to "WindowsApps" + String PackageRootDirectoryInImage; + + /// Overwrite output file if it already exists. + /// Defaults to true. + Boolean OverwriteOutputFileIfExists; + }; + + runtimeclass PackageIdentity + { + /// Name of the package + /// Defaults to empty. + String Name; + + /// Publisher of the package + /// Defaults to empty. + String Publisher; + + /// FamilyName of the package + /// Defaults to empty. + String FamilyName; + + /// Version of the package + /// Defaults to 0.0.0.0. + Windows.ApplicationModel.PackageVersion Version { get; set; }; + + /// Architecture of the package + /// Defaults to Unknown. + Windows.System.ProcessorArchitecture Architecture; + + /// ResourceId of the package + /// Defaults to empty. + String ResourceId; + + /// FullName of the package. + /// Defaults to empty. + String FullName; + } + + /// Static methods for creating and unpacking packages. + [contract(MakeMSIXContract, 1)] + runtimeclass MakeMSIXManager + { + /// Creates packages from the inputPath as specified by the packOptions. /// Output path for the full packaged file. - /// - String PackageFilePath{ get; set; }; + static Windows.Foundation.IAsyncAction CreatePackage( + String inputPath, + String outputFileName, + CreatePackageOptions packOptions); + + /// Creates bundles from the inputPath as specified by the bundleOptions. + /// Output path for the full bundled file. + static Windows.Foundation.IAsyncAction CreateBundle( + String inputPath, + String outputFileName, + CreateBundleOptions bundleOptions); + + /// Unpacks a package at inputFileName as specified by the extractPackageOptions. + /// The output folder to unpack the package into. + static Windows.Foundation.IAsyncAction ExtractPackage( + String inputFileName, + String outputPath, + ExtractPackageOptions extractPackageOptions); + + /// Unbundles a package at inputFileName as specified by the extractBundleOptions. + /// The output folder to unpack the bundle package into. + static Windows.Foundation.IAsyncAction ExtractBundle( + String inputFileName, + String outputPath, + ExtractBundleOptions extractBundleOptions); + + /// Creates a Kozani package from an existing package. + /// inputFileName can be an appx, appxbundle, msix, or msixbundle + /// Output path for the kozani packaged file. + /// Format must match input file. If inputFileName is an msix, then outputFileName must be an msix. + /// If inputFileName is an msixbundle, then outputFileName must be an msixbundle. + static Windows.Foundation.IAsyncAction CreateKozani( + String inputFileName, + String outputFileName, + CreateKozaniOptions createKozaniOptions); + + /// Creates a mountable image that contains the packages at inputFileNames. + /// Valid file extensions for outputFileName are vhd, vhdx, and CIM. + /// Can be used for Azure App Attach. + static Windows.Foundation.IAsyncAction CreateMountableImage( + Windows.Foundation.Collections.IVector inputFileNames, + String outputFileName, + CreateMountableImageOptions createMountableImageOptions); + + /// Gets the identity information about a package + static Windows.Foundation.IAsyncOperation GetPackageIdentity(String packageFileName); + }; +} + + +``` + +# 5. Appendix + +# 6. Existing interfaces and options + +The existing tools are: +- [Makeappx.exe](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) + Shipped in Windows' Platform SDK. + Exposes pack, unpack, bundle, and unbundle commands, as well as encryption support + via a custom key-file list file, and creation of packages via a custom mapping file + to allow creation of packages with different payloads from the same folder. + +- [Makemsix.exe](https://github.com/Microsoft/msix-packaging) Available via the msix-packaging + project. Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command only supports + creation of + [flat bundles](https://learn.microsoft.com/en-us/windows/msix/package/flat-bundles). Flat + bundles are bundles where package locations are stored as external path references rather than + being stored inside the msixbundle file itself. Flat bundles are referred to as sparse bundles + in the makeappx documentation). + +- [Msixmgr.exe](https://github.com/Microsoft/msix-packaging) Available via the msix-packaging + project and + [direct download from learn.microsoft.com](https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr). + Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. + +None of the existing tools provide a programmatic interface for all aspects of packaging. + +Makeappx.exe provides the only complete implementation of packaging and bundling but requires +process creation and does not return any specific errors from process exit. It also does not support +any of the Azure app attach scenarios. + +Makemsix.exe is a cross-platform implementation of packaging, but does not support a complete set of +bundling features. The functionality is exposed as a command line as well as flat dll exports. + +Msixmgr.exe is mostly for specific Azure app attach packaging scenarios not covered by the other +tools. To use it programmatically would require either process creation or building the +msix-packaging github project and linking the static lib it produces into a project. + +## 6.1. Makeappx + +### 6.1.1. Pack +Usage: +------ + MakeAppx pack [options] /d /p + MakeAppx pack [options] /f /p + MakeAppx pack [options] /m /f /p + MakeAppx pack [options] /r /m /f /p + MakeAppx pack [options] /d /ep /kf + MakeAppx pack [options] /d /ep /kt + +Options: +-------- + /h, /hashAlgorithm: Specifies a hash algorithm to use for creating the + block map. Valid algorithm IDs are SHA256, SHA384, and SHA512. The + default is SHA256. + /nv, /noValidation: Skips validation that ensures the package will be + installable on Windows. The validation include: existence of files + referenced in manifest, ContentGroupMap correctness, and additional + manifest validation on Protocols and FileTypeAssociation. By default, + all semantic validation is performed. + /nfv, /noFileValidation: Skips validation that ensures that the files used + to create the package (from a folder through /d or layout file through + /l) exist and are accessible. By default, all files must exist and not + be read locked. If a file is inaccessible, it will not be added to the + package. + /nc, /noCompress: Prevents MakeAppx from compressing files in the package. + By default, files in the package are compressed based on detected file + type. + /m: Specifies the path to an input app manifest which will be used as the + basis for generating the output app package or resource package's + manifest. When you use this option, you must also use /f and include a + [ResourceMetadata] section in the mapping file to specify the resource + dimensions to be included in the generated manifest. + /r: Builds a resource package. You must use the /m option with this. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /cgm: The input content group map (CGM) file path used to create a + stream-able package. Providing a content group map through this option + will disable the check that all files in the content group map exist in + the package. + /pri, /makepriExeFullPath: You can use /pri to override the default + MakePri.exe path with the custom fullpath from which makeappx.exe will + launch the tool from when needed. + /mp: Specifies the path to a main package in the same package family as the + resource package being built. You should provide this when encrypting + a resource package or when the resource package contains a content + group map. + /pb, /publisherBridging: Use this option to add publisher bridging entries + to the package or bundle. Publisher bridging is useful when the new + issued cert subject name changed but the old publisher name is still + desired to maintain package identity continuity. + /np, /noParallel: Use this option to disable parallel execution of this + command. + /ml, /memoryLimit: Specify maximum memory in bytes that MakeAppx should + consume while executing in parallel. By default, this value is set to + half of total physical memory. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. - /// - /// Overwrite PackageFilePath if it already exists. - /// Defaults to true. - /// - Boolean OverwriteFiles{ get; set; }; - /// - /// Validates elements of the package during creation. - /// Defaults to false. - /// - Boolean ValidateFiles{ get; set; }; - }; +### 6.1.2. Bundle - /// Options for creating a Kozani package - [contract(MakeMSIXContract, 1)] - runtimeclass CreateKozaniPackageOptions - { - /// - /// - /// - CreateKozaniPackageOptions(); +Usage: +------ + MakeAppx bundle [options] /d /p + MakeAppx bundle [options] /f /p + MakeAppx bundle [options] /d /ep /kf MyKeyFile.txt + MakeAppx bundle [options] /f /ep /kt - /// - /// Output path for the kozani packaged file. - /// Format must match input file. If packageFilePathToConvert is an appx or msix, then PackageFilePath must be an appx or msix. - /// If packageFilePathToConvert is an appxbundle or msixbundle, then PackageFilePath must be an appxbundle or msixbundle. - /// - String PackageFilePath{ get; set; }; +Options: +-------- + /bv: Specifies the version number of the bundle being created. The version + must be in dotted-quad notation of four integers + ... ranging from 0 to 65535 each. If the + /bv option is not specified or is set to 0.0.0.0, the bundle is created + using the current date-time formatted as the version: + .... + /mo: Generates a bundle manifest only, instead of a full bundle. Input + files must all be package manifests in XML format if this option is + specified. + /fb: Generates a fully [sparse bundle](https://learn.microsoft.com/en-us/windows/msix/package/flat-bundles) + where all packages are references to packages that exist outside of + the bundle file + /pri, /makepriExeFullPath: You can use /pri to override the default + MakePri.exe path with the custom fullpath from which makeappx.exe will + launch the tool from when needed. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. - /// - /// Optional replacement package publisher. - /// If not set, publisher from appxmanifest.xml is unchaged. - /// - String PackagePublisher{ get; set; }; +### 6.1.3. Unpack +Usage: +------ + MakeAppx unpack [options] /p /d + MakeAppx unpack [options] /ep /d /kf + MakeAppx unpack [options] /ep /d /kt - /// - /// Optional replacement package name. - /// If not set, name from appxmanifest.xml is unchaged. - /// - String PackageName{ get; set; }; +Options: +-------- + /pfn: Unpacks all files to a subdirectory under the specified output path, + named after the package full name. + /nv, /noValidation: Skips validation that ensures the package will be + installable on Windows. The validation include: existence of files + referenced in manifest, ContentGroupMap correctness, and additional + manifest validation on Protocols and FileTypeAssociation. By default, + all semantic validation is performed. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /nd: Skips decryption when unpacking an encrypted package or bundle. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. - /// - /// Overwrite PackageFilePath if it already exists. - /// Defaults to true. - /// - Boolean OverwriteFiles{ get; set; }; +### 6.1.4. Unbundle - /// - /// Validates elements of the package during creation. - /// Defaults to false. - /// - Boolean ValidateFiles{ get; set; }; - }; +Usage: +------ + MakeAppx unbundle [options] /p /d + MakeAppx unbundle [options] /ep /d /kf + MakeAppx unbundle [options] /ep /d /kt - /// Options for creating a bundle - [contract(MakeMSIXContract, 1)] - runtimeclass BundleOptions - { - /// - /// Created by callers to pass options into Bundle - /// - BundleOptions(); +Options: +-------- + /pfn: Unpacks all files to a subdirectory under the specified output path, + named after the package full name. + /kf: Use this option to encrypt or decrypt the package or bundle using a + key file. This option cannot be combined with /kt. + /kt: Use this option to encrypt or decrypt the package or bundle using the + global test key. This option cannot be combined with /kf. + /nd: Skips decryption when unpacking an encrypted package or bundle. + /o, /overwrite: Forces the output to overwrite any existing files with the + same name. By default, the user is asked whether to overwrite existing + files with the same name. You can't use this option with /no. + /no, /noOverwrite: Prevents the output from overwriting any existing files + with the same name. By default, the user is asked whether to overwrite + existing files with the same name. You can't use this option with /o. + /v, /verbose: Enables verbose output of messages to the console. + /?, /help: Displays this help text. - /// - /// Output path for the full bundled file. - /// - String BundleFilePath{ get; set; }; +## 6.2. MakeMsix - /// - /// Overwrite BundleFilePath if they already exist. - /// Defaults to true. - /// - Boolean OverwriteFiles{ get; set; }; +MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for +encryption, content groups, mapping files, and bundle support is limited to +only allowing creation of flat bundles (bundles where the appxbundle file itself +only references but does not contain the main and resource appxpackages) - /// - /// Create as a flat bundle. Package locations will be stored as external path references - /// rather than being stored inside the msixbundle file itself. - /// Defaults to false. - /// - Boolean FlatBundle{ get; set; }; +It does add the ability to optionally unpack all packages inside the bundle +in one command using the -pfn-flat option, as in: +makemsix.exe unbundle -p -d -pfn-flat - /// - /// Optional. Sets the version in the AppxBundleManifest.xml - /// If not set the version is created based on the current time. - /// - Windows.ApplicationModel.PackageVersion BundleVersion{ get; set; }; - }; +There is, however, no good option for reversing this command when recreating the +packages and bundle. Callers need to call makemsix pack on each individual +package folder and then follow it with a makemsix bundle call to generate a +flat bundle. - /// - /// Options for unpacking a package - /// - [contract(MakeMSIXContract, 1)] - runtimeclass UnpackOptions - { - /// - /// Created by callers to pass options into UnpackPackage - /// - UnpackOptions(); +### 6.2.1. Pack +Usage: +--------------- + makemsix.exe pack -d -p [options] - /// - /// The output folder to unpack the package into. - /// - String UnpackedPackageRootDirectory{ get; set; }; - /// - /// If true, the operation will overwrite existing files in the UnpackedPackageRootDirectory - /// when unpacking. - /// If false, the operation will fail if a file already exists. - /// - Boolean OverwriteFiles{ get; set; }; - }; +Options: +--------------- + -d {Required} + Input directory path. + -p {Required} + Output package file path. + -? [Flag] + Displays this help text. - /// - /// Options for unpacking a package - /// - [contract(MakeMSIXContract, 1)] - runtimeclass UnbundleOptions - { - /// - /// Created by callers to pass options into UnpackPackage - /// - UnbundleOptions(); +### 6.2.2. Bundle +Usage: +--------------- + makemsix.exe bundle -p [options] - /// - /// The output folder to unpack the bundle package into. - /// - String UnbundledPackageRootDirectory{ get; set; }; - /// - /// If true, the operation will overwrite existing files in the UnbundledPackageRootDirectory - /// when unpacking. - /// If false, the operation will fail if a file already exists. - /// - Boolean OverwriteFiles{ get; set; }; - }; +Options: +--------------- + -d + Input directory path. + -p {Required} + Output bundle file path. + -f + Mapping file path. + -bv + Specifies the version number of the bundle being created. The version must be in dotted - quad + notation of four integers ... ranging from 0 to 65535 each. If + the -bv option is not specified or is set to 0.0.0.0, the bundle is created using the current + date - time formatted as the version: .... + -mo [Flag] + Generates a bundle manifest only, instead of a full bundle. Input files must all be package + manifests in XML format if this option is specified. + -fb [Flag] + Generates a fully sparse bundle where all packages are references to packages that exist + outside of the bundle file. + -o [Flag] + Forces the output to overwrite any existing files with the same name.By default, the user is + asked whether to overwrite existing files with the same name.You can't use this option with -no. + -no [Flag] + Prevents the output from overwriting any existing files with the same name. By default, the + user is asked whether to overwrite existing files with the same name.You can't use this option + with -o. + -v [Flag] + Enables verbose output of messages to the console. + -? [Flag] + Displays this help text. - /// - /// - /// - [contract(MakeMSIXContract, 1)] - runtimeclass CreateMountableImageOptions - { - /// - /// Created by callers to pass options into CreateMountableImage - /// - CreateMountableImageOptions(); +### 6.2.3. Unpack +Usage: +--------------- + makemsix.exe unpack -p -d [options] - /// - /// Output path for the image file. - /// Valid file extensions are vhd, vhdx, and CIM. - /// - String ImageFilePath{ get; set; }; +Options: +--------------- + -p {Required} + Input package file path. + -d {Required} + Output directory path. + -pfn [Flag] + Unpacks all files to a subdirectory under the output path, named after the package full name. + -ac [Flag] + Allows any certificate. By default the signature origin must be known. + -ss [Flag] + Skips enforcement of signed packages. By default packages must be signed. + -pfn-flat [Flag] + Same behavior as -pfn for packages. + -? [Flag] + Displays this help text. - /// - /// Root directory path for packages inside the image. - /// Defaults to "WindowsApps" - /// - String PackageRootDirectoryInImage{ get; set; }; +### 6.2.4. Unbundle - /// - /// If true, the operation will overwrite an existing file in the ImageFilePath - /// when unpacking. - /// If false, the operation will fail if the file already exists. - /// - Boolean OverwriteFiles{ get; set; }; - }; +Usage: +--------------- + makemsix.exe unbundle -p -d [options] - /// - /// Static methods for creating and unpacking packages. - /// - [contract(MakeMSIXContract, 1)] - runtimeclass MakeMSIXManager - { - /// - /// Creates packages from the directoryPathToPack as specified by the packOptions. - /// - static Windows.Foundation.IAsyncAction Pack(String directoryPathToPack, PackOptions packOptions); - /// - /// Creates bundles from the directoryPathToBundle as specified by the bundleOptions. - /// - static Windows.Foundation.IAsyncAction Bundle(String directoryPathToBundle, BundleOptions bundleOptions); - /// - /// Unpacks a package at packageFilePathToUnpack as specified by the unpackOptions. - /// - static Windows.Foundation.IAsyncAction Unpack(String packageFilePathToUnpack, UnpackOptions unpackOptions); - /// - /// Unbundles a package at bundleFilePathToUnbundle as specified by the unbundleOptions. - /// - static Windows.Foundation.IAsyncAction Unbundle( - String bundleFilePathToUnbundle, - UnbundleOptions unbundleOptions); - - /// - /// Creates a Kozani package from an existing package. - /// packageFilePathToConvert can be an appx, appxbundle, msix, or msixbundle - /// - static Windows.Foundation.IAsyncAction CreateKozaniPackage( - String packageFilePathToConvert, - CreateKozaniPackageOptions createKozaniPackageOptions); - - /// - /// Creates a mountable image that contains the packages at packageFilePathsToAdd. - /// Can be used for Azure App Attach. - /// - static Windows.Foundation.IAsyncAction CreateMountableImage( - Windows.Foundation.Collections.IVector packageFilePathsToAdd, - CreateMountableImageOptions createMountableImageOptions); - }; -} +Options: +--------------- + -p {Required} + Input bundle file path. + -d {Required} + Output directory path. + -pfn [Flag] + Unpacks all files to a subdirectory under the output path, named after the package full name. + -ac [Flag] + Allows any certificate. By default the signature origin must be known. + -ss [Flag] + Skips enforcement of signed packages. By default packages must be signed. + -sl [Flag] + Skips matching packages with the language of the system. By default unpacked resources + packages will match the system languages. + -sp [Flag] + Skips matching packages with of the same system. By default unpacked application packages + will only match the platform. + -extract-all [Flag] + Extracts all packages from the bundle. + -pfn-flat [Flag] + Unpacks bundle's files to a subdirectory under the specified output path, named after the + package full name. Unpacks packages to subdirectories also under the specified output path, + named after the package full name. By default unpacked packages will be nested inside the bundle folder. + -? [Flag] + Displays this help text. +## 6.3. MSIXMgr -``` +### 6.3.1. Unpack +msixmgr.exe -Unpack -packagePath -destination [-applyacls] [-create] [-vhdSize ] [-filetype ] [-rootDirectory ] -# Appendix +-Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. + -applyACLs: optional parameter that applies ACLs to the resulting package folder(s) and + their parent folder + -create: optional parameter that creates a new image with the specified -filetype and + unpacks the packages to that image + -destination: the directory to place the resulting package folder(s) in + -fileType: the type of file to unpack packages to. Valid file types include {VHD, VHDX, CIM}. + This is a required parameter when unpacking to CIM files + -packagePath: the path to the package to unpack OR the path to a directory containing + multiple packages to unpack + -rootDirectory: root directory on an image to unpack packages to. Required parameter + for unpacking to new and existing CIM files + -validateSignature: optional parameter that validates a package's signature file before + unpacking the package. This will require that the package's certificate is installed on the machine. + -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. + Use only for VHD or VHDX files \ No newline at end of file From 8fcb5450586830668cd9579d70539931b94c7786 Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Fri, 2 Jun 2023 15:17:54 -0700 Subject: [PATCH 5/8] Update with API changes for resources. --- specs/WinRT/MakeMSIX-spec.md | 99 ++++++++++++++++++++++++------------ 1 file changed, 66 insertions(+), 33 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index ac700ee331..5017c407f3 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -110,7 +110,7 @@ it can be done as a separate operation from creation of an image file. std::wstring developerCertPfxFile{ L"D:\\test\\ContosoApp1\\developerCert.pfx" }; // Step 1. Create the full bundle. - + PackageIdentity packageIdentity = nullptr; // Iterate through package layout folders and pack each one. for (const std::filesystem::directory_entry& directoryEntry : std::filesystem::directory_iterator(packageLayoutRootPath)) { @@ -118,18 +118,32 @@ it can be done as a separate operation from creation of an image file. { continue; } + CreatePackageOptions createPackageOptions = CreatePackageOptions(); createPackageOptions.OverwriteOutputFileIfExists(true); std::wstring packageFolderName = directoryEntry.path().filename().wstring(); std::wstring packageOutputFilePath{ packageOutputRootDirectoryPath + packageFolderName + L".appx" }; MakeMSIXManager::CreatePackage(directoryEntry.path().c_str(), packageOutputFilePath.c_str(), createPackageOptions).get(); + + // All packages in the bundle have the same version in this example, grab one of them to set on the bundle. + if (packageIdentity == nullptr) + { + packageIdentity = MakeMSIXManager::GetPackageInformation(directoryEntry.path().c_str()).get().Identity(); + } } + + // Confirm the directory did contain package layouts. + if (packageIdentity == nullptr) + { + winrt::throw_hresult(E_INVALIDARG); + } + // Bundle all the packages together. CreateBundleOptions createbundleOptions = CreateBundleOptions(); createbundleOptions.OverwriteOutputFileIfExists(true); createbundleOptions.FlatBundle(true); - createbundleOptions.Version(winrt::Windows::ApplicationModel::PackageVersion(1, 1, 0, 0)); + createbundleOptions.Version(packageIdentity.Version()); MakeMSIXManager::CreateBundle(packageOutputRootDirectoryPath, bundleOutputFilePath.c_str(), createbundleOptions).get(); // Sign the bundle @@ -139,6 +153,10 @@ it can be done as a separate operation from creation of an image file. CreateKozaniPackageOptions createKozaniPackageOptions = CreateKozaniPackageOptions(); createKozaniPackageOptions.OverwriteOutputFileIfExists(true); createKozaniPackageOptions.RemoveExtensions(true); + // Append "Kozani" to the package name, and trim non-english languages from the package. + std::wstring newPackageName{ packageIdentity.Name() + L"Kozani" }; + createKozaniPackageOptions.Name(newPackageName); + createKozaniPackageOptions.Languages().Append(L"en-US"); MakeMSIXManager::CreateKozaniPackage(bundleOutputFilePath, kozaniPackageOutputFilePath.c_str(), createKozaniPackageOptions).get(); // Step 3. Create app attach vhd from the bundle created in Step 1. @@ -403,15 +421,15 @@ namespace Microsoft.Kozani.MakeMSIX CreatePackageOptions(); /// Optional replacement package name. - /// If field is empty, values from AppxManifest.xml are unchanged. + /// If field is empty, value from AppxManifest.xml is unchanged. String Name; /// Optional replacement package publisher. - /// If field is empty, values from AppxManifest.xml are unchanged. + /// If field is empty, value from AppxManifest.xml is unchanged. String Publisher; /// Optional replacement package version - /// If version is 0, version is created based on the current time. + /// If version is 0, value from AppxManifest.xml is unchanged. Windows.ApplicationModel.PackageVersion Version; /// Overwrite output file if it already exists. @@ -435,17 +453,25 @@ namespace Microsoft.Kozani.MakeMSIX Boolean RemoveExtensions; /// Optional replacement package name. - /// If field is empty, values from AppxManifest.xml are unchanged. + /// If field is empty, value from AppxManifest.xml is unchanged. String Name; /// Optional replacement package publisher. - /// If field is empty, values from AppxManifest.xml are unchanged. + /// If field is empty, value from AppxManifest.xml is unchanged. String Publisher; /// Optional replacement package version - /// If version is 0, version is created based on the current time. + /// If version is 0, value from AppxManifest.xml is unchanged. Windows.ApplicationModel.PackageVersion Version; + /// Languages supported by the output package + /// If field is empty, all languages from the original package are maintained. + Windows.Foundation.Collections.IVector Languages{ get; }; + + /// ScaleFactors supported by the output package + /// If field is empty, all scale factors from the original package are maintained. + Windows.Foundation.Collections.IVector ScaleFactors{ get; }; + /// Overwrite output file if it already exists. /// Defaults to true. Boolean OverwriteOutputFileIfExists; @@ -468,7 +494,7 @@ namespace Microsoft.Kozani.MakeMSIX Boolean FlatBundle; /// Optional. Sets the version in the AppxBundleManifest.xml - /// If not set the version is created based on the current time. + /// If version is 0, the version is created based on the current time. Windows.ApplicationModel.PackageVersion Version; /// Overwrite output file if it already exists. @@ -502,7 +528,7 @@ namespace Microsoft.Kozani.MakeMSIX Boolean OverwriteOutputFilesIfExists; }; - /// + /// Options for creating a mountable image [contract(MakeMSIXContract, 1)] runtimeclass CreateMountableImageOptions { @@ -518,54 +544,61 @@ namespace Microsoft.Kozani.MakeMSIX Boolean OverwriteOutputFileIfExists; }; + /// Identity of a package runtimeclass PackageIdentity { /// Name of the package - /// Defaults to empty. - String Name; + String Name{ get; }; /// Publisher of the package - /// Defaults to empty. - String Publisher; + String Publisher{ get; }; /// FamilyName of the package - /// Defaults to empty. - String FamilyName; + String FamilyName{ get; }; /// Version of the package - /// Defaults to 0.0.0.0. - Windows.ApplicationModel.PackageVersion Version { get; set; }; + Windows.ApplicationModel.PackageVersion Version { get; }; /// Architecture of the package - /// Defaults to Unknown. - Windows.System.ProcessorArchitecture Architecture; + Windows.System.ProcessorArchitecture Architecture{ get; }; /// ResourceId of the package - /// Defaults to empty. - String ResourceId; + String ResourceId{ get; }; /// FullName of the package. - /// Defaults to empty. - String FullName; + String FullName{ get; }; } - + + /// Information about the contents of a package + runtimeclass PackageInformation + { + /// Identity of the package + PackageIdentity Identity{ get; }; + + /// Languages supported by the package + Windows.Foundation.Collections.IVectorView Languages{ get; }; + + /// ScaleFactors supported by the package + Windows.Foundation.Collections.IVectorView ScaleFactors{ get; }; + } + /// Static methods for creating and unpacking packages. [contract(MakeMSIXContract, 1)] runtimeclass MakeMSIXManager { - /// Creates packages from the inputPath as specified by the packOptions. + /// Creates packages from the inputPath as specified by the createPackageOptions. /// Output path for the full packaged file. static Windows.Foundation.IAsyncAction CreatePackage( String inputPath, String outputFileName, - CreatePackageOptions packOptions); + CreatePackageOptions createPackageOptions); - /// Creates bundles from the inputPath as specified by the bundleOptions. + /// Creates bundles from the inputPath as specified by the createBundleOptions. /// Output path for the full bundled file. static Windows.Foundation.IAsyncAction CreateBundle( String inputPath, String outputFileName, - CreateBundleOptions bundleOptions); + CreateBundleOptions createBundleOptions); /// Unpacks a package at inputFileName as specified by the extractPackageOptions. /// The output folder to unpack the package into. @@ -586,10 +619,10 @@ namespace Microsoft.Kozani.MakeMSIX /// Output path for the kozani packaged file. /// Format must match input file. If inputFileName is an msix, then outputFileName must be an msix. /// If inputFileName is an msixbundle, then outputFileName must be an msixbundle. - static Windows.Foundation.IAsyncAction CreateKozani( + static Windows.Foundation.IAsyncAction CreateKozaniPackage( String inputFileName, String outputFileName, - CreateKozaniOptions createKozaniOptions); + CreateKozaniPackageOptions createKozaniPackageOptions); /// Creates a mountable image that contains the packages at inputFileNames. /// Valid file extensions for outputFileName are vhd, vhdx, and CIM. @@ -599,8 +632,8 @@ namespace Microsoft.Kozani.MakeMSIX String outputFileName, CreateMountableImageOptions createMountableImageOptions); - /// Gets the identity information about a package - static Windows.Foundation.IAsyncOperation GetPackageIdentity(String packageFileName); + /// Gets information about the contents of a package + static Windows.Foundation.IAsyncOperation GetPackageInformation(String packageFileName); }; } From 2c74fbe2a66ccf10d308d205a82f9320bffc9b6d Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Tue, 26 Sep 2023 13:09:34 -0700 Subject: [PATCH 6/8] API spec update --- specs/WinRT/MakeMSIX-spec.md | 293 ++++++++++++++++++----------------- 1 file changed, 155 insertions(+), 138 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index 5017c407f3..a9997d6660 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -1,87 +1,46 @@ # MakeMSIX API in Windows App SDK - [1. Background](#1-background) -- [2. Summary of existing options in relation to proposed API](#2-summary-of-existing-options-in-relation-to-proposed-API) -- [3. Examples](#4-examples) - - [3.1. Packing a package](#41-packing-a-package) - - [3.2. Unbundling and unpacking a package to change information](#32-unbundling-and-unpacking-a-package-to-change-information) - - [3.3. Helper methods for examples](#33-helper-methods-for-examples) -- [4. API Details](#4-API-details) -- [5. Appendix](#5-Appendix) -- [6. Existing interfaces and options](#6-existing-interfaces-and-options) - - [6.1. Makeappx](#61-makeappx) - - [6.1.1. pack](#611-pack) - - [6.1.2. bundle](#612-bundle) - - [6.1.3. unpack](#613-unpack) - - [6.1.4. unbundle](#614-unbundle) - - [6.2. Makemsix](#62-makemsix) - - [6.2.1. pack](#621-pack) - - [6.2.2. bundle](#622-bundle) - - [6.2.3. unpack](#623-unpack) - - [6.2.4. unbundle](#624-unbundle) - - [6.3. MSIXMgr](#63-msixmgr) - - [6.3.1. unpack](#631-unpack) +- [2. Examples](#2-examples) + - [2.1. Packing a package](#21-packing-a-package) + - [2.2. Unbundling and unpacking a package to change information](#22-unbundling-and-unpacking-a-package-to-change-information) + - [2.3. Helper methods for examples](#23-helper-methods-for-examples) +- [3. API Details](#3-API-details) +- [4. Appendix](#4-Appendix) +- [5. Existing interfaces and options](#5-existing-interfaces-and-options) + - [5.1. Makeappx](#51-makeappx) + - [5.1.1. pack](#511-pack) + - [5.1.2. bundle](#512-bundle) + - [5.1.3. unpack](#513-unpack) + - [5.1.4. unbundle](#514-unbundle) + - [5.2. Makemsix](#52-makemsix) + - [5.2.1. pack](#521-pack) + - [5.2.2. bundle](#522-bundle) + - [5.2.3. unpack](#523-unpack) + - [5.2.4. unbundle](#524-unbundle) + - [5.3. MSIXMgr](#53-msixmgr) + - [5.3.1. unpack](#531-unpack) +- [6. Summary of existing options in relation to proposed API](#6-summary-of-existing-options-in-relation-to-proposed-API) # 1. Background -The MakeMSIX API allows developers to create app packages for distribution. The API provides a WinRT -interface that joins the functionality from multiple existing command line tools in one developer -friendly location to allow creation of msix\appx packages and msixbundle\appxbundle packages, -unpacking of those packages, and creation of package images for Azure App Attach. In addition to -functionality from those existing tools, this API adds support for creation of Kozani packages. +The MakeMSIX API allows developers to create app packages for distribution. +The API provides a WinRT interface that supports: +1. Creation of msix packages and msixbundle bundle packages, +2. Unpacking and unbundling of packages +3. Creation of vhd\vdx\cim images for Azure App Attach. +4. Creation of Kozani packages. -# 2. Summary of existing options in relation to proposed API +Creation of Kozani packages is a new feature that has not existed in other msix packaging tools. The +other features have previously been available in command line tools discussed in the appendix. -Existing options all treat pack and bundle as separate commands. Though they are similar, each -command does have some options that are irrelevant to the other command. Options that are relevant -to bundling but not packaging are the Bundle Version, which nearly all callers will specify (the -alternative is the tool setting the version based on a timestamp), and allowing creation of flat -bundles. Options that are relevant for packaging but not bundling are hash algorithm, and creation -of streamable packages and resource packages. - -In the long term it's likely that these differences in the information and flags required to support -creating a bundle and creating a package will only continue to grow. +Kozani packages are msix packages that have been optimized for size by removing unnecessary +resources. All existing tools that work with msix packages are compatible with Kozani packages. +Kozani packages can be created manually without this API by modifying the files, manifest, and +resource index of an existing package using existing win32 apis. -It is less likely that differences will arise between unbundle and unpack. However, as seen with the -existing makemsix tool, as long as pack and bundle are separate operations it may not be especially -useful for unbundle and unpack to be joined. Unpacking of all bundled packages makes sense for a -command line tool where an IT admin may be typing in individual commands each time and so there's a -good reason to avoid having to run multiple commands. In cases where the API is actually being used -programattically, to do something like extract all packages and update every manifest with a -different version number, there is no avoiding use of filesystem APIs to find each unpacked package. -In such a case merging the unbundle and unpack command into one API just adds hidden complexity into -the API in the form of either requiring more configuration options or hiding decisions about default -choices like folder naming decisions in the API documentation. +# 2. Examples -For these reasons pack, bundle, unpack, unbundle have been left as separate commands. - -MSIXMgr defines an unpack command which has fairly different usage from the makeappx unpack command. -The msixmgr unpack command merges creation of an image file and adding packages to that image file -into one command. The msixmgr command line supports adding all packages in a folder to an image at -once. The CIM file format it supports always expands to fit those apps, but msixmgr does not -currently support expandable vhdx files. This means that the caller needs to know up front the size -of all unpacked packages when creating vhds, which will be difficult to know without either -unpacking them first or reading each package's block map to get file sizes. This API proposes -getting rid of that requirement and having the API implementation itself determine the size of all -packages being added in order to create an image of the correct size. This behavior is more -consistent with other types of package creation and eliminates the API differences between CIM files -and VHD\VHDX files. - -As the API supports creation of .cim, .vhd, and .vhdx file as well as .msix and .msixbundle files, -callers may find it strange to need to first pack their directory to an msix, and then create the -vhdx from the msix, rather than just creating the vhdx directly. However, signing is not integrated -into any of these tools, and is a separate step that is done after packaging is complete. The Appx -SIP knows how to sign packages, but not vhdx and other mounted directories. The result is that -creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, -then signing it, then creating the vhdx. Allowing creation of a cim, vhd, or vhdx file directly from -a directory seems likely to cause confusion for developers who may not realize that they won't be -able to sign the package in the image. This API always applies ACLs for the package -to the folder, and never validates the signature. If signature validation of a package (checking -that the root cert of a package is trusted on the current machine) is an important API for callers -it can be done as a separate operation from creation of an image file. - -# 3. Examples - -## 3.1. Packing a package +## 2.1. Packing a package ```cpp void CreatePackagesFromFolderExample() @@ -93,7 +52,7 @@ it can be done as a separate operation from creation of an image file. // Initial Conditions: The build has already produced a directory that contains the exact layout of each individual package // that it wants to bundle. // Each folder has an appxmanifest.xml in the root directory. - + // A full non-relative path to those layout folder is needed, as below. // Layout directories are inside this root, for example: packageLayoutRootPath\\ContosoApp1_2023.302.1739.686_x64__8wekyb3d8bbwe. std::wstring packageLayoutRootPath{ L"D:\\test\\ContosoApp1\\BuildOutput\\PackageLayouts" }; @@ -103,9 +62,9 @@ it can be done as a separate operation from creation of an image file. // The build tool wants to create the full bundle at this location std::wstring bundleOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle" }; // The build tool wants to create the kozani package at this location - std::wstring kozaniPackageOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\kozaniPackagedOutput.msix" }; + std::wstring kozaniPackageOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\kozaniPackagedOutput.msixbundle" }; // The build tool wants to create the app attach image at this location - std::wstring appAttachImageFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\appAttachOutput.vhdx" }; + std::wstring appAttachImageFilePath{ L"D:\\xample\\ContosoApp1\\BuildOutput\\Package\\appAttachOutput.vhdx" }; // The certificate to sign the package is at std::wstring developerCertPfxFile{ L"D:\\test\\ContosoApp1\\developerCert.pfx" }; @@ -129,7 +88,7 @@ it can be done as a separate operation from creation of an image file. // All packages in the bundle have the same version in this example, grab one of them to set on the bundle. if (packageIdentity == nullptr) { - packageIdentity = MakeMSIXManager::GetPackageInformation(directoryEntry.path().c_str()).get().Identity(); + packageIdentity = MakeMSIXManager::GetPackageInformation(packageOutputFilePath.c_str()).get().Identity(); } } @@ -162,14 +121,15 @@ it can be done as a separate operation from creation of an image file. // Step 3. Create app attach vhd from the bundle created in Step 1. CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); mountableImageOptions.OverwriteOutputFileIfExists(true); - winrt::Windows::Foundation::Collections::IVector packagesToAddToImage{ winrt::single_threaded_vector() }; + winrt::Windows::Foundation::Collections::IVector packagesToAddToImage{ + winrt::single_threaded_vector() }; packagesToAddToImage.Append(bundleOutputFilePath); MakeMSIXManager::CreateMountableImage(packagesToAddToImage, appAttachImageFilePath.c_str(), mountableImageOptions).get(); } ``` -## 3.2. Unbundling and unpacking a package to change information +## 2.2. Unbundling and unpacking a package to change information ```cpp void ChangeVersionOfAllPackagesInBundle() @@ -179,8 +139,8 @@ it can be done as a separate operation from creation of an image file. { // Scenario: A developer is writing code to update the version of all packages in a bundle. - std::filesystem::path bundleFilePath{ L"D:\\test\\bundle.msixbundle" }; - std::wstring outputDirRoot{ L"D:\\test\\unpackedBundleOutput" }; + std::filesystem::path bundleFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle" }; + std::wstring outputDirRoot{ L"D:\\test\\ContosoApp1\\BuildOutput\\newVersion" }; winrt::Windows::ApplicationModel::PackageVersion newVersion = winrt::Windows::ApplicationModel::PackageVersion(1, 0, 0, 0); std::filesystem::path outputDirForBundle{ outputDirRoot }; @@ -214,11 +174,9 @@ it can be done as a separate operation from creation of an image file. extractPackageOptions.OverwriteOutputFilesIfExists(true); MakeMSIXManager::ExtractPackage(file.path().c_str(), outputDirForPackage.c_str(), extractPackageOptions).get(); } - + std::filesystem::path outputDirRootForPackedPackages{ outputDirRoot }; outputDirRootForPackedPackages /= L"packedPackages"; - std::filesystem::path outputPathForBundle{ outputDirRoot }; - outputPathForBundle /= bundleFilePath.filename(); // Iterate through packages and repack each one. for (const auto& packageDir : std::filesystem::directory_iterator(outputDirRootForPackages)) @@ -234,6 +192,8 @@ it can be done as a separate operation from creation of an image file. MakeMSIXManager::CreatePackage(packageDir.path().c_str(), outputPackagePath.c_str(), createPackageOptions).get(); } + std::filesystem::path outputPathForBundle{ outputDirRoot }; + outputPathForBundle /= bundleFilePath.filename(); // Re-bundle with the new version auto createBundleOptions = CreateBundleOptions(); createBundleOptions.OverwriteOutputFileIfExists(true); @@ -248,7 +208,7 @@ it can be done as a separate operation from creation of an image file. } ``` -## 3.3. Helper methods for examples +## 2.3. Helper methods for examples Windows has existing non-WinRT APIs for signing packages available in the CryptoAPI https://learn.microsoft.com/en-us/windows/win32/seccrypto/using-cryptography @@ -402,7 +362,7 @@ from the Windows SDK are included here. } ``` -# 4. API Details +# 3. API Details ```c# // Copyright (c) Microsoft Corporation and Contributors. @@ -470,7 +430,7 @@ namespace Microsoft.Kozani.MakeMSIX /// ScaleFactors supported by the output package /// If field is empty, all scale factors from the original package are maintained. - Windows.Foundation.Collections.IVector ScaleFactors{ get; }; + Windows.Foundation.Collections.IVector ScaleFactors{ get; }; /// Overwrite output file if it already exists. /// Defaults to true. @@ -553,17 +513,17 @@ namespace Microsoft.Kozani.MakeMSIX /// Publisher of the package String Publisher{ get; }; - /// FamilyName of the package - String FamilyName{ get; }; - /// Version of the package - Windows.ApplicationModel.PackageVersion Version { get; }; + Windows.ApplicationModel.PackageVersion Version{ get; }; /// Architecture of the package Windows.System.ProcessorArchitecture Architecture{ get; }; /// ResourceId of the package String ResourceId{ get; }; + + /// FamilyName of the package + String FamilyName{ get; }; /// FullName of the package. String FullName{ get; }; @@ -579,7 +539,7 @@ namespace Microsoft.Kozani.MakeMSIX Windows.Foundation.Collections.IVectorView Languages{ get; }; /// ScaleFactors supported by the package - Windows.Foundation.Collections.IVectorView ScaleFactors{ get; }; + Windows.Foundation.Collections.IVectorView ScaleFactors{ get; }; } /// Static methods for creating and unpacking packages. @@ -636,20 +596,18 @@ namespace Microsoft.Kozani.MakeMSIX static Windows.Foundation.IAsyncOperation GetPackageInformation(String packageFileName); }; } - - ``` -# 5. Appendix +# 4. Appendix -# 6. Existing interfaces and options +# 5. Existing interfaces and options The existing tools are: - [Makeappx.exe](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) Shipped in Windows' Platform SDK. - Exposes pack, unpack, bundle, and unbundle commands, as well as encryption support - via a custom key-file list file, and creation of packages via a custom mapping file - to allow creation of packages with different payloads from the same folder. + Exposes pack, unpack, bundle, unbundle, encrypt, decrypt and content group mapping. + Makeappx defines and uses its own file formats for encryption key files and content group + mapping to allow creation of packages with different payloads from the same folder. - [Makemsix.exe](https://github.com/Microsoft/msix-packaging) Available via the msix-packaging project. Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command only supports @@ -677,9 +635,9 @@ Msixmgr.exe is mostly for specific Azure app attach packaging scenarios not cove tools. To use it programmatically would require either process creation or building the msix-packaging github project and linking the static lib it produces into a project. -## 6.1. Makeappx +## 5.1. Makeappx -### 6.1.1. Pack +### 5.1.1. Pack Usage: ------ MakeAppx pack [options] /d /p @@ -746,7 +704,7 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -### 6.1.2. Bundle +### 5.1.2. Bundle Usage: ------ @@ -841,23 +799,21 @@ Options: /v, /verbose: Enables verbose output of messages to the console. /?, /help: Displays this help text. -## 6.2. MakeMsix +## 5.2. MakeMsix -MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for -encryption, content groups, mapping files, and bundle support is limited to -only allowing creation of flat bundles (bundles where the appxbundle file itself -only references but does not contain the main and resource appxpackages) +MakeMSIX is generally a subset of MakeAppx's functionality. It lacks support for encryption, content +groups, mapping files, and bundle support is limited to only allowing creation of flat bundles +(bundles where the appxbundle file itself only references but does not contain the main and resource +appxpackages) -It does add the ability to optionally unpack all packages inside the bundle -in one command using the -pfn-flat option, as in: -makemsix.exe unbundle -p -d -pfn-flat +It does add the ability to optionally unpack all packages inside the bundle in one command using the +-pfn-flat option, as in: makemsix.exe unbundle -p -d -pfn-flat -There is, however, no good option for reversing this command when recreating the -packages and bundle. Callers need to call makemsix pack on each individual -package folder and then follow it with a makemsix bundle call to generate a -flat bundle. +There is, however, no good option for reversing this command when recreating the packages and +bundle. Callers need to call makemsix pack on each individual package folder and then follow it with +a makemsix bundle call to generate a flat bundle. -### 6.2.1. Pack +### 5.2.1. Pack Usage: --------------- makemsix.exe pack -d -p [options] @@ -962,24 +918,85 @@ Options: -? [Flag] Displays this help text. -## 6.3. MSIXMgr - -### 6.3.1. Unpack -msixmgr.exe -Unpack -packagePath -destination [-applyacls] [-create] [-vhdSize ] [-filetype ] [-rootDirectory ] - --Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. - -applyACLs: optional parameter that applies ACLs to the resulting package folder(s) and - their parent folder - -create: optional parameter that creates a new image with the specified -filetype and - unpacks the packages to that image - -destination: the directory to place the resulting package folder(s) in - -fileType: the type of file to unpack packages to. Valid file types include {VHD, VHDX, CIM}. - This is a required parameter when unpacking to CIM files - -packagePath: the path to the package to unpack OR the path to a directory containing - multiple packages to unpack - -rootDirectory: root directory on an image to unpack packages to. Required parameter - for unpacking to new and existing CIM files - -validateSignature: optional parameter that validates a package's signature file before - unpacking the package. This will require that the package's certificate is installed on the machine. - -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. - Use only for VHD or VHDX files \ No newline at end of file +## 5.3. MSIXMgr + +### 5.3.1. Unpack + +Usage: +--------------- +msixmgr.exe -Unpack -packagePath -destination [-applyacls] +[-create] [-vhdSize ] [-filetype ] [-rootDirectory ] + +Options: +--------------- + -Unpack: Unpack a package (.appx, .msix, .appxbundle, .msixbundle) and extract its contents to a folder. + -applyACLs: optional parameter that applies ACLs to the resulting package folder(s) and + their parent folder + -create: optional parameter that creates a new image with the specified -filetype and + unpacks the packages to that image + -destination: the directory to place the resulting package folder(s) in + -fileType: the type of file to unpack packages to. Valid file types include {VHD, VHDX, CIM}. + This is a required parameter when unpacking to CIM files + -packagePath: the path to the package to unpack OR the path to a directory containing + multiple packages to unpack + -rootDirectory: root directory on an image to unpack packages to. Required parameter + for unpacking to new and existing CIM files + -validateSignature: optional parameter that validates a package's signature file before + unpacking the package. This will require that the package's certificate is installed on the machine. + -vhdSize: the desired size of the VHD or VHDX file in MB. Must be between 5 and 2040000 MB. + Use only for VHD or VHDX files + +# 6. Summary of existing options in relation to proposed API + +# 6.1 Pack and Bundle +Existing tools all treat pack and bundle as separate commands. Though they are similar, each command +does have some options that are irrelevant to the other command. Options that are relevant to +bundling but not packaging are the Bundle Version, which nearly all callers will specify (the +alternative is the tool setting the version based on a timestamp), and allowing creation of flat +bundles. Options that are relevant for packaging but not bundling are hash algorithm, and creation +of streamable packages and resource packages. + +In the long term it's likely that these differences in the information and flags required to support +creating a bundle and creating a package will only continue to grow. + +# 6.2 Unpack and Unbundle + +Existing tools have generally chosen to have unpack and unbundle as separate commands. + +In cases where the API is being used programattically to do something like extract all packages and +update every manifest with a different version number, developers will have to handle the +differences between bundles and packages when recreating the package so the benefit of merging +unpack and unbundle seems small. + +MSIXMgr defines an unpack command which has different usage from the unpack command of other tools. +The msixmgr unpack command merges creation of an image file and adding packages to that image file +into one command. Since the tool is specifically for creating Azure App Attach images developers are +unlikely to ever need to further interact with the resulting files inside the Azure App Attach +image. For the purposes of this API the unpack command of MSIXMgr seems more similar to a package +conversion like CreateKozaniPackage and so has been renamed to CreateMountableImage rather than +trying to fit all options that are only relevant for "unpacking" to a mountable image into the +unpack command. + +The msixmgr unpack command line supports adding all packages in a folder to an image at once. The +CIM file format it supports always expands to fit those apps, but msixmgr does not currently support +expandable vhdx files. This means that the caller needs to know up front the size of all unpacked +packages when creating vhds, which will be difficult to know without either unpacking them first or +reading each package's block map to get file sizes. This API proposes getting rid of that +requirement and having the API implementation itself determine the size of all packages being added +in order to create an image of the correct size. This behavior is more consistent with other types +of package creation and eliminates the API differences between CIM files and VHD\VHDX files. + +Callers may find it strange to need to first pack their directory to an msix or msixbundle, and then +create the vhdx from the package, rather than just creating the vhdx directly from the directory. +However, signing is not integrated into any of these tools, and is a separate step that is done +after packaging is complete. The Appx SIP knows how to sign packages, but not vhdx and other mounted +directories. The result is that creating a vhdx with an appx signature file (appxsignature.p7x) +requires first creating the package, then signing it, then creating the vhdx. Allowing creation of a +cim, vhd, or vhdx file directly from a directory seems likely to cause confusion for developers who +may not realize that they won't be able to sign the package in the image. + +This API always applies ACLs for the package to the folder, and never validates the signature so +those options in msixmgr have not been exposed. If signature validation of a package (checking that +the root cert of a package is trusted on the current machine) is an important API for callers it can +be done as a separate operation from creation of an image file. This keeps signature concerns +completely outside the scope of this API. From a3d382573e0124227af2b7d217cc1148920b0383 Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Wed, 4 Oct 2023 16:33:28 -0700 Subject: [PATCH 7/8] Change samples to c# --- specs/WinRT/MakeMSIX-spec.md | 490 +++++++++++++++-------------------- 1 file changed, 207 insertions(+), 283 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index a9997d6660..2ad672d241 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -42,170 +42,157 @@ resource index of an existing package using existing win32 apis. ## 2.1. Packing a package -```cpp - void CreatePackagesFromFolderExample() - { - winrt::init_apartment(); - - // Scenario: A developer is creating their own app. A build tool wants to create every type of distributable package for them. - - // Initial Conditions: The build has already produced a directory that contains the exact layout of each individual package - // that it wants to bundle. - // Each folder has an appxmanifest.xml in the root directory. - - // A full non-relative path to those layout folder is needed, as below. - // Layout directories are inside this root, for example: packageLayoutRootPath\\ContosoApp1_2023.302.1739.686_x64__8wekyb3d8bbwe. - std::wstring packageLayoutRootPath{ L"D:\\test\\ContosoApp1\\BuildOutput\\PackageLayouts" }; - - // The build tool packages each folder and puts the resulting packages at this location - std::wstring packageOutputRootDirectoryPath{ L"D:\\test\\ContosoApp1\\BuildOutput\\packagedOutput\\" }; - // The build tool wants to create the full bundle at this location - std::wstring bundleOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle" }; - // The build tool wants to create the kozani package at this location - std::wstring kozaniPackageOutputFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\kozaniPackagedOutput.msixbundle" }; - // The build tool wants to create the app attach image at this location - std::wstring appAttachImageFilePath{ L"D:\\xample\\ContosoApp1\\BuildOutput\\Package\\appAttachOutput.vhdx" }; - // The certificate to sign the package is at - std::wstring developerCertPfxFile{ L"D:\\test\\ContosoApp1\\developerCert.pfx" }; - - // Step 1. Create the full bundle. - PackageIdentity packageIdentity = nullptr; - // Iterate through package layout folders and pack each one. - for (const std::filesystem::directory_entry& directoryEntry : std::filesystem::directory_iterator(packageLayoutRootPath)) - { - if (!directoryEntry.is_directory()) - { - continue; - } - - CreatePackageOptions createPackageOptions = CreatePackageOptions(); - createPackageOptions.OverwriteOutputFileIfExists(true); - std::wstring packageFolderName = directoryEntry.path().filename().wstring(); - std::wstring packageOutputFilePath{ packageOutputRootDirectoryPath + packageFolderName + L".appx" }; - - MakeMSIXManager::CreatePackage(directoryEntry.path().c_str(), packageOutputFilePath.c_str(), createPackageOptions).get(); - - // All packages in the bundle have the same version in this example, grab one of them to set on the bundle. - if (packageIdentity == nullptr) - { - packageIdentity = MakeMSIXManager::GetPackageInformation(packageOutputFilePath.c_str()).get().Identity(); - } - } +```c# +static async Task CreatePackagesFromFolderExample() +{ + // Scenario: A developer is creating their own app. A build tool wants to create every type of distributable package for them. + + // Initial Conditions: The build has already produced a directory that contains the exact layout of each individual package + // that it wants to bundle. + // Each folder has an appxmanifest.xml in the root directory. + + // A full non-relative path to those layout folder is needed, as below. + // Layout directories are inside this root, for example: packageLayoutRootPath\\ContosoApp1_2023.302.1739.686_x64__8wekyb3d8bbwe. + string packageLayoutRootPath = "D:\\test\\ContosoApp1\\BuildOutput\\PackageLayouts"; + // The build tool packages each folder and puts the resulting packages at this location + string packageOutputRootDirectoryPath = "D:\\test\\ContosoApp1\\BuildOutput\\packagedOutput\\"; + // The build tool wants to create the full bundle at this location + string bundleOutputFilePath = "D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle"; + // The build tool wants to create the kozani package at this location + string kozaniPackageOutputFilePath = "D:\\test\\ContosoApp1\\BuildOutput\\Package\\kozaniPackagedOutput.msixbundle"; + // The build tool wants to create the app attach image at this location + string appAttachImageFilePath = "D:\\xample\\ContosoApp1\\BuildOutput\\Package\\appAttachOutput.vhdx"; + // The certificate to sign the package is at + string developerCertPfxFile = "D:\\test\\ContosoApp1\\developerCert.pfx"; + + // Step 1. Create the full bundle. + // Iterate through package layout folders and pack each one. + DirectoryInfo packageLayoutRootDirectory = new DirectoryInfo(packageLayoutRootPath); + var packTasks = packageLayoutRootDirectory.EnumerateDirectories() + .Select(async directory => + { + PackageInformation packageInfo = await MakeMSIXManager.GetPackageInformation(directory.FullName); - // Confirm the directory did contain package layouts. - if (packageIdentity == nullptr) - { - winrt::throw_hresult(E_INVALIDARG); - } + var createPackageOptions = new CreatePackageOptions(); + createPackageOptions.OverwriteOutputFileIfExists = true; + string packageOutputFilePath = packageOutputRootDirectoryPath + "\\" + packageInfo.Identity.FullName + ".appx"; - // Bundle all the packages together. - CreateBundleOptions createbundleOptions = CreateBundleOptions(); - createbundleOptions.OverwriteOutputFileIfExists(true); - createbundleOptions.FlatBundle(true); - createbundleOptions.Version(packageIdentity.Version()); - MakeMSIXManager::CreateBundle(packageOutputRootDirectoryPath, bundleOutputFilePath.c_str(), createbundleOptions).get(); - - // Sign the bundle - Example_SignPackage(developerCertPfxFile, bundleOutputFilePath); - - // Step 2. Create the kozani package from the bundle created in Step 1. - CreateKozaniPackageOptions createKozaniPackageOptions = CreateKozaniPackageOptions(); - createKozaniPackageOptions.OverwriteOutputFileIfExists(true); - createKozaniPackageOptions.RemoveExtensions(true); - // Append "Kozani" to the package name, and trim non-english languages from the package. - std::wstring newPackageName{ packageIdentity.Name() + L"Kozani" }; - createKozaniPackageOptions.Name(newPackageName); - createKozaniPackageOptions.Languages().Append(L"en-US"); - MakeMSIXManager::CreateKozaniPackage(bundleOutputFilePath, kozaniPackageOutputFilePath.c_str(), createKozaniPackageOptions).get(); - - // Step 3. Create app attach vhd from the bundle created in Step 1. - CreateMountableImageOptions mountableImageOptions = CreateMountableImageOptions(); - mountableImageOptions.OverwriteOutputFileIfExists(true); - winrt::Windows::Foundation::Collections::IVector packagesToAddToImage{ - winrt::single_threaded_vector() }; - packagesToAddToImage.Append(bundleOutputFilePath); - - MakeMSIXManager::CreateMountableImage(packagesToAddToImage, appAttachImageFilePath.c_str(), mountableImageOptions).get(); + await MakeMSIXManager.CreatePackage(directory.FullName, packageOutputFilePath, createPackageOptions); + return packageOutputFilePath; + }) + .ToArray(); + await Task.WhenAll(packTasks); + + // Confirm the directory did contain package layouts. + if (packTasks.Length == 0) + { + throw new Exception("No packages found at path: " + packageLayoutRootPath); } + + PackageInformation packageInfo = await MakeMSIXManager.GetPackageInformation(packTasks[0].Result); + + // Bundle all the packages together. + CreateBundleOptions createbundleOptions = new CreateBundleOptions() + { + OverwriteOutputFileIfExists = true, + FlatBundle = true, + Version = packageInfo.Identity.Version, + }; + + await MakeMSIXManager.CreateBundle(packageOutputRootDirectoryPath, bundleOutputFilePath, createbundleOptions); + + // Sign the bundle + await Example_SignPackage(developerCertPfxFile, bundleOutputFilePath); + + // Step 2. Create the kozani package from the bundle created in Step 1. + // Append "Kozani" to the package name, and trim non-english languages from the package. + string newPackageName = packageInfo.Identity.Name + "Kozani"; + CreateKozaniPackageOptions createKozaniPackageOptions = new CreateKozaniPackageOptions() + { + OverwriteOutputFileIfExists = true, + RemoveExtensions = true, + Name = newPackageName, + }; + createKozaniPackageOptions.Languages.Append("en-US"); + await MakeMSIXManager.CreateKozaniPackage(bundleOutputFilePath, kozaniPackageOutputFilePath, createKozaniPackageOptions); + + // Step 3. Create app attach vhd from the bundle created in Step 1. + CreateMountableImageOptions mountableImageOptions = new CreateMountableImageOptions + { + OverwriteOutputFileIfExists = true + }; + + List packagesToAddToImage = new List + { + bundleOutputFilePath + }; + + await MakeMSIXManager.CreateMountableImage(packagesToAddToImage, appAttachImageFilePath, mountableImageOptions); + return; +} ``` ## 2.2. Unbundling and unpacking a package to change information -```cpp - void ChangeVersionOfAllPackagesInBundle() - { - winrt::init_apartment(); - try - { - // Scenario: A developer is writing code to update the version of all packages in a bundle. - - std::filesystem::path bundleFilePath{ L"D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle" }; - std::wstring outputDirRoot{ L"D:\\test\\ContosoApp1\\BuildOutput\\newVersion" }; - winrt::Windows::ApplicationModel::PackageVersion newVersion = winrt::Windows::ApplicationModel::PackageVersion(1, 0, 0, 0); - - std::filesystem::path outputDirForBundle{ outputDirRoot }; - outputDirForBundle /= L"bundle"; - std::filesystem::path outputDirRootForPackages{ outputDirRoot }; - outputDirRootForPackages /= L"unpackedPackages"; - - // Unbundle the bundle to its own folder. After the operation the folder will contain the - // bundled packages as appx\msix packages, as well as the bundle metadata. - auto extractBundleOptions = ExtractBundleOptions(); - extractBundleOptions.OverwriteOutputFilesIfExists(true); - MakeMSIXManager::ExtractBundle(bundleFilePath.c_str(), outputDirForBundle.c_str(), extractBundleOptions).get(); - - // Iterate through the bundled packages and unpack each one. - for (const auto& file : std::filesystem::directory_iterator(outputDirForBundle)) - { - // Skip the metadata files. - std::wstring fileExtension{ file.path().extension() }; - std::transform(fileExtension.begin(), fileExtension.end(), fileExtension.begin(), towlower); - if ((fileExtension.compare(L".appx") != 0) && - (fileExtension.compare(L".msix") != 0)) +```c# +static async Task ChangeVersionOfAllPackagesInBundle() +{ + // Scenario: A developer is writing code to update the version of all packages in a bundle. + + // Sample arguments. + var bundleFilePath = new FileInfo("D:\\test\\ContosoApp1\\BuildOutput\\Package\\bundleOutput.msixbundle"); + var outputDirRoot = new FileInfo("D:\\test\\ContosoApp1\\BuildOutput\\newVersion"); + // Staging directories for the files while unpacking and repacking. + string outputDirForBundle = outputDirRoot + "\\bundle"; + string outputDirRootForPackages = outputDirRoot + "\\unpackedPackages"; + string outputDirForPackedPackages = outputDirRoot + "\\packedPackages"; + + Windows.ApplicationModel.PackageVersion newVersion = new Windows.ApplicationModel.PackageVersion(1, 0, 0, 0); + + // Unbundle the bundle to a folder. After the operation the folder will contain the + // bundled packages as appx\msix packages, as well as the bundle metadata. + var extractBundleOptions = new ExtractBundleOptions(); + extractBundleOptions.OverwriteOutputFilesIfExists = true; + await MakeMSIXManager.ExtractBundle(bundleFilePath.FullName, outputDirForBundle, extractBundleOptions); + + // Iterate through the bundled packages and unpack and repack each one with the changed version. + DirectoryInfo outputBundleDirInfo = new DirectoryInfo(outputDirForBundle); + var extractTasks = outputBundleDirInfo.EnumerateFiles().Where(file => { - continue; - } - - // Name the unpacked folders based on the package file name. - std::filesystem::path outputDirForPackage{ outputDirRootForPackages }; - outputDirForPackage /= file.path().stem(); - - auto extractPackageOptions = ExtractPackageOptions(); - extractPackageOptions.OverwriteOutputFilesIfExists(true); - MakeMSIXManager::ExtractPackage(file.path().c_str(), outputDirForPackage.c_str(), extractPackageOptions).get(); - } - - std::filesystem::path outputDirRootForPackedPackages{ outputDirRoot }; - outputDirRootForPackedPackages /= L"packedPackages"; - - // Iterate through packages and repack each one. - for (const auto& packageDir : std::filesystem::directory_iterator(outputDirRootForPackages)) - { - std::filesystem::path outputPackagePath{ outputDirRootForPackedPackages }; - std::wstring outputPackageFileName = packageDir.path().filename().wstring() + L".appx"; - outputPackagePath /= outputPackageFileName; - - // Change the version when re-packing - auto createPackageOptions = CreatePackageOptions(); - createPackageOptions.OverwriteOutputFileIfExists(true); - createPackageOptions.Version(newVersion); - MakeMSIXManager::CreatePackage(packageDir.path().c_str(), outputPackagePath.c_str(), createPackageOptions).get(); - } - - std::filesystem::path outputPathForBundle{ outputDirRoot }; - outputPathForBundle /= bundleFilePath.filename(); - // Re-bundle with the new version - auto createBundleOptions = CreateBundleOptions(); - createBundleOptions.OverwriteOutputFileIfExists(true); - createBundleOptions.Version(newVersion); - MakeMSIXManager::CreateBundle(outputDirRootForPackedPackages.c_str(), outputPathForBundle.c_str(), createBundleOptions).get(); - } - catch (winrt::hresult_error const& ex) - { - OutputDebugString(ex.message().c_str()); - winrt::check_hresult(ex.code()); - } - } + // Ignore bundle footprint and metadata files in the folder. + return (String.Compare(file.Extension, ".appx", comparisonType: StringComparison.OrdinalIgnoreCase) == 0) || + (String.Compare(file.Extension, ".msix", comparisonType: StringComparison.OrdinalIgnoreCase) == 0); + }) + .Select(async file => + { + // Name the output folder for the unpacked package as the file name of the package + string outputDirForPackage = outputDirRootForPackages + "\\" + file.Name; + + var extractPackageOptions = new ExtractPackageOptions(); + extractPackageOptions.OverwriteOutputFilesIfExists = true; + await MakeMSIXManager.ExtractPackage(file.FullName, outputDirForPackage, extractPackageOptions); + + string outputPackagePath = outputDirForPackedPackages + "\\" + file.Name; + + // Change the version when re-packing + var createPackageOptions = new CreatePackageOptions(); + createPackageOptions.OverwriteOutputFileIfExists = true; + createPackageOptions.Version = newVersion; + await MakeMSIXManager.CreatePackage(outputDirForPackage, outputPackagePath, createPackageOptions); + return; + }) + .ToArray(); + await Task.WhenAll(extractTasks); + + string outputPathForBundle = outputDirRoot + "\\" + bundleFilePath.Name; + // Re-bundle with the new version + var createBundleOptions = new CreateBundleOptions(); + createBundleOptions.OverwriteOutputFileIfExists = true; + createBundleOptions.Version = newVersion; + await MakeMSIXManager.CreateBundle(outputDirForPackedPackages, outputPathForBundle, createBundleOptions); + + return; +} ``` ## 2.3. Helper methods for examples @@ -214,152 +201,89 @@ Windows has existing non-WinRT APIs for signing packages available in the Crypto https://learn.microsoft.com/en-us/windows/win32/seccrypto/using-cryptography For completeness purposes some helper methods to instead find and call signtool.exe from the Windows SDK are included here. -```cpp - /// - /// Create a process and synchronously wait for it to exit. - /// - /// CommandLine argument for CreateProcess - /// Exit code of the process - /// Success if process is created and exit code is returned. - HRESULT CreateProcessAndWaitForExitCode(std::wstring commandLine, DWORD& exitCode) +```c# +/// +/// Create a process and synchronously wait for it to exit. +/// +/// FileName argument for Process.StartInfo +/// Arguments for Process.StartInfo +/// Exit code of the process. +static async Task CreateProcessAndWaitForExitCodeAsync(string fileName, string arguments) +{ + Process process = new Process(); + process.StartInfo.FileName = fileName; + process.StartInfo.Arguments = arguments; + process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden; + if (!process.Start()) { - STARTUPINFO startupInfo; - ZeroMemory(&startupInfo, sizeof(startupInfo)); - startupInfo.cb = sizeof(startupInfo); - - PROCESS_INFORMATION processInformation; - ZeroMemory(&processInformation, sizeof(processInformation)); - - if (!CreateProcess( - nullptr, - const_cast(commandLine.c_str()), - nullptr, - nullptr, - FALSE, - 0, - nullptr, - nullptr, - &startupInfo, - &processInformation - )) - { - return HRESULT_FROM_WIN32(GetLastError()); - } + throw new Exception("Child process failed to start"); + } + await process.WaitForExitAsync(); + return process.ExitCode; +} - DWORD waitResult = WaitForSingleObject(processInformation.hProcess, INFINITE); - switch (waitResult) - { - case WAIT_OBJECT_0: - break; - case WAIT_FAILED: - return HRESULT_FROM_WIN32(GetLastError()); - case WAIT_ABANDONED: - case WAIT_TIMEOUT: - default: - return E_UNEXPECTED; - } - if (!GetExitCodeProcess(processInformation.hProcess, &exitCode)) +static string GetPlatformSDKPath() +{ + using (RegistryKey? sdkKey = Registry.LocalMachine.OpenSubKey("SOFTWARE\\WOW6432Node\\Microsoft\\Microsoft SDKs\\Windows\\v10.0")) + { + string? installPath; + string? productVersion; + if ((sdkKey == null) || + (sdkKey.GetValueKind("InstallationFolder") != RegistryValueKind.String) || + (sdkKey.GetValueKind("ProductVersion") != RegistryValueKind.String) || + string.IsNullOrEmpty(installPath = sdkKey.GetValue("InstallationFolder") as string) || + string.IsNullOrEmpty(productVersion = sdkKey.GetValue("ProductVersion") as string)) { - return HRESULT_FROM_WIN32(GetLastError()); + throw new InvalidOperationException(); } - return S_OK; - } - - std::filesystem::path GetPlatformSDKPath() - { - wil::unique_hkey hKey; - std::wstring sdkRegPath{ LR"(SOFTWARE\WOW6432Node\Microsoft\Microsoft SDKs\Windows\v10.0)" }; - winrt::check_hresult(RegOpenKeyEx(HKEY_LOCAL_MACHINE, sdkRegPath.c_str(), 0, KEY_READ, &hKey)); - - std::wstring installPathKeyName{ L"InstallationFolder" }; - WCHAR installPathBuffer[MAX_PATH]; - DWORD installPathBufferLength = sizeof(installPathBuffer); - winrt::check_hresult(RegGetValueW( - hKey.get(), - nullptr, - installPathKeyName.c_str(), - RRF_RT_REG_SZ, - nullptr /* pdwType */, - &installPathBuffer, - &installPathBufferLength)); - std::wstring installPathString{ installPathBuffer }; - - std::wstring productVersionKeyName{ L"ProductVersion" }; - WCHAR productVersionBuffer[MAX_PATH]; - DWORD productVersionBufferLength = sizeof(productVersionBuffer); - winrt::check_hresult(RegGetValueW( - hKey.get(), - nullptr, - productVersionKeyName.c_str(), - RRF_RT_REG_SZ, - nullptr /* pdwType */, - &productVersionBuffer, - &productVersionBufferLength)); - std::wstring productVersionString{ productVersionBuffer }; // The install path of the platform sdk in the filesystem always uses version format x.x.x.x // even though the version string in the registry may be stored as x.x if the full version is // x.x.0.0 - auto versionPartsFound = std::count(productVersionString.begin(), productVersionString.end(), '.') + 1; - INT64 versionPartsRequired = 4; - INT64 versionDigitsToAdd = versionPartsRequired - versionPartsFound; - for (auto i = 0; i < versionDigitsToAdd; i++) + int versionPartsFound = productVersion.Count(c => c == '.') + 1; + int versionPartsRequired = 4; + int versionDigitsToAdd = versionPartsRequired - versionPartsFound; + for (int i = 0; i < versionDigitsToAdd; i++) { - productVersionString += L".0"; + productVersion += ".0"; } - std::wstring processorArchitecture{}; - SYSTEM_INFO systemInfo; - GetNativeSystemInfo(&systemInfo); - switch (systemInfo.wProcessorArchitecture) + string processorArchitecture = ""; + switch (System.Runtime.InteropServices.RuntimeInformation.OSArchitecture) { - case PROCESSOR_ARCHITECTURE_AMD64: - processorArchitecture += L"x64"; - break; - case PROCESSOR_ARCHITECTURE_ARM: - processorArchitecture += L"arm"; - break; - case PROCESSOR_ARCHITECTURE_ARM64: - processorArchitecture += L"arm64"; - break; - case PROCESSOR_ARCHITECTURE_INTEL: - processorArchitecture += L"x86"; - break; - case PROCESSOR_ARCHITECTURE_IA64: - case PROCESSOR_ARCHITECTURE_UNKNOWN: - winrt::throw_hresult(E_UNEXPECTED); + case System.Runtime.InteropServices.Architecture.X64: + processorArchitecture = "x64"; + break; + case System.Runtime.InteropServices.Architecture.Arm: + processorArchitecture = "arm"; + break; + case System.Runtime.InteropServices.Architecture.Arm64: + processorArchitecture = "arm64"; + break; + case System.Runtime.InteropServices.Architecture.X86: + processorArchitecture = "x86"; + break; + default: + throw new Exception("Unknown Windows SDK architecture"); } - std::filesystem::path platformSDKExecutablePath = installPathString; - platformSDKExecutablePath /= L"bin"; - platformSDKExecutablePath /= productVersionString; - platformSDKExecutablePath /= processorArchitecture; - + string platformSDKExecutablePath = Path.Combine(installPath, "bin", productVersion, processorArchitecture); return platformSDKExecutablePath; } - void LaunchSignToolWithArguments(std::wstring arguments) - { - std::filesystem::path signToolPath = GetPlatformSDKPath(); - signToolPath /= L"signtool.exe"; - - std::wstring commandLine{ signToolPath }; - commandLine += L" " + arguments; - DWORD exitCode{}; - winrt::check_hresult(CreateProcessAndWaitForExitCode(commandLine, exitCode)); - if (exitCode == EXIT_FAILURE) - { - winrt::throw_hresult(E_FAIL); - } - } + throw new Exception("Windows SDK Path not found in registry"); +} - void Example_SignPackage(std::wstring pfxFile, std::wstring packageFile) +static async Task Example_SignPackage(string pfxFile, string packageFile) +{ + string signToolPath = Path.Combine(GetPlatformSDKPath(), "signtool.exe"); + string signToolArguments = "sign /fd SHA256 /f " + pfxFile + " " + packageFile; + int exitCode = await CreateProcessAndWaitForExitCodeAsync(signToolPath, signToolArguments); + if (exitCode != 0) { - std::wstring signToolArguments; - signToolArguments.append(L"sign /fd SHA256 /f "); - signToolArguments.append(pfxFile); - signToolArguments.append(L" " + packageFile); - LaunchSignToolWithArguments(signToolArguments); + throw new Exception("Signtool failed with argumemnts: " + signToolArguments); } + return; +} ``` # 3. API Details From cb9440ddcc726bd9b87b79594b6895660cad6e8b Mon Sep 17 00:00:00 2001 From: sreading-MSFT <123122426+sreading-MSFT@users.noreply.github.com> Date: Mon, 9 Oct 2023 19:57:40 -0700 Subject: [PATCH 8/8] Update for comments. --- specs/WinRT/MakeMSIX-spec.md | 62 +++++++++++++++++++++--------------- 1 file changed, 37 insertions(+), 25 deletions(-) diff --git a/specs/WinRT/MakeMSIX-spec.md b/specs/WinRT/MakeMSIX-spec.md index 2ad672d241..5adf36e129 100644 --- a/specs/WinRT/MakeMSIX-spec.md +++ b/specs/WinRT/MakeMSIX-spec.md @@ -31,12 +31,13 @@ The API provides a WinRT interface that supports: 4. Creation of Kozani packages. Creation of Kozani packages is a new feature that has not existed in other msix packaging tools. The -other features have previously been available in command line tools discussed in the appendix. +other features in this spec have previously been available in command line tools discussed in the +appendix. Kozani packages are msix packages that have been optimized for size by removing unnecessary resources. All existing tools that work with msix packages are compatible with Kozani packages. Kozani packages can be created manually without this API by modifying the files, manifest, and -resource index of an existing package using existing win32 apis. +resource index of an existing package using existing APIs. # 2. Examples @@ -356,6 +357,10 @@ namespace Microsoft.Kozani.MakeMSIX /// If field is empty, all scale factors from the original package are maintained. Windows.Foundation.Collections.IVector ScaleFactors{ get; }; + /// DirectX feature levels supported by the output package + /// If field is empty, all DirectX feature levels from the original package are maintained. + Windows.Foundation.Collections.IVector DXFeatureLevels{ get; }; + /// Overwrite output file if it already exists. /// Defaults to true. Boolean OverwriteOutputFileIfExists; @@ -373,7 +378,7 @@ namespace Microsoft.Kozani.MakeMSIX CreateBundleOptions(); /// Create as a flat bundle. Package locations will be stored as external path references - /// rather than being stored inside the msixbundle file itself. + /// to files outside the msixbundle file. /// Defaults to false. Boolean FlatBundle; @@ -436,7 +441,10 @@ namespace Microsoft.Kozani.MakeMSIX /// Publisher of the package String Publisher{ get; }; - + + /// PublisherId of the package + String PublisherId{ get; }; + /// Version of the package Windows.ApplicationModel.PackageVersion Version{ get; }; @@ -464,6 +472,9 @@ namespace Microsoft.Kozani.MakeMSIX /// ScaleFactors supported by the package Windows.Foundation.Collections.IVectorView ScaleFactors{ get; }; + + /// DirectX feature levels supported by the package + Windows.Foundation.Collections.IVectorView DXFeatureLevels{ get; }; } /// Static methods for creating and unpacking packages. @@ -527,35 +538,36 @@ namespace Microsoft.Kozani.MakeMSIX # 5. Existing interfaces and options The existing tools are: -- [Makeappx.exe](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) - Shipped in Windows' Platform SDK. - Exposes pack, unpack, bundle, unbundle, encrypt, decrypt and content group mapping. - Makeappx defines and uses its own file formats for encryption key files and content group - mapping to allow creation of packages with different payloads from the same folder. - -- [Makemsix.exe](https://github.com/Microsoft/msix-packaging) Available via the msix-packaging - project. Exposes pack, unpack, unbundle, and bundle* commands (*Bundle command only supports - creation of - [flat bundles](https://learn.microsoft.com/en-us/windows/msix/package/flat-bundles). Flat - bundles are bundles where package locations are stored as external path references rather than - being stored inside the msixbundle file itself. Flat bundles are referred to as sparse bundles - in the makeappx documentation). +- [Makeappx.exe](https://learn.microsoft.com/en-us/windows/win32/appxpkg/make-appx-package--makeappx-exe-) + - Shipped in Windows' Platform SDK. + - Exposes pack, unpack, bundle, unbundle, encrypt, decrypt and content group mapping. + - Makeappx defines and uses its own file formats for encryption key files and content group + mapping to allow creation of packages with different payloads from the same folder. + +- [Makemsix.exe](https://github.com/Microsoft/msix-packaging) + - Available via the msix-packaging project. + - Exposes pack, unpack, unbundle, and bundle* commands + - (*Bundle command only supports creation of + [flat bundles](https://learn.microsoft.com/en-us/windows/msix/package/flat-bundles). Flat + bundles are bundles where package locations are stored as external path references rather + than being stored inside the msixbundle file itself. Flat bundles are referred to as + sparse bundles in the makeappx documentation). -- [Msixmgr.exe](https://github.com/Microsoft/msix-packaging) Available via the msix-packaging - project and - [direct download from learn.microsoft.com](https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr). - Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. +- [Msixmgr.exe](https://github.com/Microsoft/msix-packaging) + - Available via the msix-packaging project and + [direct download from learn.microsoft.com](https://learn.microsoft.com/en-us/azure/virtual-desktop/app-attach-msixmgr). + - Exposes "unpack" command to convert msix packages to vhd, vhdx, cim image files. None of the existing tools provide a programmatic interface for all aspects of packaging. -Makeappx.exe provides the only complete implementation of packaging and bundling but requires +- Makeappx.exe provides the only complete implementation of packaging and bundling but requires process creation and does not return any specific errors from process exit. It also does not support any of the Azure app attach scenarios. -Makemsix.exe is a cross-platform implementation of packaging, but does not support a complete set of +- Makemsix.exe is a cross-platform implementation of packaging, but does not support a complete set of bundling features. The functionality is exposed as a command line as well as flat dll exports. -Msixmgr.exe is mostly for specific Azure app attach packaging scenarios not covered by the other +- Msixmgr.exe is mostly for specific Azure app attach packaging scenarios not covered by the other tools. To use it programmatically would require either process creation or building the msix-packaging github project and linking the static lib it produces into a project. @@ -917,7 +929,7 @@ after packaging is complete. The Appx SIP knows how to sign packages, but not vh directories. The result is that creating a vhdx with an appx signature file (appxsignature.p7x) requires first creating the package, then signing it, then creating the vhdx. Allowing creation of a cim, vhd, or vhdx file directly from a directory seems likely to cause confusion for developers who -may not realize that they won't be able to sign the package in the image. +may not realize that they won't be able to sign the package in the image. This API always applies ACLs for the package to the folder, and never validates the signature so those options in msixmgr have not been exposed. If signature validation of a package (checking that