Authoring Installer for Large Resources Overcoming MSI Size Limits (from a developer perspective)

Windows Installer has its limitations when it comes to managing large file installation. We’ll explore the available options for building installers with very large resources inside.

Before diving into, let’s summarize the current MSI limitations. The MSI utilizes CAB (Cabinet) files as storage containers for the files it includes.

• Cab Maximum File Size: The maximum size of a CAB file is 2 GB. This limitation is imposed by the CAB file format itself.
• Installer size: The MSI with resources inside cannot exceed 2 GB and EXE with resources inside cannot exceed 4GB. This is a Windows limitation.

If trying to package an installer that exceeds more than 2 GB, we’ll get the following error during build:

So, the sample installer has a total size for the resources of 5.2GB and more than 11k files, quite a large installer.

We can create multiple CAB files with Advanced Layout but this will create the MSI/EXE unpacked, with files next to the installer, however this may create difficulties for distribution.

If we want to create a single installer, the only option is to build an EXE installer which can download the resources from the internet, at install time.

To create a web installer, we have some options that can be taken into consideration:

• 1. Web installer and resources stored into LZMA archive:
• 2. Creating a wrapper installer that just downloads a zip archive & file extract operation for files
• 3. Mixed installer with files inside and the large files downloaded during the installation
• 4. Mixed installer with files contained by the package and large files downloaded using CA
• 5. Let’s summarize our tests:

1. Web installer and resources stored into LZMA archive:

For the LZMA compression, there are few compression levels available, you can choose what option works best for you depending on your application resources number and size.

During tests, the following results showed up:

• Resources size: 5.2 GB (with 4 .zip files of 1GB each)
• Files: over 11k
• Folders: over 2k

The performance of the installer is significantly influenced by the quality and capabilities of the computer hardware in use.

In our practical example, the build machine had the following hardware:

• CPU: 12th Gen Intel(R) Core(TM) i7-12700K 3.60 GHz
• RAM: 64.0 GB (63.7 GB usable)

The time and resulting size of an archive depend heavily on the types of files being included. Text-based and uncompressed files typically compress well and reduce significantly in size, while already compressed files may not compress much further.

In our case, 3.9 GB of our resources are already .zip files, while only 1.28 GB are regular files (exe, dll, txt).

So it's important to note that our .7z resources are already compressed and may not compress much further when added to an archive. In some cases, attempting to compress already compressed files may even result in a larger archive size due to overhead.

2. Creating a wrapper installer that just downloads a zip archive & file extract operation for files

Instead of adding them as part of the installer, thus registered in the MSI database, we can download .zip files during installation and extract them into the installation folder.

Since the archive is created outside the installer, you have full control of the compression.

That means, the Files and Folders view will contain only the File Download & File Extract operations:

To keep things simple, we’ll create a single archive, containing all the files. The MSI will be a wrapper MSI that will just download & extract the .zip archive in the installation folder selected by the user.

For the same resources as in the previous example, the install time was about 10 minutes. That means that in 10 minutes, the installer downloaded the .zip archive of 4.6 GB and extracted it in the installation folder selected by the user.

When using the first approach, the installer is aware of all files from the package. That means a separate component is created for most of the files from the package.

With the second approach, we have a lot fewer components registered by the MSI, thus the faster installation time. This is happening because of the actions which are usually executed by the MSI and now are skipped.

Here are some examples of standard actions related to files and components that are skipped since the MSI does not have any information registered in the database about files that are getting installed:

• File Installation: Actions to copy files from the MSI package to the destination directories on the user's system.
• Component Registration: Actions to register components with the Windows Installer service, ensuring proper tracking and management of installed components.
• File Versioning: Actions to check file versions and determine whether to overwrite existing files with newer versions during installation.

With this approach, the MSI just executes those 2 actions, without being aware of the files that are being installed, without registering the files with the Windows Installer (their components). The installer cannot be customized into features, so the user cannot choose what resources to install. That can be overcome by having multiple file download and extract operations, and conditionally executing them based on the user selection or certain conditions.

In this scenario, it's essential to manually verify whether the machine has enough storage space, particularly considering the necessity to download and subsequently extract a zip archive.

Check the full support for downloading and extracting archives in the Download files at install time video.

Here’s how the two operations to download the .zip archive and extracts its content are configured:

Even if creating multiple file download & file extract operations, they are executed synchronously, in a single operation by the setup. That means, the files are downloaded one at a time and once all files are downloaded they will be extracted one after the other.

Another downside of this approach is that we cannot use further references of the files, let’s say that some files are used for creating some services, some scheduled tasks etc.

In this case, we need to keep references for those files in the installer and only the files that are not further referenced by the installer to download them during installation.

So, in most cases, there will be a mix between the files from the installer (not more than 2GB for MSI and 4GB for EXE) and the files that are downloaded from the internet.

3. Mixed installer with files inside and the large files downloaded during the installation

So, about 4GB of resources which are represented by the large files will be downloaded during installation.

The rest of the files (about 1.2 GB) are added in the installer, they will not be downloaded during installation.

This allows us to further reference the files e.g. for installing services, creating scheduled tasks or using any of our files in custom actions.

• Build time: Total build time: 2 min 36 sec.
• Installation time: 3 min and 12 sec.
• Installer size: 580 MB

Even if creating multiple file download & file extract operations, they are executed synchronously, in a single operation by the setup. That means, the files are downloaded one at a time and once all the files are downloaded they will be extracted one after the other.

If you want to schedule parallel downloads and extract operations, for large files which are hosted online, you can use custom actions.

Instead of relying on the predefined support provided by Advanced Installer to download and extract the .zip archive, you have the option to manage the download and extraction operations within a custom action. By incorporating these operations into a custom action, you gain the flexibility to execute them independently of the installation process. This means that the installation does not need to wait for the custom action to complete its operations, allowing for smoother and more efficient installation workflows.

4. Mixed installer with files contained by the package and large files downloaded using CA

Instead of relying on a single download and extract operation, we have the option to split large files into multiple .zip archives. Custom actions can then be triggered concurrently in the background to download and extract these archives simultaneously.

This approach allows for greater efficiency in handling large files, as the installer does not need to wait for each custom action to complete before proceeding. Consequently, multiple custom actions may still be executing in the background, handling the download and extraction processes concurrently.

As an extreme example of this approach, the installer continues its operations without waiting for the completion of all custom actions. This means that some custom actions, particularly those related to downloading and extracting archives, may still be in progress while the installation proceeds.

To ensure the completeness of the installation, a custom action can be placed at the very end of the installation process. This action checks if all necessary files have been successfully downloaded and extracted. In the event that any files are missing, the installation can be postponed for an additional 10 seconds to allow the remaining operations to complete.

Sample function to handle the .zip download archive:

public static void DownloadFile(string url, string outputPath)
    {
        try
        {
            using (WebClient webClient = new WebClient())
            {
                webClient.DownloadFile(url, outputPath);
                Console.WriteLine("File downloaded successfully to: " + outputPath);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Failed to download file from " + url + ". Error: " + ex.Message);
        }
    }

Sample function to handle the .zip extract operation:

public static void ExtractZipFile(string zipFilePath, string extractPath) { try { ZipFile.ExtractToDirectory(zipFilePath, extractPath); Console.WriteLine("File extracted successfully to: " + extractPath); } catch (Exception ex) { Console.WriteLine("Failed to extract file " + zipFilePath + ". Error: " + ex.Message); } }

To create and integrate the custom action in the installer, check the How to create a .NET Custom Action article with related information for hints and guidance.

Since the resources are downloaded through custom actions, these files are not registered in the MSI database, we need to manually remove them on uninstall.

For this, we can use the Uninstall Cleanup predefined functionality from the Files and Folders view, no need to create a custom action to handle the files removal:

The wizard will assist you with the uninstall cleanup configuration. This step is necessary to remove resources downloaded from the internet using custom actions.

5. Let’s summarize our tests:

The performance of the installer is significantly influenced by the quality and capabilities of the computer hardware in use.

In our practical example, the build machine had the following hardware:

• CPU: 12th Gen Intel(R) Core(TM) i7-12700K 3.60 GHz
• RAM: 64.0 GB (63.7 GB usable)

The time and resulting size of an archive depend heavily on the types of files being included. Text-based and uncompressed files typically compress well and reduce significantly in size, while already compressed files may not compress much further.