Upgrade

Upgrading from one version of Butler to another is usually quite straightforward.

This page describes the upgrade process.

First: Don't panic

Upgrading Butler is usually a smooth process:

• Get the new version from the assets section on the download page. Extract the ZIP file.
• Back up your existing Butler configuration file.
• Edit the configuration file to match the new version's requirements.
• Stop the Butler process/service.
• Replace the Butler binary with the new version.
• Start the process/service.
• Verify that Butler is running as expected.
• 🥳 Celebrate!

Then: The details

Version number hints

Different kind of upgrades (usually) result in different levels on modifications needed in the main config file.

• "Small" Butler upgrades move from one patch version to another, without changing the feature version.
  Example: Upgrading from 7.3.0 > 7.3.4.
• "Medium" upgrades involves moving from one minor version to another, without changing the major version. Example: Upgrading from 7.2.3 > 7.3.0
• "Major" upgrades is when you move to a new major version. Example: 7.4.2 > 8.0.0

The versioning scheme used in Butler is described in the about section (coming soon).

WARNING

You should always upgrade to the latest available version.
That version has the latest features, bug fixes and security patches.

Minor upgrades

The new release includes bug fixes, security patches, minor updates to documentation etc - but no new features.

In theory there should never be any changes to the config files when doing a minor upgrade.

Medium upgrades

This scenario means that new features are added to Butler.
Usually there are also various bug fixes included.

Most new features need to be configured somehow, meaning that medium upgrades usually require modification to the config files.
The most common change by far is that it's the main config file that needs to be modified, but a new scheduler related feature could for example mean that the scheduler config file must be modified too.

The changes needed to the config files are usually additive in nature, i.e. some settings must be added to the config file, but the existing settings and general structure of the file remain the same.

Major upgrades

This scenario involves breaking changes of some kind.

These almost certainly require changes to the config files, sometimes even significant ones in the sense that the structure of the config file may have changed.

If very major rework has been done to Butler, this may also motivate a major version bump.

Know your config file

Butler is entirely driven by its configuration files, with the main YAML config file being the most important one.
There are other config files too, containing for example scheduling information.

InfluxDB considerations

Some Butler versions include changes to the InfluxDB schema, meaning that you need to do some manual work in order to upgrade to the new schema.

The easiest way to do this is to delete the InfluxDB database used by Butler, then let Butler re-create it using the new schema.
If the InfluxDB database specified in the Butler config file does not exist, Butler will automatically create it for you.

Deleting the InfluxDB database "senseops" can be done with a command similar to this:

influx --host <ip-where-influxdb-is-running> --port <influxdb-port-usually-8086>
drop database senseops
exit

Upgrade checklist

INFO

Starting with Butler version 9.0 there is a check that the config file has the correct format.

This means that if you forget to add or change some setting in the main YAML config file, Butler will tell you what's missing and refuse to start.
A consequence of this is that all settings are now mandatory, even if you don't use them.

1. Make a backup of your YAML configuration file before upgrading. Just... do it.
2. Look at the release notes to get a general feeling for what is new and what has changed.
   Those are the areas that may require changes in the config file.
3. Compare your existing main config file with the template config file available on GitHub.
   This comparison is a manual process and can be a bit tedious, but knowing your config file is really needed in order to make full and correct use of Butler.
   1. That file is also included in the Butler ZIP file available on the download page.
   2. A more in-depth description of the config file is available in the Reference docs > Config file syntax section of the documentation.
4. The result of the comparison will show you what parts of the config file are new (for medium-sized upgrades) and which parts have changed in a significant way (for major upgrades).
5. Get the binaries for the new Butler version from the download page.
6. Start the new Butler version and let it run for a few minutes.
   1. Review the console logs (or the log files) to make sure there are no warnings or errors.
   2. If there are warnings or errors it can be helpful to start Butler with more verbose logging.
      Adding --log-level verbose or even --log-level debug will give you more details on what Butler is doing and what might be causing the problems you are experiencing.

Version-Specific Upgrade Notes

Upgrading to Butler 16.0.0

Butler 16.0.0 adds support for InfluxDB v2 and v3, in addition to the existing v1 support. This brings Butler in line with Butler SOS, which already supports all three InfluxDB versions.

New Features

• InfluxDB v2 and v3 support - Butler can now store metrics in InfluxDB v1, v2, or v3. The version is selected using the Butler.influxDb.version configuration option.

Breaking Changes

This release includes breaking changes that require configuration file updates:

1. New InfluxDB configuration structure - The InfluxDB configuration has been restructured to use version-specific blocks:

   • Butler.influxDb.version - Set to 1, 2, or 3 to select InfluxDB version
   • Butler.influxDb.v1Config - Settings for InfluxDB v1.x (database name, retention policy, authentication)
   • Butler.influxDb.v2Config - Settings for InfluxDB v2.x (organization, bucket, token)
   • Butler.influxDb.v3Config - Settings for InfluxDB v3.x (database, token, timeouts)

   The old flat configuration (Butler.influxDb.auth, Butler.influxDb.dbName, Butler.influxDb.retentionPolicy) is no longer supported.

2. Backward compatibility removed - If you were using the legacy v1 configuration format, you must migrate to the new versioned structure.

InfluxDB Version Behavior Differences

Version	Behavior
v1	Butler automatically creates the database and default retention policy if they don't exist when Butler starts.
v2	The InfluxDB bucket must be created before starting Butler. Butler will not auto-create buckets.
v3	The InfluxDB database must be created before starting Butler. Butler will not auto-create databases.

Example: Migrating to v2/v3 Configuration

Old format (no longer works):

Butler:
  influxDb:
    enable: true
    hostIP: influxdb.mycompany.com
    hostPort: 8086
    auth:
      enable: false
      username: user_joe
      password: joesecret
    dbName: butler
    retentionPolicy:
      name: 10d
      duration: 10d

New format for InfluxDB v1:

Butler:
  influxDb:
    enable: true
    hostIP: influxdb.mycompany.com
    hostPort: 8086
    version: 1
    v1Config:
      auth:
        enable: false
        username: user_joe
        password: joesecret
      dbName: butler
      retentionPolicy:
        name: 10d
        duration: 10d

New format for InfluxDB v2:

Butler:
  influxDb:
    enable: true
    hostIP: influxdb.mycompany.com
    hostPort: 8086
    version: 2
    v2Config:
      org: my-org
      bucket: butler
      token: my-v2-token
      description: Butler metrics
      retentionDuration: 10d

New format for InfluxDB v3:

Butler:
  influxDb:
    enable: true
    hostIP: influxdb.mycompany.com
    hostPort: 8086
    version: 3
    v3Config:
      database: butler
      token: my-v3-token
      description: Butler metrics
      retentionDuration: 10d
      writeTimeout: 10000
      queryTimeout: 60000

Upgrade Steps for 16.0.0

1. Back up your existing configuration file
2. Download Butler 16.0.0 from the releases page
3. Compare your config file with the template config file
4. Add the version setting to Butler.influxDb (set to 1, 2, or 3 based on your InfluxDB version)
5. Move your existing v1 settings into the v1Config block, or create v2Config / v3Config if using InfluxDB v2 or v3
6. If using InfluxDB v2, create the bucket before starting Butler
7. If using InfluxDB v3, create the database before starting Butler
8. Stop the old Butler process
9. Start Butler 16.0.0 and verify operation in logs

Upgrading to Butler 15.0.0

Butler 15.0.0 is a major release with significant new features for task monitoring and alerting. This version expands InfluxDB and email support to cover all Qlik Sense task types.

Breaking Changes

This release includes breaking changes that require configuration file updates:

1. New InfluxDB configuration sections - If you use InfluxDB for task monitoring, you must add the following new sections to your config file under Butler.influxDb:

   • userSyncTaskSuccess - Store successful user sync tasks in InfluxDB
   • userSyncTaskFailure - Store failed user sync tasks in InfluxDB
   • externalProgramTaskSuccess - Store successful external program tasks in InfluxDB
   • externalProgramTaskFailure - Store failed external program tasks in InfluxDB
   • distributeTaskSuccess - Store successful distribution tasks in InfluxDB
   • distributeTaskFailure - Store failed distribution tasks in InfluxDB
   • preloadTaskSuccess - Store successful preload tasks in InfluxDB
   • preloadTaskFailure - Store failed preload tasks in InfluxDB

2. New email notification sections - If you use email notifications, you must add these new sections under Butler.emailNotification:

   • distributeTaskSuccess - Email alerts for successful distribution tasks
   • distributeTaskFailure - Email alerts for failed distribution tasks
   • preloadTaskSuccess - Email alerts for successful preload tasks
   • preloadTaskFailure - Email alerts for failed preload tasks

3. Reload task success emails - New settings for including script log excerpts in reload success emails:

   • Butler.emailNotification.reloadTaskSuccess.headScriptLogLines - Lines from start of script log
   • Butler.emailNotification.reloadTaskSuccess.tailScriptLogLines - Lines from end of script log

InfluxDB Schema Changes

Butler 15.0.0 introduces new InfluxDB measurements for all task types. If upgrading from an earlier version, you may need to recreate your InfluxDB database to accommodate the new schema:

influx --host <influxdb-host> --port 8086
drop database butler
exit

Butler will automatically recreate the database with the new measurements on startup.

New InfluxDB measurements in 15.0.0:

• user_sync_task_success / user_sync_task_failed
• external_program_task_success / external_program_task_failed
• distribute_task_success / distribute_task_failed
• preload_task_success / preload_task_failed

XML Log Appender Updates

If you use Butler's UDP-based task failure/success detection, you must update your Qlik Sense log appender configuration (LocalLogConfig.xml) to support preload task detection.

Add these filters to the TaskFailLogger appender:

<!-- Preload tasks may have either of these log messages when failing -->
<filter type="log4net.Filter.StringMatchFilter">
    <param name="stringToMatch" value="Failed to start session" />
</filter>
<filter type="log4net.Filter.StringMatchFilter">
    <param name="stringToMatch" value="Could not reserve an executor for task" />
</filter>

See the log appender setup documentation for complete configuration examples.

New Email Template Files

Butler 15.0.0 includes new email template files for distribution and preload tasks:

• failed-distribute-qseow.handlebars
• success-distribute-qseow.handlebars
• failed-preload-qseow.handlebars
• success-preload-qseow.handlebars

These template files are included in the Butler distribution and should be placed in your email templates directory.

Bug Fixes and Improvements

• Certificate loading fix: The --no-qs-connection command line option now correctly skips loading Qlik Sense API certificates. This is useful when running Butler in isolated mode without Qlik Sense connectivity.

• Improved stability: Added global unhandled rejection handler and comprehensive async error handling to prevent Butler crashes from unhandled promise rejections, particularly when processing External Program Task completions.

• Hostname validation: Added hostname validation for Windows service monitoring to provide clearer error messages when service hosts are misconfigured.

• Config validation: More informative logging when the configuration file is invalid, making it easier to identify and fix configuration issues.

Upgrade Steps for 15.0.0

1. Back up your existing configuration file
2. Download Butler 15.0.0 from the releases page
3. Compare your config file with the template config file
4. Add all new required sections (see Breaking Changes above)
5. Update your Qlik Sense LocalLogConfig.xml if using task failure detection
6. Copy new email template files to your templates directory
7. (Optional) Drop and recreate your InfluxDB database if using InfluxDB
8. Stop the old Butler process
9. Start Butler 15.0.0 and verify operation in logs

Upgrading to Butler 15.1.x

Butler 15.1.x includes minor improvements and bug fixes.

Changes

• Node.js 24: Release binaries now bundle Node.js 24 (updated from Node.js 22)
• CI improvements: Updated CI workflows for better reliability
• OSV scanner disabled: OSV scanner action disabled as it does not support pinned dependencies

Upgrade Steps

No configuration changes are required when upgrading from 15.0.x to 15.1.x. Simply download the new release and replace the Butler binary.

Finally: When things aren't working - check the logs

By far the most common problem when upgrading to a new Butler version (or doing a fresh install) is an incorrect config file.

If some feature in the config file is disabled, its associated settings are not mandatory and will not be checked/verified.
If a feature is enabled, its associated settings are mandatory.

Butler is pretty good at figuring out what is wrong with the config file, but there may be cases where it's not obvious what is wrong.

Thus, double check your config file, then triple check it.

Then start Butler and read the logs carefully.
If you need more details, start Butler with the --log-level verbose or even --log-level debug options to get more details on what's going on.

If things still don't work you can post a question in the Butler forums.

By sharing your installation and upgrade challenges/issues you enable future improvements, which will benefit both yourself and others.