Changelog: Fusion

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.4.15-amd64.json'
}

Feature updates and improvements​

Snapshots​

• Added incremental dumps on 2.4.
• Bumped snapshots to 1.6.0 in 2.4.15

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.5.4-amd64.json'
}

Feature updates and improvements​

Snapshots​

• Added incremental dumps.
• Updated fusion snapshots to 1.6.0

Configuration​

• Added support for checksum algorithm.
• Added ability to disable OTEL machinery if tracing not enabled.
• Added fallback to use AZ_BATCH_TASK_WORKING_DIR as FUSION_CACHE_LOCATION.

Dependencies​

• Bumped github.com/docker/docker from 28.2.2+incompatible to 28.3.3+incompatible
• Bumped hoverkraft-tech/compose-action from 2.3.0 to 2.4.0
• Bumped google.golang.org/grpc from 1.75.0 to 1.75.1
• Bumped actions/download-artifact from 4.3.0 to 5.0.0
• Bumped insightsengineering/coverage-action from 2.7.2 to 3.0.1

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.4.14-amd64.json'
}

Feature updates and improvements​

Azure​

• Added exponential backoff retries to Azure Blob Storage.
• Added fallback to use AZ_BATCH_TASK_WORKING_DIR as FUSION_CACHE_LOCATION.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.5.3-amd64.json'
}

Feature updates and improvements​

Snapshots​

• Enabled Fusion FS + Fusion Snapshots publish on ARM architecture.
• Added Snapshots-enabled ARM builds.
• Bumped fusion-snapshots to 1.3.0

Performance​

• Added the ability to track "in progress" uploads to prevent duplicates.
• Removed timers governing chunk uploads.

Azure​

• Added an add exponential backoff retry strategy for Azure Blob Storage.

Configuration​

• Added a warning for expected Spot termination.
• Added env variables to disable pattern matching.
• Added functionality to generate post-mortem runtime traces if requested.

Logging​

• Added an option to send logs to a remote IP.

Dependencies​

• Upgraded golangci.yml to version 2
• Bumped github.com/go-jose/go-jose/v4 from 4.0.4 to 4.0.5
• Bumped golangci/golangci-lint-action from 7.0.0 to 8.0.0

Bug fixes​

General​

• Added a fix to suppress excessive checksum validation warnings.
• Fixed empty chunk allocation for file gaps.
• Fixed race conditions when flushing chunks.
• Solved a bug that caused the SpanExporter to always be NoOp.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.4.13-amd64.json'
}

Feature updates and improvements​

Snapshots​

• Bumped Fusion Snapshots v1.2.0 to add ARM support.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.4.12-amd64.json'
}

Feature updates and improvements​

Snapshots​

• Bumped Fusion Snapshot to 1.1.0.

Configuration​

• Fixed warning on Spot reclaim.

Bug fixes​

Azure​

• Fixed EISDIR on Azure accounts with hieararchical namespaces enabled.

General​

• Fixed unlock function to match mutex type.

Fusion Snapshot 1.1.0 changelog​

Snapshots​

• Returns a specific error code (175) on failed restore.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.5.2-amd64.json'
}

Feature updates and improvements​

Azure​

• Added support for authenticating with Azure Managed Identities.

Logging​

• Added goroutine ID to log messages.

Configuration​

• Added ability to generate post-mortem pprof profile data.
• Added ability to enable monitoring of goroutines and memory usage.

Bugfix release to improve stability.

Bug fixes​

Logging​

• Fixed logs removed when removeLogs is false.

General​

• Fixed unique IDs in file fragments.
• Removed exit code on serveFusion.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.5.1-amd64.json'
}

Fusion 2.5.0 significantly enhances performance and efficiency in file operations.

Fusion now handles small files more effectively and intelligently manages Nextflow task files by minimizing intermediate file uploads. A key improvement is the selective retention of only essential task output files upon completion, reducing storage overhead. Network optimization has been implemented to decrease bandwidth usage by consolidating upload operations. Additionally, the underlying architecture has undergone substantial refactoring to support the future integration of diverse remote storage solutions, setting the foundation for expanded storage capabilities.

Feature updates and improvements​

Performance​

• Chunked async content management.
• Nextflow-compatible pattern classifier.

Configuration​

• Defined a default exit code on Fusion-IO errors.
• Object API refactoring.

Dependencies​

• Bumped golang.org/x/crypto to 0.31.0.
• Bumped golang.org/x/net to 0.33.0.

Bug fixes​

General​

• Fixed AsyncStore move issues.
• Fixed log file deletion during spot instance reclamation.
• Fixed transient object persistence.

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.5.0-amd64.json'
}

How to pin this version​

fusion {
  enabled = true
  containerConfigUrl = 'https://fusionfs.seqera.io/releases/v2.4.8-amd64.json'
}

Bugfix release to improve stability.

Bug fixes​

Logging​

• Do not delete the logs file during spot reclaim
• Uploaded logs file in parts to support files larger than 5 GB

General​

• Removed leftover .bak file when using BAM and Tabix index

Security​

• Security: Bumped golang.org/x/crypto from 0.25.0 to 0.31.0