IBM-Aspera Service Status Status

All Systems Operational

IBM-Aspera API Services Operational

api.ibmaspera.com Operational

ats.aspera.io Operational

platform.bss.asperasoft.com Operational

Frontend Operational

AoC Legacy Managed Nodes for Built-in Storage Operational

Amsterdam (ams) Operational

Dallas (dal) Operational

Frankfurt (fra) Operational

Toronto (tor) Operational

Sydney (syd) Operational

Amazon Transfer Clusters Operational

Frankfurt (eu-central-1) Operational

Ireland (eu-west-1) Operational

Oregon (us-west-2) Operational

São Paulo (sa-east-1) Operational

Singapore (ap-southeast-1) Operational

Sydney (ap-southeast-2) Operational

Tokyo (ap-northeast-1) Operational

Virgina (us-east-1) Operational

London (eu-west-2) Operational

N. California (us-west-1) Operational

IBM Cloud Transfer Clusters Operational

Amsterdam (ams) Operational

Dallas (dal) Operational

Frankfurt (fra) Operational

London (lon) Operational

Milan (mil) Operational

San Jose (sjc) Operational

Sao Paulo (sao) Operational

Sydney (syd) Operational

Tokyo (tok) Operational

Toronto (tor) Operational

Washington D.C. (wdc) Operational

Osaka (osa) Operational

Chennai (che) Operational

Azure Transfer Clusters Operational

Central United States (ats-azure-centralus) Operational

East United States (ats-azure-eastus) Operational

North Europe (ats-azure-northeurope) Operational

West Europe (westeurope) Operational

West United States (ats-azure-westus) Operational

South East Asia (azure-southeastasia) Operational

London (ats-azure-uksouth) Operational

Google Transfer Clusters Operational

United States Central 1 (us-central1) Operational

London (europe-west2) Operational

Los Angeles (us-west2) Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

Scheduled Maintenance

[AoC:API] - Monthly Maintenance Notice Aug 3, 2024 09:00-19:00 PDT

Update - We will be delaying this maintenance window by one week. The new maintenance window will be next Saturday, August 3rd.
Jul 26, 2024 - 12:21 PDT

Scheduled - We will be applying monthly patches across our backend infrastructure starting at 2024-07-27 at 09:00 PDT. The maintenance period is expected to last for 10 hours total. In scope for this maintenance window are the following changes:
* Rolling restart of kubernetes worker nodes to apply latest patches
* Upgrade of several backend service deployments to apply new patches
* Deploying new transfer nodes virtual machine images for all ATS clusters
Monthly patching is necessary for us to be on top of the latest security fixes for the software we run. A monthly cadence ensures that we remediate vulnerabilities within a timely window.

Since there is rolling restart of the backend infrastructure serving the AoC API during this time, there may be times that backend services become unavailable as processes get restarted and re-shuffled around. API calls, including those that start transfers, might fail during this time, or events being ingested may take longer to be processed. However, it is expected that these interruptions should self-resolve quickly. If you experience a prolonged interruption during this time, please open a support ticket. The Devops team will be actively monitoring the upgrade during this time.
Jul 23, 2024 - 15:52 PDT

Past Incidents

Jul 27, 2024

No incidents reported today.

Jul 26, 2024

[IBM-Aspera] - [AWS:US-WEST-2] - Service Disruption

Resolved - This incident has been resolved.
Jul 26, 14:56 PDT

Monitoring - A fix has been implemented and we are monitoring the results.
Jul 26, 14:01 PDT

Update - We are continuing to work on a fix for this issue.
Jul 26, 13:08 PDT

Identified - The issue has been identified and a fix is being implemented.
Jul 26, 13:08 PDT

Update - We are continuing to investigate this issue.
Jul 26, 12:41 PDT

Investigating - We have been alerted to degraded performance affecting: ATS AWS US-WEST-2. Our engineers are currently investigating the incident and will provide updates when more information is available.
Jul 26, 12:35 PDT

Jul 25, 2024

[IBM-Aspera] - [AWS:EU-WEST-2] - Service Disruption

Resolved - This incident has been resolved.
Jul 25, 20:32 PDT

Monitoring - We have enacted a fix and the cluster is back to being health. We are continuing to monitor
Jul 25, 20:21 PDT

Investigating - We have been alerted to a service disruption affecting: ATS AWS EU-WEST-2. Our engineers are currently investigating the incident and will provide updates when more information is available.
Jul 25, 20:11 PDT

Jul 24, 2024

[IBM-Aspera] - [AWS:US-WEST-2] - Service Disruption

Resolved - We believe this issue to be fully resolved, but some customers might have problems due to data loss caused by the database restoration process. Support tickets should be raised for help resolving those issues. To be clear, the type of data loss incurred would be database only. This might means problems with navigating folders within the UI. Files that exist in S3 buckets in us-west-2 would not be affected at all.

An RCA will provided within the next 48 hours.
Jul 24, 17:57 PDT

Monitoring - A fix has been implemented and we are monitoring the results.
Jul 24, 17:32 PDT

Update - Service is restored again. We have taken action to resolve the cause of the issue. We will continue to work with our database vendor and monitor the environment closely.
Jul 24, 17:32 PDT

Update - We are performing database restoration actions again from our July 24, 2024, 12:36 PM backup. We have identified the cause of the issue and will perform some additional actions after database restoration is complete to prevent further disruptions going forward.
Jul 24, 17:13 PDT

Identified - We are working with out database vendor at the moment to bring the cluster back online.
Jul 24, 16:58 PDT

Update - As we were actively monitoring the database, it entered a bad state due to sudden spikes in resource consumption. We are working on resolving the issue.
Jul 24, 16:45 PDT

Monitoring - We have finished operations to bring the database back to a known good state. Service has been restored, but there would have been some data loss as we restored from a backup that was from July 24, 2024, 12:36. Support tickets should be raised should your organization have continued issues.

We are continuing to monitor and are evaluating preventative measures to ensure we avoid the issue re-occurring.
Jul 24, 16:12 PDT

Update - Re-sharding continues. We are also working on other paths to restore service.
Jul 24, 15:31 PDT

Update - We are working on resharding data in our database. The issue seems to be the result of a huge spike in usage that our current database settings were not configured appropriately to handle. Another update will happen in 30 minutes.
Jul 24, 15:00 PDT

Identified - We are continuing to work to restore this database. Our next update will be in 30 minutes.
Jul 24, 14:29 PDT

Update - We have identified the problem as belonging to the database servicing this region. We are currently working with our database vendor to bring it back online.
Jul 24, 14:00 PDT

Investigating - We have been alerted to a service disruption affecting: ATS AWS US-WEST-2. Our engineers are currently investigating the incident and will provide updates when more information is available.
Jul 24, 13:22 PDT

Jul 23, 2024

No incidents reported.

Jul 22, 2024

No incidents reported.

Jul 21, 2024

No incidents reported.

Jul 20, 2024

No incidents reported.

Jul 19, 2024

No incidents reported.

Jul 18, 2024

[IBM-Aspera] - [Azure: CentralUS] - Service Disruption

Resolved - This incident has been resolved.
Jul 18, 22:59 PDT

Monitoring - A fix has been implemented and we are monitoring the results.
Jul 18, 22:47 PDT

Update - Azure in centralus is still experiencing an outage https://azure.status.microsoft/en-us/status
Jul 18, 21:29 PDT

Update - We continue to work with Azure on this issue.
Jul 18, 20:39 PDT

Identified - Azure is experiencing multiple issues in the centralus region, which is affecting our service: https://app.azure.com/h/1K80-N_8/110553
Jul 18, 16:59 PDT

Investigating - We have been alerted to a service disruption affecting: ATS Azure Central US. Our engineers are currently investigating the incident and will provide updates when more information is available.
Jul 18, 16:39 PDT

[IBM-Aspera] - [IBM: FRA] - Service Disruption

Resolved - This incident has been resolved.
Jul 18, 15:53 PDT

Monitoring - A fix has been implemented and we are monitoring the results.
Jul 18, 15:14 PDT

Investigating - We have been alerted to a service disruption affecting: ATS IBM Cloud Frankfurt. Our engineers are currently investigating the incident and will provide updates when more information is available.
Jul 18, 14:45 PDT

Jul 17, 2024

No incidents reported.

Jul 16, 2024

No incidents reported.

Jul 15, 2024

No incidents reported.

Jul 14, 2024

No incidents reported.

Jul 13, 2024

[ATS:AWS][ATS:Google] - Maintenance Notice

Completed - The scheduled maintenance has been completed.
Jul 13, 19:00 PDT

Update - issue is resolved. We are still doing some maintenance tasks, but no further interruptions are expected to happen
Jul 13, 14:02 PDT

Update - We've run into an unexpected problem and downtime will be considerably longer than expected. We are working on remediating as quickly as possible.
Jul 13, 13:05 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 13, 09:00 PDT

Scheduled - We will performing database migrations across our ATS AWS and Google infrastructure starting on 2024-07-13, at 09:000 PDT. The maintenance period is expected to last for 10 hours total.

The scope of this maintenance is to perform database migrations that are necessary to enable new features. Transfers are expected to be disrupted at some point during the migration. Aspera Connect transfers would be expected to restart automatically, but node-to-node transfers would have to be manually restarted. Downtime might be incurred. If it does occur, it is not expected to be more than 5-10 minutes at most.
Jul 10, 15:36 PDT