CurrentWave AI - Question and answer slowness – Incident details

Question and answer slowness

Resolved
Operational
Started 11 days agoLasted about 11 hours

Affected

CXplain.ai

Operational from 1:22 PM to 12:31 AM

Updates
  • Resolved
    Resolved

    This incident has been resolved. Our updates have remained stable, and Microsoft has resolved the underlying outage that caused the performance issues. We appreciate your patience as we worked through this disruption!

  • Monitoring
    Monitoring

    CurrentWave AI has deployed our emergency fix to production, and all workloads including Question and Answer speeds appear to be back to normal.

    We are still monitoring the Microsoft outage, and we will keep this incident open until MSFT has resolved the outage on their side as well.

  • Update
    Update

    CurrentWave AI has completed an emergency code release to work around the Microsoft AI outage. We will be deploying the emergency fix to production in the next few minutes.

    Microsoft continues to report the outage on their side is ongoing but they are making progress on their fix. We'll continue to post updates here.

  • Update
    Update

    Microsoft continues to report that they are having a significant outage for AI services. We have moved our workloads out of the affected region, but we are still seeing significant slowness and latency. We will continue to update as we know more.

  • Identified
    Identified

    Microsoft has verified that there is an issue with AI services within Azure, where CXplainAI is hosted. We are working with these teams as they work toward identifying the problem and providing a resolution.

  • Investigating
    Investigating

    We are aware that of slowness with some question and answers transactions. We are currently investigating this issue and will provide an update as soon as possible.