Alerify Datacenter - Phone system issues – Incident details

Phone system issues

Resolved
Partial outage
Started 4 months agoLasted about 2 hours

Affected

Phone System

Partial outage from 4:52 PM to 6:20 PM, Degraded performance from 6:20 PM to 6:35 PM

Updates
  • Resolved
    Resolved

    Microsoft is reporting that the issue affecting inbound calls to Auto Attendants and Call Queues has been resolved. Alerify has confirmed the resolution in our testing. Microsoft’s incident TM907891 is included below.

    "Details
    Title: Users' calls routed through Auto attendants and Call queues within Microsoft Teams may fail

    User impact: Users' calls routed through Auto attendants and Call queues within Microsoft Teams may have failed.

    Final status: We've routed the affected call requests to a healthy portion of service infrastructure and confirmed through monitoring service telemetry that impact has been remediated.

    Scope of impact: Any user who attempted to make calls routed through Auto attendants and Call queues within Microsoft Teams may have been impacted.

    Start time: Thursday, October 10, 2024, at 3:51 PM UTC
    End time: Thursday, October 10, 2024, at 5:50 PM UTC

    Root cause: Auto attendant and Call queue call routed call requests were being routed to an unhealthy portion of service infrastructure, resulting in impact.

    Next steps:
    - We're continuing to monitor the affected infrastructure closely to gain further insight into the source of the underlying issue and better prevent similar impact in the future.
    - We’re assessing our automated recovery mechanisms in conjunction with our findings into the underlying problem to identify areas for improvement to reduce the time to mitigate similar future issues.

    This is the final update for the event."


    Alerify will be closing this master case and all associated cases.

  • Monitoring
    Monitoring

    Microsoft is working on rerouting traffic to fix this issue now and will provide further updates shortly. We are seeing some successful calls in our internal testing and testing reported by customers.

  • Investigating
    Investigating

    An issue has been identified with the Alerify phone system that is preventing inbound calls from functioning as expected. A ticket has been opened with the provider and they are investigating the issue. For immediate assistance, please email support@alerify.com