The issue has been resolved and the system is now stable.

We are continuing to actively monitor the affected services.

A DNS outage is causing networking issues across all GPU clusters. As a result, we have seen increased error rates for all inference requests.

These networking issues are preventing job requests and job responses from flowing between the API server and the inference servers.