Input Output CEO Charles Hoskinson He said that due to the transient nature of the Cardano node anomaly, it is difficult to pinpoint the exact cause.
However, he added that the network handled the outage exactly as designed. Additionally, he applauded the IO staff and stake pool operators (SPOs) who have rallied to offset the problem.
“Sh*t always takes a break on Sunday morning, or Monday morning, late at night when everyone is asleep. That’s how things work.
The upside is that it seems like a temporary problem. “
50% of Cardano nodes are down for a short time
On January 22nd, 50% of Cardano nodes experienced a “temporary glitch” and were disconnected and restarted.
This affected block production for 2-5 minutes, causing the chain to temporarily go out of sync when the affected node restarted. There was a short period of network degradation, but this was remedied by what Hoskinson called “self-healing.”
Initial investigation by IO found no apparent root cause. IO’s CEO has since elaborated on this, saying that the anomaly is likely due to the convergence of multiple factors, making it difficult to recreate the same conditions that caused the problem.
“It’s probably a collection of things that happened at the same time, which means it’s less reproducible.”
Hoskinson details
In the course of further investigation, a call error was identified, but the triggering event has not yet been identified, Hoskinson said.
Hoskinson elaborated on this, blaming it for an “emergency bug”, adding that in distributed systems, global frameworks can cause this kind of anomaly.
“The problem is that sometimes distributed systems create so-called emergency bugs. They can’t be reproduced locally, but the collection of things creates a collective global state that for some reason is something , causing the entire system to basically stop.”
These “once in five years” glitches can sometimes go unnoticed. Sometimes it can be worked out, but “it’s never going to drive you crazy,” he says, Hoskinson.
The team continues to investigate this issue and a follow-up investigation will be conducted.
