Incidents and maintenance reported on status page for Techno Animal https://status.technoanimal.net/ en https://status.technoanimal.net/incident/446509 Mon, 21 Oct 2024 14:06:00 -0000 https://status.technoanimal.net/incident/446509#e2926eb48a77d9ebb8698fa1651083cd7653f1e451ff4a33620de67ab70a56f9 Incident It seems that at some point in the past the minio instance running within podman started terminating repeatedly, causing intermittent downtime (as seen in the historical graphs). It had no real reason to be containerised and thus the binary was run as a low-privileged user directly, and the problem ceased to occur. https://status.technoanimal.net/incident/343915 Thu, 21 Mar 2024 04:21:00 -0000 https://status.technoanimal.net/incident/343915#c1ff74894dcd4d1874c1c6c31bea88aa9d2cfeec869d9896800e018bdbe1a9fa Incident At around 09:50pm CET the upstream VPN provider had issues: Mar 20 20:53:52 flood mullvad-daemon[633]: [talpidcore::tunnelstatemachine::connectingstate][DEBUG] WireGuard tunnel timed out Mar 20 20:53:52 flood mullvad-daemon[633]: [talpidcore::tunnelstatemachine::connectingstate][DEBUG] Tunnel monitor exited with block reason: None Mar 20 20:53:52 flood mullvad-daemon[633]: [mullvadrelayselector][INFO] Selected relay fr-par-wg-004 at 193.32.126.69 Mar 20 20:53:52 flood mullvad-daemon[633]: [mullvadrelayselector][DEBUG] Relay matched on highest preference for retry attempt 0 Mar 20 20:53:52 flood mullvad-daemon[633]: [mullvad_daemon::tunnel][DEBUG] Same IP is NOT being used Unfortunately the automatic recovery did not work as expected, instead looping with: Mar 20 20:54:04 flood mullvad-daemon[633]: [talpidcore::tunnelstatemachine::connectingstate][DEBUG] WireGuard tunnel timed out Mar 20 20:54:04 flood mullvad-daemon[633]: [talpidcore::tunnelstatemachine::connectingstate][DEBUG] The tunnel disconnected unexpectedly Mar 20 20:54:04 flood mullvad-daemon[633]: [talpidcore::tunnelstatemachine::connectingstate][DEBUG] Tunnel monitor exited with block reason: None The machine was rebooted and service restored. There will be followup work to make this more resilient, and to alert via OpsGenie for quicker response through work in the Sentinel. Only downloading was affected.