TOR switch replacement in SD6
Scheduled Maintenance Report for
The scheduled maintenance has been completed.
Posted Nov 16, 2023 - 08:30 UTC
Verification is currently underway for the maintenance items.
Posted Nov 15, 2023 - 10:59 UTC
All nodes have been dual-attached, the monitoring looks good. It done.
No more cumulus/dell switches at SD6
Posted Nov 15, 2023 - 10:59 UTC
Lf-2-b.sd6 is operational and MLAG is up in good state, uplinks are up.
We are plugging nodes and turning the interfaces up
Posted Nov 15, 2023 - 10:37 UTC
lf-2-b.sd6 has been isolated and shutted down. We are swapping it with the arista.
Posted Nov 15, 2023 - 09:28 UTC
lf-2-r.sd6 is back in production. We are moving on the isolation of lf-2-b.sd6
Posted Nov 15, 2023 - 08:47 UTC
We are resuming the node interfaces turnup on lf-2-r.sd6.
Posted Nov 15, 2023 - 08:40 UTC
Due to the PAAS incident at SD6 ( We have stopped the maintenance to avoid messing around at the same time.

We will continue tomorrow
Posted Nov 14, 2023 - 17:02 UTC
lf-2-r.sd6 is back online, uplinks are up
We are turning up the node links
Posted Nov 14, 2023 - 14:31 UTC
Switch swap is ongoing
Posted Nov 14, 2023 - 13:50 UTC
lf-2-r.sd6 is isolated, we are shutting it down
Posted Nov 14, 2023 - 13:46 UTC
lf-2-r.sd6 is being isolated
Posted Nov 14, 2023 - 13:27 UTC
lf-1-r.sd6 is in production, all active/standby and LAG LACP nodes are up as well.

We will move on with lf-2 shortly.
Posted Nov 14, 2023 - 12:18 UTC
MLAG is in good state, turning up the upplinks
Posted Nov 14, 2023 - 11:37 UTC
We are reloading lf-1-r.sd6 to make sure the MLAG setup is okay
Posted Nov 14, 2023 - 11:23 UTC
lf-1-r.sd6 replacement is on the way
Posted Nov 14, 2023 - 10:41 UTC
We are starting to isolate lf-1-r.sd6.
Posted Nov 14, 2023 - 08:50 UTC
We have successfully replaced one switch. It took more time than expected due to some logistics issues and rack layout.

We will continue tomorrow.
Posted Nov 13, 2023 - 16:58 UTC
The new switch is ready, we are recabling all the nodes and turning the interfaces up
Posted Nov 13, 2023 - 15:07 UTC
We are turning up the uplinks on the new switch
Posted Nov 13, 2023 - 14:54 UTC
We are shutting down lf-1-b.sd6 and replacing it.
Posted Nov 13, 2023 - 14:12 UTC
We are isolating `lf-1-b.sd6`
Posted Nov 13, 2023 - 10:39 UTC
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Nov 13, 2023 - 08:30 UTC
We are preparing the replacement our TOR switches at SD6.
We have 2 pairs of dell/cumulus switches running CLAG. One switch of each pair has a disk starting to show IO errors.

As it was in the roadmap to replace them with Arista, we will do it a the same time.

All nodes are dual-attached using either LAG or active/standby.

From our previous intervention, isolating one link to allow one switch removal is close to hitless (0 to 2 packet loss).
During the migration to the new Arista, 10 secs of forwarding loss is expected.
Posted Nov 13, 2023 - 08:26 UTC
This scheduled maintenance affected: Hosting (Simple Hosting FR-SD6, Cloud FR-SD6, GandiCloud VPS FR-SD6).