Turns out the spat with the neighbours happened after the Alitos flew the flag upside down. According to the NYT at least.
If so, Alito must go. Not only for flying the flag for the insurrectionists. But also for making things up off the cuff upon being discovered and for blaming his wife. This isn't the behaviour of a SCOTUS justice.
He no longer has an explanation for raising the flag. Nor will reasonable people believe him if he offers a new explanation.
He's also obliterated SCOTUS' reputation.