That is part of the problem, I seem to not be able to reproduce the issue if I want to. I hammer the sites and systems in the containers with updates at the same time, nothing happens. But then just when I wrote here the other day and loaded webmin in 1 or 2 containers my good old friend 522 came back.
I since suspect that the new webmin stats history are too much for the systems and temporarily overload. But then why can I usually access the containers directly long before they come back through cloudflare? The mistery continues.
I will take your comment though as a tip to test. Will shut off ufw everywhere and hope I won't get attacked.
I do not get any block messages besides some ipv4 addresses in the ufw log though...
Thanks @tersor !