Hello,
I've just noticed that one of our high-visit WP websites is frequently going down due to facebook crawler bombarding our access log file.
We're using WO with wpredis + CloudFlare with APO. However, all the requests see to be MISS and not getting cached properly as well.
Those are coming from IPs such as:
2a03:2880:11ff:70::face:b00c MISS
2a03:2880:11ff:76::face:b00c MISS
2a03:2880:11ff:2::face:b00c MISS
2a03:2880:11ff:b::face:b00c MISS
2a03:2880:11ff:17::face:b00c MISS
2a03:2880:31ff:10::face:b00c MISS
2a03:2880:11ff:c::face:b00c MISS
2a03:2880:10ff:71::face:b00c MISS
2a03:2880:11ff:10::face:b00c MISS
And full log line example:
2a03:2880:11ff:14::face:b00c 0.304 MISS [17/Jan/2021:00:25:47 +0000] DOMAIN "GET URI HTTP/1.1" 301 198 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
Using Wordfence I tried to block them, but it doesn't seem to work.
Basically this is causing massive load on the server and random issues.
Any ideas welcome so I can figure this thing out. Thank you.