Are You a Bot?
Let's talk Captchas for a second. They suck even when you're human. Use a VPN? Captcha. Privacy plugins? Captcha. Botting, scrapping, etc? Oh yeah. So when are you getting a captcha? Well I recently came across this handy dandy website which gives you a captcha score. Normally, for an upstanding citizen you're looking at a score between 0.7-1. But, as long as you are not under the websites threshold it doesn't really matter.
"success": true,
"hostname": "recaptcha-demo.appspot.com",
"challenge_ts": "2020-07-24T__:__:__Z",
"apk_package_name": null,
"score": 0.7,
"action": "examples/v3scores",
"error-codes": []
}
Let's Load Selenium
OUCH. As you can see a 0.1 is pretty bad and doesn't meet the threshold. You'll definitely get hit by a captcha for whatever you're doing. Let's do a few things to make selenium seem a bit friendlier.
First of all.... Let's tell WebDriver to not fucking tell the world you're a bot.
driver.ExecuteChromeCommand("Page.addScriptToEvaluateOnNewDocument",
new System.Collections.Generic.Dictionary<string, object> {
{ "source", @"
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
})"
}
}
);
Thanks WebDriver! 🙄
- Browse a friendly site and get a couple cookies. Maybe a minute?
- Search something from a 'good' IP, aka don't get flagged for your shitty free proxy/vpn.
- Don't go directly there. Use the website's flow. Site Home -> Link. Better yet. Google -> Site Home -> Link. Bonus points if they reliabily have an ad!
Let's see how we did?
Look at that. A respectable 0.9, no one is flagging our actions! And as you can see from the timestamp it was roughly 10 min for me to code a quick 'action routine' to be a good upstanding internet citizen, run the code, and hit the target page.