False positive network errors detection due to goggle CAPTCHA on reference site
Message boards :
Questions and problems :
False positive network errors detection due to goggle CAPTCHA on reference site
Message board moderation
| Author | Message |
|---|---|
|
Send message Joined: 29 Apr 19 Posts: 19 
|
Hello. I have notices a lot of such error on all of my BOINC machines (it just few examples from just one computer - actually there were much more of same errors on all my computers) : 14-Feb-2020 19:13:41 [---] Project communication failed: attempting access to reference site 14-Feb-2020 19:13:43 [---] BOINC can't access Internet - check network connection or proxy configuration. ---------- cut ---------- 15-Feb-2020 17:45:56 [---] Project communication failed: attempting access to reference site 15-Feb-2020 17:45:58 [---] BOINC can't access Internet - check network connection or proxy configuration. ---------- cut ---------- 17-Feb-2020 06:54:05 [---] Project communication failed: attempting access to reference site 17-Feb-2020 06:54:07 [---] BOINC can't access Internet - check network connection or proxy configuration. ---------- cut ---------- 19-Feb-2020 02:09:37 [---] Project communication failed: attempting access to reference site 19-Feb-2020 02:09:39 [---] Internet access OK - project servers may be temporarily down. ---------- cut ---------- 19-Feb-2020 02:10:25 [---] Project communication failed: attempting access to reference site 19-Feb-2020 02:10:26 [---] BOINC can't access Internet - check network connection or proxy configuration. ---------- cut ---------- 20-Feb-2020 22:53:19 [---] Project communication failed: attempting access to reference site 20-Feb-2020 22:53:20 [---] BOINC can't access Internet - check network connection or proxy configuration. BOINC thinks that there is no internet connection so delay all project requests(Backing off). But it all were false positives - connection is OK, i monitor it independently and know there were no connection issues at all that times. I opened "lookup_website.html" in BOINC folder (which AFAIK saves last result get from reference site) file and have found the reason: google banned BOINC clients as a "bot activity": ERROR: ID: 96008 ·  |
 Jord Send message Joined: 29 Aug 05 Posts: 15483 
|
BOINC thinks that there is no internet connection so delay all project requests(Backing off). But it all were false positives - connection is OK, i monitor it independently and know there were no connection issues at all that times.You may think there was an internet connection, but that doesn't mean BOINC can connect to project servers correctly. For a next round, please enable http_debug in the Advanced view Event Log Options menu (View, Advanced view, Options, Event Log options), communicate with one or more projects and put the output of that in a post. google banned BOINC clients as a "bot activity":We don't connect to Google, we just send some GET headers that way and see if we get an answer. If we do, the internet connection is OK, and if we don't it's not OK. Referencing PDW's log here which shows that sequence in full working order: 21/02/2020 16:55:56 | | [http] HTTP_OP::init_get(): http://www.google.com/ 21/02/2020 16:55:57 | | [http] [ID#0] Info: Connected to www.google.com (216.58.213.4) port 80 (#1468) 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: GET / HTTP/1.1 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: Host: www.google.com 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.22) 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: Accept: */* 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: Accept-Language: en_GB 21/02/2020 16:55:57 | | [http] [ID#0] Sent header to server: 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: HTTP/1.1 200 OK 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Date: Fri, 21 Feb 2020 16:55:49 GMT 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Expires: -1 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Cache-Control: private, max-age=0 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Content-Type: text/html; charset=ISO-8859-1 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info." 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Content-Encoding: gzip 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Server: gws 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Content-Length: 5106 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: X-XSS-Protection: 0 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: X-Frame-Options: SAMEORIGIN 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Set-Cookie: 1P_JAR=2020-02-21-16; expires=Sun, 22-Mar-2020 16:55:49 GMT; path=/; domain=.google.com; Secure 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: Set-Cookie: NID=198=BHNpVhOCa9Oq0BwuIKnxllyHPuRBBaYRDMelLxu_ynTNnogxMSIA0LH6fa695AjZGpiy-7bqsm4ILSieq_-yMuKIxHkJb_aVOitNPX8tdDsMAqrEWBLfg3W1JXKHNP4oogFsieHfAjohGn8G-OpN2_raEDupsv5_Dn9-9J-ONFY; expires=Sat, 22-Aug-2020 16:55:49 GMT; path=/; domain=.google.com; HttpOnly 21/02/2020 16:55:57 | | [http] [ID#0] Received header from server: 21/02/2020 16:55:57 | | [http] [ID#0] Info: Connection #1468 to host www.google.com left intact 21/02/2020 16:55:57 | | Internet access OK - project servers may be temporarily down. Since in that same thread you say you have set http_debug already, why not post the full outcome? ID: 96010 · |
|
Send message Joined: 29 Apr 19 Posts: 19
|
If BOINC does not actually connect to Google, how lookup_website.html file gets it content? If i delete it BOINC recreates this file after some time and it has fresh content inside each time - my IP and date/time of last access - all generated by Goggle and somehow downloaded and saved to disk by BOINC client. Here is contend of one o such files: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head><meta http-equiv="content-type" content="text/html; charset=utf-8"><meta name="viewport" content="initial-scale=1"><title>http://www.google.com/</title></head>
<body style="font-family: arial, sans-serif; background-color: #fff; color: #000; padding:20px; font-size:18px;" onload="e=document.getElementById('captcha');if(e){e.focus();}">
<div style="max-width:400px;">
<hr noshade size="1" style="color:#ccc; background-color:#ccc;"><br>
<form id="captcha-form" action="index" method="post">
<script src="https://www.google.com/recaptcha/api.js" async defer></script>
<script>var submitCallback = function(response) {document.getElementById('captcha-form').submit();};</script>
<div id="recaptcha" class="g-recaptcha" data-sitekey="6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b" data-callback="submitCallback" data-s="DoSc4_miAI-qVjUtEoUlnteeT9Q9Bf59QLu8jDTpyQ8CT90ihe7HbLY1pJGX2CFt9RvIz7YDJhdJABGnQKclpySQ0SbcW9sf0JwrcQ8W6OxQKapvD6sk_caRd43IZIp6JjqtzfDYXy8ta1w90ATF47xsn2T1QVhA5cGOBQr8iWOdx_wmeqWkVw9Pz7ri0alB7jb0T4f_O_uDeRgSlKzGgla9mp3fQ6DsET5X1HV02urupwe_fooQtrA"></div>
<input type='hidden' name='q' value='EgRZs32xGK_Hu_IFIhkA8aeDS4AG37biC27dRj3UUyv57cYQe7YNMgFy'><input type="hidden" name="continue" value="http://www.google.com/">
</form>
<hr noshade size="1" style="color:#ccc; background-color:#ccc;">
<div style="font-size:13px;">
<b>About this page</b><br><br>
Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot. <a href="#" onclick="document.getElementById('infoDiv').style.display='block';">Why did this happen?</a><br><br>
<div id="infoDiv" style="display:none; background-color:#eee; padding:10px; margin:0 0 15px 0; line-height:1.4em;">
This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the <a href="//www.google.com/policies/terms/">Terms of Service</a>. The block will expire shortly after those requests stop. In the meantime, solving the above CAPTCHA will let you continue to use our services.<br><br>This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests. If you share your network connection, ask your administrator for help — a different computer using the same IP address may be responsible. <a href="//support.google.com/websearch/answer/86640">Learn more</a><br><br>Sometimes you may be asked to solve the CAPTCHA if you are using advanced terms that robots are known to use, or sending requests very quickly.
</div>
IP address: 89.179.***.***<br>Time: 2020-02-20T19:53:20Z<br>URL: http://www.google.com/<br>
</div>
</div>
</body>
</html>As for http_debug i turned it on only on one of my computers. And already turned it off as it caused BOINC manager to work unstable as described in other thread. I searched if any reference site contacts were logged while http_debug was on but looks like last is already deleted by BOINC - stdoutdae.txt and stdoutdae.old contains only last ~30 hours and there were no reference site contacts during this time on that computer. Probable BOINC delete schedule of "old" logs is based not on time but on number of lines or file size. As on other machines where is no any debug flags set it > month worth of logs. Is any way to trigger this check intentionally without waiting for next http error on one of the attached projects? [/code] ID: 96014 · |
|
Send message Joined: 29 Apr 19 Posts: 19
|
I have caught it another time, now with http_debug on. No BOINC does not just get headers, it try to retrieve page, failed it (due to google ban bot activity) and declare that no internet connection is available (while it just actually get connection to goggle successfully). Because BOINC want only HTTP "200 - OK" server response as working internet connection, and interpret all other server responses (like 302, 403, 429 etc) as if there is not internet connection (lol, how it get these error codes from server then? ) 29/02/2020 21:32:35 | | Re-reading cc_config.xml 29/02/2020 21:32:35 | | Config: use all coprocessors 29/02/2020 21:32:35 | | log flags: file_xfer, sched_ops, task, http_debug 29/02/2020 21:32:35 | Rosetta@home | Found app_config.xml 29/02/2020 21:32:46 | Rosetta@home | [http] HTTP_OP::init_get(): http://boinc.bakerlab.org/rosetta/download/fc/rb_02_24_16848_16671_ab_t000__h002_robetta.zip 29/02/2020 21:32:46 | Rosetta@home | [http] HTTP_OP::libcurl_exec(): ca-bundle 'D:\Boinc\ca-bundle.crt' 29/02/2020 21:32:46 | Rosetta@home | [http] HTTP_OP::libcurl_exec(): ca-bundle set 29/02/2020 21:32:46 | Rosetta@home | Started download of rb_02_24_16848_16671_ab_t000__h002_robetta.zip 29/02/2020 21:32:46 | Rosetta@home | [http] [ID#10370] Info: Connection 4149 seems to be dead! 29/02/2020 21:32:46 | Rosetta@home | [http] [ID#10370] Info: Closing connection 4149 29/02/2020 21:32:46 | Rosetta@home | [http] [ID#10370] Info: Connection 4150 seems to be dead! 29/02/2020 21:32:46 | Rosetta@home | [http] [ID#10370] Info: Closing connection 4150 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Info: Trying 128.95.160.157... 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Info: Connected to boinc.bakerlab.org (128.95.160.157) port 80 (#4151) 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: GET /rosetta/download/fc/rb_02_24_16848_16671_ab_t000__h002_robetta.zip HTTP/1.1 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: Host: boinc.bakerlab.org 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.22) 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: Accept: */* 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: Accept-Encoding: deflate, gzip 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: Content-Type: application/x-www-form-urlencoded 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: Accept-Language: en_GB 29/02/2020 21:32:47 | Rosetta@home | [http] [ID#10370] Sent header to server: 29/02/2020 21:32:48 | Rosetta@home | [http] [ID#10370] Received header from server: HTTP/1.1 200 OK 29/02/2020 21:37:48 | Rosetta@home | [http] [ID#10370] Info: Operation too slow. Less than 10 bytes/sec transferred the last 300 seconds 29/02/2020 21:37:48 | Rosetta@home | [http] [ID#10370] Info: Closing connection 4151 29/02/2020 21:37:48 | Rosetta@home | [http] HTTP error: Timeout was reached 29/02/2020 21:37:48 | Rosetta@home | Temporarily failed download of rb_02_24_16848_16671_ab_t000__h002_robetta.zip: transient HTTP error 29/02/2020 21:37:48 | Rosetta@home | Backing off 03:06:40 on download of rb_02_24_16848_16671_ab_t000__h002_robetta.zip 29/02/2020 21:37:49 | | Project communication failed: attempting access to reference site 29/02/2020 21:37:49 | | [http] HTTP_OP::init_get(): http://www.google.com/ 29/02/2020 21:37:49 | | [http] HTTP_OP::libcurl_exec(): ca-bundle set 29/02/2020 21:37:50 | | [http] [ID#0] Info: Trying 64.233.165.105... 29/02/2020 21:37:50 | | [http] [ID#0] Info: Connected to www.google.com (64.233.165.105) port 80 (#4154) 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: GET / HTTP/1.1 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: Host: www.google.com 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.22) 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: Accept: */* 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: Accept-Language: en_GB 29/02/2020 21:37:50 | | [http] [ID#0] Sent header to server: 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: HTTP/1.1 302 Found 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Location: http://www.google.com/sorry/index?continue=http://www.google.com/&q=EgRZs32xGPXe6vIFIhkA8aeDS8WqECNzKWsTIfjUf6t7VcsFxSoZMgFy 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Date: Sat, 29 Feb 2020 18:37:42 GMT 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Pragma: no-cache 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Expires: Fri, 01 Jan 1990 00:00:00 GMT 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Cache-Control: no-store, no-cache, must-revalidate 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Content-Type: text/html; charset=UTF-8 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Server: HTTP server (unknown) 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Content-Length: 325 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: X-XSS-Protection: 0 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: 29/02/2020 21:37:51 | | [http] [ID#0] Info: Ignoring the response-body 29/02/2020 21:37:51 | | [http] [ID#0] Info: Connection #4154 to host www.google.com left intact 29/02/2020 21:37:51 | | [http] [ID#0] Info: Issue another request to this URL: 'http://www.google.com/sorry/index?continue=http://www.google.com/&q=EgRZs32xGPXe6vIFIhkA8aeDS8WqECNzKWsTIfjUf6t7VcsFxSoZMgFy' 29/02/2020 21:37:51 | | [http] [ID#0] Info: Found bundle for host www.google.com: 0x3b7fc10 29/02/2020 21:37:51 | | [http] [ID#0] Info: Re-using existing connection! (#4154) with host www.google.com 29/02/2020 21:37:51 | | [http] [ID#0] Info: Connected to www.google.com (64.233.165.105) port 80 (#4154) 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: GET /sorry/index?continue=http://www.google.com/&q=EgRZs32xGPXe6vIFIhkA8aeDS8WqECNzKWsTIfjUf6t7VcsFxSoZMgFy HTTP/1.1 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Host: www.google.com 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.22) 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Accept: */* 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Referer: http://www.google.com/ 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: Accept-Language: en_GB 29/02/2020 21:37:51 | | [http] [ID#0] Sent header to server: 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: HTTP/1.1 429 Too Many Requests 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Date: Sat, 29 Feb 2020 18:37:42 GMT 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Pragma: no-cache 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Expires: Fri, 01 Jan 1990 00:00:00 GMT 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Cache-Control: no-store, no-cache, must-revalidate 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Content-Type: text/html 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Server: HTTP server (unknown) 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: Content-Length: 2798 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: X-XSS-Protection: 0 29/02/2020 21:37:51 | | [http] [ID#0] Received header from server: 29/02/2020 21:37:51 | | [http] [ID#0] Info: Connection #4154 to host www.google.com left intact 29/02/2020 21:37:51 | | BOINC can't access Internet - check network connection or proxy configuration. Also page it get from redirect URL ( http://www.google.com/sorry/index?continue=http://www.google.com/&q=EgRZs32xGPXe6vIFIhkA8aeDS8WqECNzKWsTIfjUf6t7VcsFxSoZMgFy) was saved to lookup_website.html file again. Now i know why Google so often ask me solve CAPTCHA and block me for a "bot activity" on my personal computer - it because of few BOINC clients in the same home network with same public IP keep triggering it. ID: 96211 · |
|
Send message Joined: 25 Nov 05 Posts: 1654 
|
29/02/2020 21:37:48 | Rosetta@home | [http] [ID#10370] Info: Operation too slow. Less than 10 bytes/sec transferred the last 300 seconds Something's wrong at Rosetta. ID: 96219 · |
|
Send message Joined: 29 Apr 19 Posts: 19
|
Yes, definitely something wrong at Rosetta (their servers can not send some of files properly) AND something wrong with BOINC client at same time. Because after communication error with one of attached projects (R@H in this example but it can be any other) it try to check Internet connection by "pinging" google but handle goggle responses wrong and declare internet connection is down while it actually works fine (and issues only with just one file from one of the projects, not with internet connection): 29/02/2020 21:37:51 | | BOINC can't access Internet - check network connection or proxy configuration. ID: 96221 · |
|
Send message Joined: 29 Apr 19 Posts: 19
|
Catch it again on another computer with other (never and latest stable release - 7.14.2 ) BOINC client version. Much more detail in debug in this BOINC ver but exactly same behavior and result - any responses from google other than "200 - OK" is interpreted for some reason as the lack of an Internet connection. Although the simple fact of receiving at least some kind of response (including error codes from the server) means the exactly opposite - that the Internet connection is in order and these are just some errors on the server’s side (in this case, Google banned my ip again due to many BOINC clients often send requests to it and trigger google anti-bot protection). 01/03/2020 00:48:06 | Rosetta@home | [http] [ID#10522] Info: Operation too slow. Less than 10 bytes/sec transferred the last 300 seconds ID: 96224 · |
|
Send message Joined: 5 Oct 06 Posts: 5082 
|
Tested with a later test version - Windows v7.16.5. Internet availablity test worked properly from here in the UK, but I don't think that's due to any change in BOINC - more likely a difference in our patterns of use. I'm a regular user of Google Chrome, so Google sees my machines quite a lot and knows I'm not a robot. 01/03/2020 08:21:59 | | Project communication failed: attempting access to reference site ID: 96232 · |
Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.