Actions
Bug #8090
closedBroken pipe on qr1hi
Status:
Closed
Priority:
Normal
Assigned To:
-
Category:
-
Target version:
-
Story points:
-
Description
https://cloud.curoverse.com/collections/bf9fbb64de0fd5e21bb845082e37bbf1+174/qr1hi-8i9sb-e6n3aji6gtjk9ro.log.txt
https://cloud.curoverse.com/collections/40c21fa3b284a960b2dd9960f40540e2+89/qr1hi-8i9sb-8sadq9o6sr4p3c1.log.txt
2015-12-30_02:09:14 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr Traceback (most recent call last): 2015-12-30_02:09:14 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job/src/crunch_scripts/run-command", line 445, in <module> 2015-12-30_02:09:14 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr 'progress':1.0 2015-12-30_02:09:14 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/oauth2client/util.py", line 140, in positional_wrapper 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr return wrapped(*args, **kwargs) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/googleapiclient/http.py", line 722, in execute 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr body=self.body, headers=self.headers) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/arvados/api.py", line 54, in _intercept_http_request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr return self.orig_http_request(uri, **kwargs) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1609, in request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1351, in _request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr (response, content) = self._conn_request(conn, request_uri, method, body, headers) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/tmp/crunch-job-work/.arvados.venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1273, in _conn_request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr conn.request(method, request_uri, body, headers) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/httplib.py", line 962, in request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr self._send_request(method, url, body, headers) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/httplib.py", line 996, in _send_request 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr self.endheaders(body) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/httplib.py", line 958, in endheaders 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr self._send_output(message_body) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/httplib.py", line 818, in _send_output 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr self.send(msg) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/httplib.py", line 794, in send 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr self.sock.sendall(data) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/ssl.py", line 229, in sendall 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr v = self.send(data[count:]) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr File "/usr/lib/python2.7/ssl.py", line 198, in send 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr v = self._sslobj.write(data) 2015-12-30_02:09:15 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr socket.error: [Errno 32] Broken pipe 2015-12-30_02:09:16 qr1hi-8i9sb-e6n3aji6gtjk9ro 24024 0 stderr skippinsrun: error: compute8: task 0: Exited with exit code 1
Updated by Brett Smith about 10 years ago
Both of these jobs use a Python SDK that predates the retry fix from #7587. Is it possible to upgrade them? If it works, 637b32482c7fd0fc14197b120b3657793646216f would be an ideal version.
Updated by Bryan Cosca about 10 years ago
Yep, trying 637b32482c7fd0fc14197b120b3657793646216f. Thanks!
Actions