Arvados

The httplib2 project released a new version On Nov 2nd (https://pypi.org/project/httplib2/#history) and it broke our testing pipeline on the install phase:

======= install env
Requirement already satisfied: setuptools>=18.5 in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (40.8.0)
Requirement already satisfied: pip>=7 in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (18.1)
Requirement already satisfied: wheel in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (0.37.0)
Requirement already satisfied: PyYAML in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (6.0)
Requirement already satisfied: httplib2 in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (0.20.2)
Requirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2; python_version > "3.0" in /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages (from httplib2) (3.0.4)
...
Installed /home/lucas/.cache/arvados-build/VENV3DIR/lib/python3.7/site-packages/arvados_python_client-2.3.0.dev20211013202728-py3.7.egg
Processing dependencies for arvados-python-client==2.3.0.dev20211013202728
error: pyparsing 3.0.4 is installed but pyparsing<3,>=2.0.2 is required by {'packaging'}
Leaving behind temp dirs in /home/lucas/.cache/arvados-build
Fatal: installing PyYAML and sdk/python failed (encountered in install_env at /home/lucas/arvados/build/run-tests.sh line 667)

This problem sometimes isn't visible, mainly when the temp directory already exists from previous runs.
The way I was able to work around it is by setting an upper limit version on sdk/python/setup.py like this:

diff --git a/sdk/python/setup.py b/sdk/python/setup.py
index 8d637303b..f82d44ab6 100644
--- a/sdk/python/setup.py
+++ b/sdk/python/setup.py
@@ -50,7 +50,7 @@ setup(name='arvados-python-client',
           'future',
           'google-api-python-client >=1.6.2, <2',
           'google-auth<2',
-          'httplib2 >=0.9.2',
+          'httplib2 >=0.9.2, <0.20.2',
           'pycurl >=7.19.5.1',
           'ruamel.yaml >=0.15.54, <0.17.11',
           'setuptools',

But let's dig a little deeper to confirm this is a proper solution.