Project

General

Profile

Actions

Bug #23107

closed

Collection.update does not reuse API client

Added by Moritz Gilsdorf 7 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
SDKs
Target version:
Story points:
-
Release relationship:
Auto

Description

I experienced an issue saving a collection after a file upload using a Collection object initialized with a API client. After downgrading the SDK to version 3.0.0 it works so I assume this is a regression bug.

The used Arvados Instance is also running on version 3.1.2.

import shutil

import arvados
import arvados.collection

api = arvados.api(
    "v1",
    token="TOKEN",
    host="API_HOST",
)

collection = arvados.collection.Collection(
    manifest_locator_or_text="COLLECTION_UUID", api_client=api
)
print(collection.api_response())

with (
    open("some_file", "rb") as src,
    collection.open("some_file", "wb") as dst,
):
    shutil.copyfileobj(src, dst)

collection.save()

The line print(collection.api_response()) confirms the connection is working.

Complete Traceback

Traceback (most recent call last):
  File "/home/gitpod/.pyenv/versions/3.10.18/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/gitpod/.pyenv/versions/3.10.18/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 71, in <module>
    cli.main()
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 501, in main
    run()
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 351, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 310, in run_path
    return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 127, in _run_module_code
    _run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
  File "/home/gitpod/.vscode-server/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 118, in _run_code
    exec(code, run_globals)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/test.py", line 23, in <module>
    collection.save()
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/arvfile.py", line 411, in must_be_writable_wrapper
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/arvfile.py", line 212, in synchronized_wrapper
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/retry.py", line 245, in num_retries_setter
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/collection.py", line 1556, in save
    self.update()
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/arvfile.py", line 212, in synchronized_wrapper
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/retry.py", line 245, in num_retries_setter
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/collection.py", line 1284, in update
    self.apply(baseline.diff(other))
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/arvfile.py", line 212, in synchronized_wrapper
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/collection.py", line 856, in diff
    holding_collection = Collection(api_client=self._my_api(), keep_client=self._my_keep())
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/arvfile.py", line 212, in synchronized_wrapper
    return orig_func(self, *args, **kwargs)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/collection.py", line 1294, in _my_api
    self._api_client = ThreadSafeAPIClient(self._config, version='v1')
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/api.py", line 290, in __init__
    self._api_kwargs = normalize_api_kwargs(version, **api_params)
  File "/workspace/hamsters-dwd-workspace-gitpod/resources/arvados-autolab-components/.venv/lib/python3.10/site-packages/arvados/api.py", line 488, in normalize_api_kwargs
    raise ValueError("neither discoveryServiceUrl nor host provided")
ValueError: neither discoveryServiceUrl nor host provided


Subtasks 1 (0 open1 closed)

Task #23212: Review 23107-api-passthroughResolvedBrett Smith10/15/2025Actions
Actions #1

Updated by Brett Smith 7 months ago

  • Release set to 79
Actions #2

Updated by Brett Smith 7 months ago

  • Target version set to Development 2025-09-03
Actions #3

Updated by Brett Smith 7 months ago

  • Release changed from 79 to 83
  • Target version deleted (Development 2025-09-03)
Actions #4

Updated by Brett Smith 5 months ago

  • Release changed from 83 to 79
  • Target version set to Development 2025-10-29
  • Assigned To set to Brett Smith
  • Status changed from New to In Progress
  • Subject changed from "ValueError: neither discoveryServiceUrl nor host provided" on collection.save() in 3.1.2 to Collection.update does not reuse API client
Actions #5

Updated by Brett Smith 5 months ago

23107-api-passthrough @ 1f7949fd27ecc003bb36dc0c3f4c884f3c19f457 - developer-run-tests: #4908

Very straightforward bugfix. I hope the commit message+tests make the issue clear.

  • All agreed upon points are implemented / addressed. Describe changes from pre-implementation design.
    • Yes
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • N/A
  • Code is tested and passing, both automated and manual, what manual testing was done is described.
    • See above. Added several tests to prevent regressions.
  • Tested code incorporates recent main branch changes.
    • Yes
  • New or changed UI/UX and has gotten feedback from stakeholders.
    • N/A
  • Documentation has been updated.
    • N/A
  • Behaves appropriately at the intended scale (describe intended scale).
    • N/A
  • Considered backwards and forwards compatibility issues between client and server.
    • N/A
  • Follows our coding standards and GUI style guidelines.
    • Yes
Actions #6

Updated by Brett Smith 5 months ago

  • Subtask #23212 added
Actions #7

Updated by Tom Clegg 5 months ago

23107-api-passthrough LGTM, thanks

Actions #8

Updated by Brett Smith 5 months ago

  • Target version changed from Development 2025-10-29 to Development 2025-10-15
Actions #9

Updated by Brett Smith 5 months ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF