Arvados

22958-ansible-docker-package-builds @ 789804194674e671dddfb56cff5f124558da3780 - build-packages-rocky9: #3

• All agreed upon points are implemented / addressed. Describe changes from pre-implementation design.
  • Yes. The branch splits up the big development playbook to assign different host groups to different dependecy tasks. The main groups in use now are arvados_build_one_target (for building a packaging Docker image) and arvados_test_all (to install everything needed for Arvados development).
• Anything not implemented (discovered or discussed during work) has a follow-up story.
  • N/A
• Code is tested and passing, both automated and manual, what manual testing was done is described.
  • See above. Manually tested ubuntu2004 and rocky8 targets.
• New or changed UI/UX and has gotten feedback from stakeholders.
  • N/A
• Documentation has been updated.
  • Yes.
• Behaves appropriately at the intended scale (describe intended scale).
  • As before, this build is probably slower due to the overhead of Ansible, but we're paying that price for code deduplication.
• Considered backwards and forwards compatibility issues between client and server.
  • This does rename the main playbook to better reflect its expanded scope. This change will need to be carried over to the Jenkins Ansible stuff. That's a straightforward search-and-replace.
• Follows our coding standards and GUI style guidelines.
  • N/A

Here're some questions & comments:

• File tools/ansible/install-dev-tools.yml Line 26: Trailing colon typo on the example comment.
• File tools/ansible/files/default-test-config.yml still mentions the old playbook name install-test-env.yml
• File build/README might need an update: Mentioning ansible as a prerequisite will be helpful.
• Got the following error while trying run-build-test-package-one-target.sh:
  • ModuleNotFoundError: No module named 'requests'
  • Fixed by installing it manually via pip inside the ansible virtualenv
• (Only when using --target ubuntu2204, rockyN or other ubuntuNNNN versions don't seem to have issues) cgo: C compiler "aarch64-linux-gnu-gcc" not found: exec: "aarch64-linux-gnu-gcc": executable file not found in $PATH
  • This happens when no --arch=X is used or when passing --arch=arm64, obviously.
  • The args help say that when no --arch=X is passed, it will build the packages in the native architecture, so maybe something got lost in the way (or this documentation was already wrong)

Lucas Di Pentima wrote in #note-5:

• File tools/ansible/install-dev-tools.yml Line 26: Trailing colon typo on the example comment.

No, that colon is important. Ansible expects hosts to be a hash where every key is a hostname and every value is a hash of host variables, which can optionally be null. The trailing colon is what makes the hostname a dictionary key and not a string. See the Ansible inventory documentation for examples, including exactly this.

• File tools/ansible/files/default-test-config.yml still mentions the old playbook name install-test-env.yml
• File build/README might need an update: Mentioning ansible as a prerequisite will be helpful.

Done.

• Got the following error while trying run-build-test-package-one-target.sh:
  • ModuleNotFoundError: No module named 'requests'

I can't reproduce if I follow the instructions in tools/ansible/README.md. Notice requests mentioned in the last line here:

arvdev % py -m venv ~/Scratch/venv
arvdev % ~/Scratch/venv/bin/pip install -r tools/ansible/requirements.txt
[…]
Successfully installed MarkupSafe-3.0.2 PyYAML-6.0.2 ansible-core-2.15.13 argcomplete-3.6.2 certifi-2025.4.26 cffi-1.17.1 charset_normalizer-3.4.2 cryptography-45.0.4 docker-7.1.0 idna-3.10 jinja2-3.1.6 packaging-25.0 pycparser-2.22 requests-2.32.4 resolvelib-1.0.1 tomlkit-0.13.3 urllib3-2.4.0 xmltodict-0.14.2 yq-3.4.3

Did you do the same? Hopefully the pointer in build/README will help with this.

• (Only when using --target ubuntu2204, rockyN or other ubuntuNNNN versions don't seem to have issues) cgo: C compiler "aarch64-linux-gnu-gcc" not found: exec: "aarch64-linux-gnu-gcc": executable file not found in $PATH

We discussed at standup that we were okay dropping support for arm64 packages, since they've never been in production or fully baked. I have updated the documentation and build scripts to account for this change.

Now at 7058804ef5c767a1791f857efe632b682c815c06. build-packages-rocky9: #4

Brett Smith wrote in #note-6:

• ModuleNotFoundError: No module named 'requests'

I can't reproduce if I follow the instructions in tools/ansible/README.md. Notice requests mentioned in the last line here:

[...]

Did you do the same? Hopefully the pointer in build/README will help with this.

Sorry, was using a previous virtualenv that hadn't all the dependencies installed.

Now at 7058804ef5c767a1791f857efe632b682c815c06. build-packages-rocky9: #4

This LGTM, thanks!