Project

General

Profile

Actions

Feature #22957

closed

Build package test Docker images with Ansible

Added by Brett Smith 10 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
CI
Target version:
Story points:
-
Release relationship:
Auto

Description

Spun out of #22943. Do this first because it requires the least playbook work, but will introduce a lot of the infrastructure.

  • Write a playbook to do what the package-test-dockerfiles do (this might literally only be installing the packages required by our package test scripts, and then package repository)
  • Finish the playbook to build Docker images
  • Write an inventory that defines base images and tags for the package test images
  • Update our package test scripts to build these images with Ansible instead of docker build
  • Update build/README to describe how to add a new test image

This ticket means our Ansible deployment is going to start having dependencies on pip modules and Ansible collections. Aim to declare these in requirements-like files to give us the most flexibility to bootstrap an Ansible installation in different environments.


Subtasks 1 (0 open1 closed)

Task #22962: Review 22957-ansible-package-testsResolvedBrett Smith06/03/2025Actions

Related issues 1 (0 open1 closed)

Blocked by Arvados - Bug #22944: Ansible bootstrap tasks aren't bootstrappy enoughResolvedBrett SmithActions
Actions #1

Updated by Brett Smith 10 months ago

  • Subtask #22962 added
Actions #2

Updated by Brett Smith 10 months ago

  • Status changed from New to In Progress
Actions #3

Updated by Brett Smith 10 months ago

  • Blocked by Bug #22944: Ansible bootstrap tasks aren't bootstrappy enough added
Actions #4

Updated by Brett Smith 10 months ago

22957-ansible-package-tests @ ff946732fcf7a6dfb85f9fd5f7486d76fd4d99ca

  • All agreed upon points are implemented / addressed. Describe changes from pre-implementation design.
    • Yes
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • See below
  • Code is tested and passing, both automated and manual, what manual testing was done is described.
    • For now I have manually tested by building and testing packages on both ubuntu2204 and rocky9. This covers both Debian and Red Hat, and an older and newer distribution. I can't test on Jenkins right now because, basically, I will need to build a new Jenkins test image with changes in this branch. So you should at least look at it and make sure you're roughly okay with it before we do automated testing. If you are, then next steps are:
      1. Update arvados-dev/ansible to remove the old ci_ansible role and use the new arvados_ansible role instead. The old role can be removed completely from the tests node playbook, because the new one will be run as part of install-test-env.yml instead.
      2. Build a new node from the branch.
      3. Run a package build+test from the branch on the new node. If this passes then we're good.
  • New or changed UI/UX and has gotten feedback from stakeholders.
    • N/A
  • Documentation has been updated.
    • Yes
    • Note I have not updated the Ansible install instructions for building a compute node. That is okay because for now the old instructions will still work. I would like to get farther, and at least do #22958, before we update those instructions. This is early enough I'm worried that there might be more churn with further development. I don't want to rewrite docs only to rewrite them again next branch.
  • Behaves appropriately at the intended scale (describe intended scale).
    • This will probably run slower than the Dockerfile builds, but we're intentionally making that trade for code deduplication.
  • Considered backwards and forwards compatibility issues between client and server.
    • N/A, pure build change
  • Follows our coding standards and GUI style guidelines.
    • N/A, no Ansible style guide
Actions #5

Updated by Lucas Di Pentima 10 months ago

Very interesting read. I did a test run on my side to see how it went with Ubuntu 24.04, all good.

LGTM, thanks

Actions #6

Updated by Brett Smith 10 months ago

packer-build-jenkins-image-arvados-tests: #123

developer-run-tests: #4802 - Failed only on the "cannot mmap empty file" issue, looks good

build-packages-rocky9: #2 - Confirmed you can see the output from Ansible building the test Docker image

I'm not going to run any other build-packages jobs because of #22858 but I'm calling this good.

Actions #7

Updated by Brett Smith 10 months ago

  • Status changed from In Progress to Resolved
Actions #8

Updated by Brett Smith 6 months ago

  • Release set to 79
Actions

Also available in: Atom PDF