Arvados

14947-run-tests-interactive @ 0be3e2c040d8d785bbd5c3e8d9de62c0fbaee0ec

Example

~/arvados/build/run-tests.sh WORKSPACE=~/arvados --temp ~/.cache/arvados-build --interactive
...

What next? test lib/cmd
======= test lib/cmd
ok      git.curoverse.com/arvados.git/lib/cmd   0.002s  coverage: 77.5% of statements
======= test lib/cmd -- 0s
What next? test lib/dispatchcloud
Starting API, keepproxy, keep-web, ws, arv-git-httpd, and nginx ssl proxy...
...
ARVADOS_TEST_API_HOST=0.0.0.0:51399
======= test lib/dispatchcloud
ok      git.curoverse.com/arvados.git/lib/dispatchcloud 0.915s  coverage: 73.2% of statements
======= test lib/dispatchcloud -- 3s
What next? help
== Interactive commands:
dir           (short for 'test dir')
test dir
test dir:py3  (test with python3)
install dir
install deps  (go/python libs)
reset         (...services used by integration tests)
exit
== Test targets:
cmd/arvados-client              lib/dispatchcloud/worker        sdk/go/manifest                 services/keep-balance
cmd/arvados-server              sdk/cwl                         sdk/go/stats                    services/keepproxy
lib/cli                         sdk/cwl:py3                     sdk/pam                         services/keepstore
lib/cloud                       sdk/go/arvados                  sdk/python                      services/keep-web
lib/cloud/azure                 sdk/go/arvadosclient            sdk/python:py3                  services/nodemanager
lib/cloud/ec2                   sdk/go/asyncbuf                 services/arv-git-httpd          services/ws
lib/cmd                         sdk/go/auth                     services/crunch-dispatch-local  tools/crunchstat-summary
lib/controller                  sdk/go/blockdigest              services/crunch-dispatch-slurm  tools/keep-block-check
lib/crunchstat                  sdk/go/crunchrunner             services/crunch-run             tools/keep-exercise
lib/dispatchcloud               sdk/go/dispatch                 services/crunchstat             tools/keep-rsync
lib/dispatchcloud/container     sdk/go/health                   services/dockercleaner:py3      tools/sync-groups
lib/dispatchcloud/scheduler     sdk/go/httpserver               services/fuse
lib/dispatchcloud/ssh_executor  sdk/go/keepclient               services/health

• --skip-install (and alt spelling --skip install) skips installing dependencies, too
• --skip sanity skips dev env sanity checks (saves 1-2 seconds)
• "starting/skipping/finished X" banners are much less verbose

https://ci.curoverse.com/view/Developer/job/developer-run-tests/1128/

It seems like whenever I exclude the --temp flag I get the following error or something similar:

Starting API, keepproxy, keep-web, ws, arv-git-httpd, and nginx ssl proxy...
sdk/python/tests/run_test_server.py:772: YAMLLoadWarning: calling yaml.load() without Loader=... 
is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
sdk/python/tests/run_test_server.py:742: YAMLLoadWarning: calling yaml.load() without Loader=... 
is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  SERVICES_SRC_DIR, 'api', 'config', 'database.yml')))
Traceback (most recent call last):
  File "sdk/python/tests/run_test_server.py", line 887, in <module>
    run_controller()
  File "sdk/python/tests/run_test_server.py", line 443, in run_controller
    stdin=open('/dev/null'), stdout=logf, stderr=logf, close_fds=True)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
./build/run-tests.sh: line 392: deactivate: command not found
======= test sdk/python -- failed to start services
./build/run-tests.sh: line 404: /tmp/tmp.s59VFWfgvG/VENVDIR/bin/activate: No such file or directory
Sent SIGTERM to 7653 (/home/eric/projects/arvados/tmp/api.pid)
./build/run-tests.sh: line 414: deactivate: command not found
All test suites passed.

• previous "install deps" is now called "install env"
• new "install deps" includes apiserver, keepproxy, etc. (things that need to be installed for tests to work)
• default interactive command is "install deps" if your temp dir hasn't been set up yet

• suppress the "all test suites passed" banner when no tests ran
• error out earlier if python "activate" fails (e.g., testing anything before installing env or deps)

14947-run-tests-interactive @ 19ab7c19fcd56b05ff2d06f7fb4b66a515c90f4b https://ci.curoverse.com/view/Developer/job/developer-run-tests/1135/

When using install deps, during services/api install I get the following (shortened by me):

Called from /home/eric/.rvm/gems/ruby-2.3.8@arvados-tests-869b3fba62569460/gems/activesupport-4.2.11/lib/
active_support/dependencies.rb:240:in `load_dependency'
FATAL:  password authentication failed for user "arvados" 
FATAL:  password authentication failed for user "arvados" 
/home/eric/.rvm/gems/ruby-2.3.8@arvados-tests-869b3fba62569460/gems/activerecord-4.2.11/lib/active_record/
connection_adapters/postgresql_adapter.rb:651:in `initialize'
.
.
.
Couldn't drop arvados_test
Defaulting to memory cache, because /home/eric/projects/arvados/services/api/tmp/cache does not exist

Maybe my setup is wrong? I'm pretty sure I have database.yml set up correctly. I'm getting the same problem using arvbox. Let me know if you want the full stack trace.

FATAL:  password authentication failed for user "arvados"

Hm. This looks like a real password mismatch, but you can check by getting the password from the "test" section of services/api/config/database.yml and pasting it to the psql prompt:

psql -U arvados -W -c 'select 1'

14947-run-tests-interactive @ e6f350e56674e3f19d24755b94a22e59cf8478b6

What next? test sdk/python --test-suite=tests.test_arvfile.ArvadosFileWriterTestCase.test_truncate
...
running build_ext
test_truncate (tests.test_arvfile.ArvadosFileWriterTestCase) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK