Actions
Support #16598
closed
Make sure Hacking_prerequisites is up-to-date
Added by Nico César over 4 years ago. Updated over 4 years ago.
Status:
Closed
Priority:
Normal
Assigned To:
Category:
Documentation
Target version:
Start date:
09/23/2020
Due date:
% Done:
100%
Estimated time:
(Total: 0.00 h)
Story points:
-
Updated by Nico César over 4 years ago
Starting from https://dev.arvados.org/projects/arvados/wiki/Hacking_prerequisites/71:
root@a88a5742be7f:~# apt-get update
Get:1 http://deb.debian.org/debian buster InRelease [121 kB]
Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [208 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7905 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [7868 B]
Fetched 8360 kB in 4s (2048 kB/s)
Reading package lists... Done
root@a88a5742be7f:~# apt-get -y install sudo
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
lsb-base
The following NEW packages will be installed:
lsb-base sudo
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1273 kB of archives.
After this operation, 3928 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 lsb-base all 10.2019051400 [28.4 kB]
Get:2 http://deb.debian.org/debian buster/main amd64 sudo amd64 1.8.27-1+deb10u2 [1245 kB]
Fetched 1273 kB in 1s (2214 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package lsb-base.
(Reading database ... 6677 files and directories currently installed.)
Preparing to unpack .../lsb-base_10.2019051400_all.deb ...
Unpacking lsb-base (10.2019051400) ...
Selecting previously unselected package sudo.
Preparing to unpack .../sudo_1.8.27-1+deb10u2_amd64.deb ...
Unpacking sudo (1.8.27-1+deb10u2) ...
Setting up lsb-base (10.2019051400) ...
Setting up sudo (1.8.27-1+deb10u2) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
root@a88a5742be7f:~# adduser me
Adding user `me' ...
Adding new group `me' (1000) ...
Adding new user `me' (1000) with group `me' ...
Creating home directory `/home/me' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for me
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n]
Installing prerequisites:
root@a88a5742be7f:~# apt-get install --no-install-recommends build-essential ca-certificates git golang libpam0g-dev Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu bzip2 cpp cpp-8 dpkg-dev g++ g++-8 gcc gcc-8 git-man golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc golang-go golang-src libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcurl3-gnutls libdpkg-perl liberror-perl libexpat1 libgcc-8-dev libgdbm-compat4 libgdbm6 libgomp1 libgssapi-krb5-2 libisl19 libitm1 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common liblsan0 libmpc3 libmpfr6 libmpx2 libnghttp2-14 libpcre2-8-0 libperl5.28 libpsl5 libquadmath0 librtmp1 libsasl2-2 libsasl2-modules-db libssh2-1 libssl1.1 libstdc++-8-dev libtsan0 libubsan1 linux-libc-dev make openssl patch perl perl-modules-5.28 xz-utils Suggested packages: binutils-doc bzip2-doc cpp-doc gcc-8-locales debian-keyring g++-multilib g++-8-multilib gcc-8-doc libstdc++6-8-dbg gcc-multilib manpages-dev autoconf automake libtool flex bison gdb gcc-doc gcc-8-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg libquadmath0-dbg gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn bzr | brz mercurial subversion glibc-doc gnupg | gnupg2 sensible-utils bzr gdbm-l10n krb5-doc krb5-user libstdc++-8-doc make-doc ed diffutils-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl libb-debug-perl liblocale-codes-perl Recommended packages: fakeroot gnupg | gnupg2 libalgorithm-merge-perl less ssh-client pkg-config manpages manpages-dev libfile-fcntllock-perl liblocale-gettext-perl krb5-locales publicsuffix libsasl2-modules netbase The following NEW packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu build-essential bzip2 ca-certificates cpp cpp-8 dpkg-dev g++ g++-8 gcc gcc-8 git git-man golang golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc golang-go golang-src libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libcurl3-gnutls libdpkg-perl liberror-perl libexpat1 libgcc-8-dev libgdbm-compat4 libgdbm6 libgomp1 libgssapi-krb5-2 libisl19 libitm1 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common liblsan0 libmpc3 libmpfr6 libmpx2 libnghttp2-14 libpam0g-dev libpcre2-8-0 libperl5.28 libpsl5 libquadmath0 librtmp1 libsasl2-2 libsasl2-modules-db libssh2-1 libssl1.1 libstdc++-8-dev libtsan0 libubsan1 linux-libc-dev make openssl patch perl perl-modules-5.28 xz-utils 0 upgraded, 71 newly installed, 0 to remove and 0 not upgraded. Need to get 124 MB of archives. After this operation, 576 MB of additional disk space will be used. Do you want to continue? [Y/n] Y .... Setting up perl-modules-5.28 (5.28.1-6) ... Setting up libexpat1:amd64 (2.2.6-2+deb10u1) ... Setting up libkeyutils1:amd64 (1.6-6) ... Setting up libpsl5:amd64 (0.20.2-2) ... Setting up golang-1.11-src (1.11.6-1+deb10u3) ... Setting up libssl1.1:amd64 (1.1.1d-0+deb10u3) ... debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.) debconf: falling back to frontend: Readline Setting up binutils-common:amd64 (2.31.1-16) ... Setting up libnghttp2-14:amd64 (1.36.0-2+deb10u1) ... Setting up linux-libc-dev:amd64 (4.19.118-2+deb10u1) ... Setting up libgomp1:amd64 (8.3.0-6) ... Setting up bzip2 (1.0.6-9.2~deb10u1) ... Setting up libldap-common (2.4.47+dfsg-3+deb10u2) ... Setting up libkrb5support0:amd64 (1.17-3) ... Setting up golang-1.11-go (1.11.6-1+deb10u3) ... Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-1+deb10u1) ... Setting up libasan5:amd64 (8.3.0-6) ... Setting up make (4.2.1-1.2) ... Setting up libmpfr6:amd64 (4.0.2-1) ... Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2) ... Setting up xz-utils (5.2.4-1) ... update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode Setting up libquadmath0:amd64 (8.3.0-6) ... Setting up libmpc3:amd64 (1.1.0-1) ... Setting up libatomic1:amd64 (8.3.0-6) ... Setting up patch (2.7.6-3+deb10u1) ... Setting up libpcre2-8-0:amd64 (10.32-5) ... Setting up libk5crypto3:amd64 (1.17-3) ... Setting up libsasl2-2:amd64 (2.1.27+dfsg-1+deb10u1) ... Setting up libmpx2:amd64 (8.3.0-6) ... Setting up libubsan1:amd64 (8.3.0-6) ... Setting up libisl19:amd64 (0.20-2) ... Setting up git-man (1:2.20.1-2+deb10u3) ... Setting up libssh2-1:amd64 (1.8.0-2.1) ... Setting up libkrb5-3:amd64 (1.17-3) ... Setting up libbinutils:amd64 (2.31.1-16) ... Setting up cpp-8 (8.3.0-6) ... Setting up libc-dev-bin (2.28-10) ... Setting up golang-src (2:1.11~1) ... Setting up openssl (1.1.1d-0+deb10u3) ... Setting up libcc1-0:amd64 (8.3.0-6) ... Setting up liblsan0:amd64 (8.3.0-6) ... Setting up libitm1:amd64 (8.3.0-6) ... Setting up libgdbm6:amd64 (1.18.1-4) ... Setting up binutils-x86-64-linux-gnu (2.31.1-16) ... Setting up libtsan0:amd64 (8.3.0-6) ... Setting up golang-1.11-doc (1.11.6-1+deb10u3) ... Setting up golang-go (2:1.11~1) ... Setting up libldap-2.4-2:amd64 (2.4.47+dfsg-3+deb10u2) ... Setting up binutils (2.31.1-16) ... Setting up ca-certificates (20200601~deb10u1) ... debconf: unable to initialize frontend: Dialog debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76.) debconf: falling back to frontend: Readline Updating certificates in /etc/ssl/certs... 126 added, 0 removed; done. Setting up libgssapi-krb5-2:amd64 (1.17-3) ... Setting up libgdbm-compat4:amd64 (1.18.1-4) ... Setting up golang-1.11 (1.11.6-1+deb10u3) ... Setting up libgcc-8-dev:amd64 (8.3.0-6) ... Setting up libperl5.28:amd64 (5.28.1-6) ... Setting up cpp (4:8.3.0-1) ... Setting up libc6-dev:amd64 (2.28-10) ... Setting up golang-doc (2:1.11~1) ... Setting up libstdc++-8-dev:amd64 (8.3.0-6) ... Setting up golang (2:1.11~1) ... Setting up gcc-8 (8.3.0-6) ... Setting up libpam0g-dev:amd64 (1.3.1-5) ... Setting up libcurl3-gnutls:amd64 (7.64.0-4+deb10u1) ... Setting up gcc (4:8.3.0-1) ... Setting up perl (5.28.1-6) ... Setting up libdpkg-perl (1.19.7) ... Setting up g++-8 (8.3.0-6) ... Setting up dpkg-dev (1.19.7) ... Setting up liberror-perl (0.17027-2) ... Setting up git (1:2.20.1-2+deb10u3) ... Setting up g++ (4:8.3.0-1) ... update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode Setting up build-essential (12.6) ... Processing triggers for libc-bin (2.28-10) ... Processing triggers for ca-certificates (20200601~deb10u1) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done.
now for the user actions::
root@a88a5742be7f:~# sudo -u me -i
me@a88a5742be7f:~$ groups
me sudo
me@a88a5742be7f:~$ cd
me@a88a5742be7f:~$ git clone https://git.arvados.org/arvados.git
Cloning into 'arvados'...
me@a88a5742be7f:~$ cd arvados
me@a88a5742be7f:~/arvados$ git checkout 16534-example-api-pkg
Branch '16534-example-api-pkg' set up to track remote branch '16534-example-api-pkg' from 'origin'.
Switched to a new branch '16534-example-api-pkg'
me@a88a5742be7f:~/arvados$ git pull
Already up to date.
me@a88a5742be7f:~/arvados$ git log | head
commit bef1b89ee6ba0225765a5fdb7e2ddf826b30cb30
Author: Tom Clegg <tom@tomclegg.ca>
Date: Thu Jul 9 10:28:05 2020 -0400
16534: Add example of calling other controller APIs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomclegg.ca>
me@a88a5742be7f:~/arvados$ go mod download
go: finding github.com/dnaeon/go-vcr v1.0.1
go: finding github.com/ghodss/yaml v1.0.0
go: finding github.com/jmoiron/sqlx v1.2.0
go: finding github.com/mitchellh/go-homedir v0.0.0-20161203194507-b8bc1bf76747
go: finding github.com/Azure/azure-sdk-for-go v19.1.0+incompatible
go: finding github.com/dimchansky/utfbom v1.0.0
go: finding github.com/aws/aws-sdk-go v1.25.30
go: finding github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: finding github.com/imdario/mergo v0.3.8-0.20190415133143-5ef87b449ca7
go: finding github.com/go-asn1-ber/asn1-ber v1.4.1
go: finding github.com/bgentry/speakeasy v0.1.0
go: finding github.com/pelletier/go-buffruneio v0.2.0
go: finding github.com/bradleypeabody/godap v0.0.0-20170216002349-c249933bc092
go: finding gopkg.in/src-d/go-git.v4 v4.0.0
go: finding github.com/arvados/goamz v0.0.0-20190905141525-1bba09f407ef
go: finding github.com/go-sql-driver/mysql v1.4.0
go: finding github.com/docker/distribution v2.6.0-rc.1.0.20180105232752-277ed486c948+incompatible
go: finding github.com/mattn/go-sqlite3 v1.9.0
go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
go: finding github.com/opencontainers/image-spec v1.0.1-0.20171125024018-577479e4dc27
go: finding rsc.io/getopt v0.0.0-20170811000552-20be20937449
go: finding github.com/kevinburke/ssh_config v0.0.0-20171013211458-802051befeb5
go: finding github.com/marstr/guid v1.1.1-0.20170427235115-8bdf7d1a087c
go: finding github.com/prometheus/common v0.7.0
go: finding github.com/src-d/gcfg v1.3.0
go: finding google.golang.org/api v0.13.0
go: finding github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: finding golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45
go: finding github.com/satori/go.uuid v1.2.1-0.20180103174451-36e9d2ebbde5
go: finding github.com/coreos/go-oidc v2.1.0+incompatible
go: finding github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223
go: finding gopkg.in/src-d/go-billy.v4 v4.0.1
go: finding gopkg.in/yaml.v2 v2.2.4
go: finding github.com/xanzy/ssh-agent v0.1.0
go: finding golang.org/x/net v0.0.0-20190613194153-d28f0bde5980
go: finding github.com/go-logfmt/logfmt v0.4.0
go: finding github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7
go: finding golang.org/x/sys v0.0.0-20190422165155-953cdadca894
go: finding github.com/go-ldap/ldap v3.0.3+incompatible
go: finding github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: finding github.com/prometheus/client_golang v1.2.1
go: finding github.com/docker/docker v1.4.2-0.20180109013817-94b8a116fbf1
go: finding github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: finding github.com/lib/pq v1.0.0
go: finding golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e
go: finding google.golang.org/appengine v1.4.0
go: finding github.com/matttproud/golang_protobuf_extensions v1.0.1
go: finding golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
go: finding github.com/hashicorp/golang-lru v0.5.1
go: finding golang.org/x/text v0.3.0
go: finding github.com/golang/protobuf v1.2.0
go: finding gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/coreos/go-systemd v0.0.0-20180108085132-cc4f39464dc7
go: finding github.com/lib/pq v1.3.0
go: finding gopkg.in/asn1-ber.v1 v1.0.0-20181015200546-f715ec2f112d
go: finding github.com/Azure/go-autorest v10.15.2+incompatible
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/julienschmidt/httprouter v1.2.0
go: finding github.com/fsnotify/fsnotify v1.4.9
go: finding gopkg.in/square/go-jose.v2 v2.3.1
go: finding golang.org/x/sys v0.0.0-20190412213103-97732733099d
go: finding gopkg.in/yaml.v2 v2.2.2
go: finding golang.org/x/sys v0.0.0-20191105231009-c1f44814a5cd
go: finding github.com/cespare/xxhash/v2 v2.1.0
go: finding github.com/Microsoft/go-winio v0.4.5
go: finding github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
go: finding github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: finding github.com/prometheus/client_golang v1.0.0
go: finding gopkg.in/warnings.v0 v0.1.2
go: finding golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9
go: finding github.com/gogo/protobuf v1.1.1
go: finding cloud.google.com/go v0.38.0
go: finding github.com/googleapis/gax-go/v2 v2.0.5
go: finding github.com/docker/go-connections v0.3.0
go: finding github.com/karalabe/xgo v0.0.0-20191115072854-c5ccff8648a7
go: finding google.golang.org/appengine v1.5.0
go: finding golang.org/x/text v0.3.2
go: finding github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
go: finding golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5
go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225
go: finding github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: finding gopkg.in/src-d/go-git-fixtures.v3 v3.5.0
go: finding github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4
go: finding golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: finding github.com/beorn7/perks v1.0.1
go: finding golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421
go: finding google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c
go: finding golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2
go: finding github.com/go-kit/kit v0.9.0
go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: finding golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6
go: finding golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4
go: finding github.com/googleapis/gax-go/v2 v2.0.4
go: finding github.com/arvados/cgofuse v1.2.0-arvados1
go: finding github.com/prometheus/procfs v0.0.5
go: finding github.com/sergi/go-diff v1.0.0
go: finding github.com/google/go-cmp v0.2.0
go: finding golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b
go: finding github.com/golang/protobuf v1.3.2
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/json-iterator/go v1.1.7
go: finding golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
go: finding golang.org/x/exp v0.0.0-20190121172915-509febef88a4
go: finding honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a
go: finding github.com/docker/go-units v0.3.3-0.20171221200356-d59758554a3d
go: finding google.golang.org/api v0.4.0
go: finding github.com/prometheus/common v0.4.1
go: finding github.com/pkg/errors v0.8.0
go: finding github.com/google/martian v2.1.0+incompatible
go: finding github.com/davecgh/go-spew v1.1.1
go: finding golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: finding go.opencensus.io v0.21.0
go: finding github.com/stretchr/testify v1.4.0
go: finding golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c
go: finding github.com/google/go-cmp v0.3.0
go: finding golang.org/x/net v0.0.0-20181114220301-adae6a3d119a
go: finding github.com/stretchr/testify v1.3.0
go: finding github.com/modern-go/reflect2 v1.0.1
go: finding github.com/go-kit/kit v0.8.0
go: finding google.golang.org/grpc v1.20.1
go: finding github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc
go: finding github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf
go: finding github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
go: finding github.com/gliderlabs/ssh v0.2.2
go: finding cloud.google.com/go v0.34.0
go: finding golang.org/x/lint v0.0.0-20190409202823-959b441ac422
go: finding github.com/prometheus/client_golang v0.9.1
go: finding golang.org/x/sys v0.0.0-20191010194322-b09406accb47
go: finding golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
go: finding google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873
go: finding github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
go: finding github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
go: finding github.com/sirupsen/logrus v1.2.0
go: finding golang.org/x/tools v0.0.0-20190312170243-e65039ee4138
go: finding github.com/golang/protobuf v1.3.1
go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
go: finding golang.org/x/tools v0.0.0-20190311212946-11955173bddd
go: finding github.com/go-logfmt/logfmt v0.3.0
go: finding github.com/go-stack/stack v1.8.0
go: finding github.com/gorilla/mux v1.6.1-0.20180107155708-5bbbb5b2b572
go: finding github.com/golang/mock v1.2.0
go: finding github.com/BurntSushi/toml v0.3.1
go: finding golang.org/x/net v0.0.0-20190311183353-d8887717615a
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/hashicorp/golang-lru v0.5.0
go: finding honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a
go: finding github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding google.golang.org/grpc v1.19.0
go: finding github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421
go: finding github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding github.com/json-iterator/go v1.1.6
go: finding gopkg.in/check.v1 v1.0.0-20161208181325-20d25e280405
go: finding google.golang.org/appengine v1.1.0
go: finding cloud.google.com/go v0.26.0
go: finding github.com/davecgh/go-spew v1.1.0
go: finding github.com/stretchr/objx v0.1.0
go: finding github.com/msteinert/pam v0.0.0-20190215180659-f29b9f28d6f9
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
go: finding github.com/dgrijalva/jwt-go v3.1.0+incompatible
go: finding github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35
go: finding github.com/google/gofuzz v1.0.0
go: finding github.com/client9/misspell v0.3.4
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3
go: finding golang.org/x/tools v0.0.0-20190114222345-bf090417da8b
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/prometheus/procfs v0.0.2
go: finding honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
go: finding github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973
go: finding github.com/golang/mock v1.1.1
go: finding google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19
go: finding gopkg.in/yaml.v2 v2.2.1
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/beorn7/perks v1.0.0
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding golang.org/x/net v0.0.0-20190213061140-3a22650c66bd
go: finding golang.org/x/tools v0.0.0-20190226205152-f727befe758c
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742
go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/AdRoll/goamz v0.0.0-20170825154802-2731d20f46f4
me@a88a5742be7f:~/arvados$ cd cmd/arvados-server
me@a88a5742be7f:~/arvados/cmd/arvados-server$ go install
# git.arvados.org/arvados.git/sdk/go/arvados
../../sdk/go/arvados/keep_service.go:155:14: undefined: http.NewRequestWithContext
Updated by Nico César over 4 years ago
arvados won't compile in buster because of golang 1.13 is required. I did this:
root@a88a5742be7f:~# echo deb http://deb.debian.org/debian buster-backports main >> /etc/apt/sources.list.d/backports.list root@a88a5742be7f:~# apt-get update Hit:1 http://security.debian.org/debian-security buster/updates InRelease Hit:2 http://deb.debian.org/debian buster InRelease Hit:3 http://deb.debian.org/debian buster-updates InRelease Get:4 http://deb.debian.org/debian buster-backports InRelease [46.7 kB] Get:5 http://deb.debian.org/debian buster-backports/main amd64 Packages [341 kB] Fetched 387 kB in 1s (473 kB/s) Reading package lists... Done root@a88a5742be7f:~# apt-get -yq --no-install-recommends -t buster-backports install golang-go Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: golang-1.14-go golang-1.14-src golang-src Suggested packages: bzr | brz mercurial subversion Recommended packages: pkg-config The following NEW packages will be installed: golang-1.14-go golang-1.14-src The following packages will be upgraded: golang-go golang-src 2 upgraded, 2 newly installed, 0 to remove and 18 not upgraded. Need to get 63.0 MB of archives. After this operation, 326 MB of additional disk space will be used. Get:1 http://deb.debian.org/debian buster-backports/main amd64 golang-1.14-src amd64 1.14-2~bpo10+1 [13.3 MB] Get:2 http://deb.debian.org/debian buster-backports/main amd64 golang-1.14-go amd64 1.14-2~bpo10+1 [49.8 MB] Get:3 http://deb.debian.org/debian buster-backports/main amd64 golang-src amd64 2:1.14~1~bpo10+1 [4780 B] Get:4 http://deb.debian.org/debian buster-backports/main amd64 golang-go amd64 2:1.14~1~bpo10+1 [23.9 kB] Fetched 63.0 MB in 27s (2379 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package golang-1.14-src. (Reading database ... 23385 files and directories currently installed.) Preparing to unpack .../golang-1.14-src_1.14-2~bpo10+1_amd64.deb ... Unpacking golang-1.14-src (1.14-2~bpo10+1) ... Selecting previously unselected package golang-1.14-go. Preparing to unpack .../golang-1.14-go_1.14-2~bpo10+1_amd64.deb ... Unpacking golang-1.14-go (1.14-2~bpo10+1) ... Preparing to unpack .../golang-src_2%3a1.14~1~bpo10+1_amd64.deb ... Unpacking golang-src (2:1.14~1~bpo10+1) over (2:1.11~1) ... Preparing to unpack .../golang-go_2%3a1.14~1~bpo10+1_amd64.deb ... Unpacking golang-go (2:1.14~1~bpo10+1) over (2:1.11~1) ... Setting up golang-1.14-src (1.14-2~bpo10+1) ... Setting up golang-src (2:1.14~1~bpo10+1) ... Setting up golang-1.14-go (1.14-2~bpo10+1) ... Setting up golang-go (2:1.14~1~bpo10+1) ... root@a88a5742be7f:~# sudo -u me -i me@a88a5742be7f:~$ cd arvados/ me@a88a5742be7f:~/arvados$ cd cmd/arvados-server me@a88a5742be7f:~/arvados/cmd/arvados-server$ go install
that worked.
Ahother problem with pre-requisites is that is unclear why it mentions to have a user, but then the user can do very little of the installation because of the package dependencies:
me@a88a5742be7f:~/arvados/cmd/arvados-server$ ~/go/bin/arvados-server install -type test E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root? INFO[2020-07-14T17:56:36.011351803Z] exiting error="exit status 100" me@a88a5742be7f:~/arvados/cmd/arvados-server$ sudo ~/go/bin/arvados-server install -type test
Updated by Nico César over 4 years ago
the process finished with some "hard to tell if it all went according to plan" message ...
inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/plexus-utils-3.1.0.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/xbean-reflect-3.7.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/ion-java-1.0.2.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/junit-platform-engine-1.3.1.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/junit-platform-commons-1.3.1.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/apiguardian-api-1.0.0.jar inflating: /var/lib/arvados/gradle-5.3.1/lib/plugins/opentest4j-1.1.1.jar + ln -sf /var/lib/arvados/gradle-5.3.1/bin/gradle /usr/local/bin/ + rm /var/lib/arvados/tmp/gradle-5.3.1-bin.zip + sed -i 's/^# *\(en_US.UTF-8\)/\1/' /etc/locale.gen + locale-gen Generating locales (this might take a while)... en_US.UTF-8... done Generation complete. INFO[2020-07-14T18:08:50.584237848Z] postgresql cluster 11-main is down; trying to start 2020-07-14 18:08:50.715 UTC [37582] LOG: listening on IPv4 address "127.0.0.1", port 5432 2020-07-14 18:08:50.715 UTC [37582] LOG: could not bind IPv6 address "::1": Cannot assign requested address 2020-07-14 18:08:50.715 UTC [37582] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry. 2020-07-14 18:08:50.720 UTC [37582] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2020-07-14 18:08:50.729 UTC [37585] LOG: database system was shut down at 2020-07-14 18:05:16 UTC 2020-07-14 18:08:50.732 UTC [37582] LOG: database system is ready to accept connections 2020-07-14 18:08:51.638 UTC [37593] root@root FATAL: role "root" does not exist + sudo systemctl restart postgresql System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down WARN[2020-07-14T18:08:51.709524468Z] `systemctl restart postgresql` failed; hoping postgresql does not need to be restarted CREATE COLLATION CREATE COLLATION 2020-07-14 18:08:51.885 UTC [37646] postgres@postgres ERROR: role "arvados" does not exist 2020-07-14 18:08:51.885 UTC [37646] postgres@postgres STATEMENT: ALTER ROLE arvados WITH LOGIN SUPERUSER ENCRYPTED PASSWORD 'insecure_arvados_test' CREATE ROLE INFO[2020-07-14T18:08:51.918523701Z] sent SIGTERM; waiting for postgres to shut down 2020-07-14 18:08:51.918 UTC [37582] LOG: received smart shutdown request 2020-07-14 18:08:51.937 UTC [37582] LOG: background worker "logical replication launcher" (PID 37591) exited with exit code 1 2020-07-14 18:08:51.937 UTC [37586] LOG: shutting down 2020-07-14 18:08:51.966 UTC [37582] LOG: database system is shut down
Updated by Nico César over 4 years ago
took some time but it finished.
time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados (...) Pass: apps/workbench_functionals tests (149s) Pass: apps/workbench_benchmark tests (348s) Failures (1): Fail: apps/workbench_integration tests (1007s) ... rm: cannot remove '/tmp/tmp.NdEARKy05e/GOPATH/pkg/mod/.... .... real 65m30.806s user 29m15.676s sys 4m24.804s
I'll dig down on the workbench_integration test, could be legit.
on the other hand I saw a lot of
rm: cannot remove '/tmp/tmp.NdEARKy05e/GOPATH/pkg/mod/....
I guess that there is a missing sudo somewhere.
Updated by Nico César over 4 years ago
second run won't work. this was the previous one: "662M /tmp/tmp.NdEARKy05e",
me@a88a5742be7f:~$ time ~/arvados/build/run-tests.sh WORKSPACE=~/arvados Checking dependencies: locale: en_US.UTF-8 virtualenv: 15.1.0 ruby: ruby 2.5.7p206 (2019-10-01 revision 67816) [x86_64-linux] go: go version go1.14 linux/amd64 gcc: gcc (Debian 8.3.0-6) 8.3.0 fuse.h: /usr/include/fuse/fuse.h gnutls.h: /usr/include/gnutls/gnutls.h Python2 pyconfig.h: /usr/include/python2.7/pyconfig.h Python3 pyconfig.h: /usr/include/python3.7m/pyconfig.h /bin/netstat nginx: nginx version: nginx/1.14.2 perl: This is perl 5, version 28, subversion 1 (v5.28.1) built for x86_64-linux-gnu-thread-multi perl ExtUtils::MakeMaker: 7.34 perl JSON: 4.02 perl LWP: 6.36 perl Net::SSL: 2.88 gitolite: /usr/bin/gitolite npm: 6.4.1 cadaver: cadaver 0.23.3 libattr1 xattr.h: /usr/include/attr/xattr.h libcurl curl.h: /usr/include/x86_64-linux-gnu/curl/curl.h libpq libpq-fe.h: /usr/include/postgresql/libpq-fe.h libpam pam_appl.h: /usr/include/security/pam_appl.h postgresql: psql (PostgreSQL) 11.7 (Debian 11.7-0+deb10u1) phantomjs: 1.9.8 xvfb: /usr/bin/Xvfb graphviz: dot - graphviz version 2.40.1 (20161225.0304) geckodriver: geckodriver 0.24.0 ( 2019-01-28) R: /usr/bin/Rscript testthat: /usr/bin/pandoc procs with /dev/fuse open: S UID PID PPID C PRI NI RSS SZ WCHAN TTY TIME CMD S 1000 86249 1 0 80 0 122520 649244 - pts/0 0:00 python setup.py test grep fuse /proc/self/mountinfo: WORKSPACE=/home/me/arvados Will install dependencies to /home/me/.gem/ruby/2.5.0 Will install arvados gems to /tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0 Gem search path is GEM_PATH=/tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0 /home/me/arvados/build/run-tests.sh: line 565: /tmp/tmp.HaSn9B1QoE/GEMHOME/.gem/ruby/2.5.0/bin/bundle: No such file or directory tee: /dev/stderr: Permission denied Fatal: install bundler (encountered in setup_ruby_environment at /home/me/arvados/build/run-tests.sh line 566) real 0m2.262s user 0m1.310s sys 0m0.241s me@a88a5742be7f:~$
doing a quick search for bundle, there are several of them installed after all this process...
/var/lib/arvados/bin/bundle /var/lib/arvados/lib/ruby/gems/2.5.0/gems/bundler-2.1.4/exe/bundle /home/me/.gem/ruby/2.5.0/bin/bundle /home/me/.gem/ruby/2.5.0/gems/rails-controller-testing-1.0.4/test/dummy/bin/bundle /home/me/.gem/ruby/2.5.0/gems/bundler-1.11.0/exe/bundle /home/me/.gem/ruby/2.5.0/gems/bundler-1.11.0/lib/bundler/man/bundle /home/me/.gem/ruby/2.5.0/gems/bundler-1.17.3/exe/bundle /home/me/.gem/ruby/2.5.0/gems/rake-13.0.1/bin/bundle /home/me/.gem/ruby/2.5.0/gems/railties-5.0.7.2/lib/rails/generators/rails/app/templates/bin/bundle /home/me/.gem/ruby/2.5.0/gems/bundler-2.0.2/exe/bundle /home/me/.gem/ruby/2.5.0/bundler/gems/arvados-c210114aa8c7/apps/workbench/bin/bundle /home/me/.gem/ruby/2.5.0/bundler/gems/arvados-c210114aa8c7/services/api/bin/bundle /home/me/.gem/ruby/2.5.0/bundler/gems/arvados-81725af5d5d2/apps/workbench/bin/bundle /home/me/.gem/ruby/2.5.0/bundler/gems/arvados-81725af5d5d2/services/api/bin/bundle /home/me/arvados/apps/workbench/bin/bundle /home/me/arvados/apps/workbench/node_modules/.bin/bundle /home/me/arvados/apps/workbench/node_modules/mithril/bundler/bin/bundle /home/me/arvados/apps/workbench/node_modules/labeled-stream-splicer/test/bundle /home/me/arvados/services/api/bin/bundle
Updated by Nico César over 4 years ago
- Target version changed from 2020-07-15 to 2020-08-12 Sprint
Ward will review this ticket and we'll update the documentation
Updated by Peter Amstutz over 4 years ago
- Target version changed from 2020-08-12 Sprint to 2020-08-26 Sprint
Updated by Nico César over 4 years ago
- Target version changed from 2020-08-26 Sprint to 2020-09-09 Sprint
Updated by Nico César over 4 years ago
- Target version changed from 2020-09-09 Sprint to 2020-09-23 Sprint
Updated by Peter Amstutz over 4 years ago
- Status changed from In Progress to Closed
Updated by Nico César over 4 years ago
Ward did some edits and Hakcking_prerequisites with debian:10 is working now.
Actions