-Welcome to Arvados!
+Prerequisites
+=============
-This is the arvados-dev source tree. It contains scripts that can be useful
-if you want to hack on Arvados itself.
+In order to build packages, you will need:
-If you are interested in using Arvados or setting up your own Arvados
-installation, you most likely do not need this source tree.
+* Docker installed
+* permission to run Docker commands
+* the `WORKSPACE` environment variable set to the absolute path of an
+ Arvados Git work tree
-For the Arvados source code, check out the git repository at
- https://github.com/curoverse/arvados
+Quickstart
+==========
-The main Arvados web site is
- https://arvados.org
+Build and test all the packages for a distribution on your architecture by
+running:
-The Arvados public wiki is located at
- https://arvados.org/projects/arvados/wiki
+ ./run-build-test-packages-one-target.sh --target DISTRO
-The Arvados public bug tracker is located at
- https://arvados.org/projects/arvados/issues
+This will build package build and test Docker images for the named target
+distribution, build all packages in a build container, then test all
+packages in a test container.
-For support see
- http://doc.arvados.org/user/getting_started/community.html
+Limit the build to a single architecture by adding the `--arch ARCH`
+option. Supported architectures are amd64 and arm64. Note cross-compilation
+from amd64 to arm64 is currently only supported on Debian 11+.
-Installation documentation is located at
- http://doc.arvados.org/install
+Limit the build to a single package by adding the `--only-build
+PACKAGE_NAME` option. This is helpful when a build is mostly in good shape
+and you're tracking down last bugs in one or two packages.
-If you wish to build the documentation yourself, follow the instructions in
-doc/README to build the documentation, then consult the "Install Guide".
+Get more verbose output by adding the `--debug` option.
-See COPYING for information about Arvados Free Software licenses.
+By default the script avoids rebuilding or retesting packages that it
+detects have already been done in past runs. You can force the script to
+rebuild or retest package(s) with the `--force-build` and `--force-test`
+options, respectively.
+
+Run the script with `--help` for more information about other options.
+
+Scripts in this directory
+=========================
+
+run-tests.sh Run unit and integration test suite.
+
+run-build-test-packages-one-target.sh Entry point, wraps
+ run-build-packages-one-target.sh to
+ perform package building and testing
+ inside Docker.
+
+run-build-packages-one-target.sh Build packages for one target inside Docker.
+
+run-build-packages-all-targets.sh Run run-build-packages-one-target.sh
+ for every target.
+
+run-build-packages.sh Actually build packages. Intended to run
+ inside Docker container with proper
+ build environment.
+
+run-build-packages-python-and-ruby.sh Build Python and Ruby packages suitable
+ for upload to PyPi and Rubygems.
+
+run-build-docker-images.sh Build arvbox Docker images.
+
+run-build-docker-jobs-image.sh Build arvados/jobs Docker image
+ (uses published debian packages)
+
+build-dev-docker-jobs-image.sh Build developer arvados/jobs Docker image
+ (uses local git tree)
+
+run-library.sh A library of functions shared by the
+ various scripts in this
+ directory.
+
+Adding a new target
+===================
+
+In order to build packages on a new distribution, you MUST:
+
+* Add a rule for `TARGET/generated` to `package-build-dockerfiles/Makefile`.
+* Add the new `TARGET/generated` rule to the `all` target in
+ `package-build-dockerfiles/Makefile`.
+* Write `package-build-dockerfiles/TARGET/Dockerfile`.
+* Write `package-test-dockerfiles/TARGET/Dockerfile`.
+* Create `package-testing/test-packages-TARGET.sh`, ideally by making it a
+ symlink to `FORMAT-common-test-packages.sh`.
+* Update the package download code near the bottom of `test_package_presence`
+ in `run-library.sh` so it can download packages for the new distribution.
+
+Of course, any part of our package build or test infrastructure may need to
+be updated to accommodate the process for new distributions. If you're
+having trouble building lots of packages, consider grepping these build
+scripts for the identifier of the closest working target, and see if you may
+need to add branches or similar hooks for your target. If you're having
+trouble building specific packages, consider doing the same for those
+packages' `fpm-info.sh` files.