From: Tom Clegg Date: Fri, 14 Feb 2020 20:12:40 +0000 (-0500) Subject: Merge branch '16039-fuse-forward-slash-sub' X-Git-Tag: 2.1.0~298 X-Git-Url: https://git.arvados.org/arvados.git/commitdiff_plain/48c38895200cdafaaeca37299bf8352878389a77?hp=a4f57793d8757c451330e6d861eb3c5af2bff3b8 Merge branch '16039-fuse-forward-slash-sub' fixes #16039 Arvados-DCO-1.1-Signed-off-by: Tom Clegg --- diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000000..5345f045ff --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,96 @@ +Arvados Code of Conduct +======================= + +The Arvados Project is dedicated to providing a harassment-free experience for +everyone. We do not tolerate harassment of participants in any form. + +This code of conduct applies to all Arvados Project spaces both online and off: +Gitter chat, Redmine issues, wiki, mailing lists, forums, video chats, and any other +Arvados spaces. Anyone who violates this code of conduct may be sanctioned or +expelled from these spaces at the discretion of the Arvados Team. + +Some Arvados Project spaces may have additional rules in place, which will be +made clearly available to participants. Participants are responsible for +knowing and abiding by these rules. + +Harassment includes, but is not limited to: + + - Offensive comments related to gender, gender identity and expression, sexual +orientation, disability, mental illness, neuro(a)typicality, physical +appearance, body size, age, race, or religion. + - Unwelcome comments regarding a person’s lifestyle choices and practices, +including those related to food, health, parenting, drugs, and employment. + - Deliberate misgendering or use of [dead](https://www.quora.com/What-is-deadnaming/answer/Nancy-C-Walker) +or rejected names. + - Gratuitous or off-topic sexual images or behaviour in spaces where they’re not +appropriate. + - Physical contact and simulated physical contact (eg, textual descriptions like +“\*hug\*” or “\*backrub\*”) without consent or after a request to stop. + - Threats of violence. + - Incitement of violence towards any individual, including encouraging a person +to commit suicide or to engage in self-harm. + - Deliberate intimidation. + - Stalking or following. + - Harassing photography or recording, including logging online activity for +harassment purposes. + - Sustained disruption of discussion. + - Unwelcome sexual attention. + - Pattern of inappropriate social contact, such as requesting/assuming +inappropriate levels of intimacy with others + - Continued one-on-one communication after requests to cease. + - Deliberate “outing” of any aspect of a person’s identity without their consent +except as necessary to protect vulnerable people from intentional abuse. + - Publication of non-harassing private communication. + +The Arvados Project prioritizes marginalized people’s safety over privileged +people’s comfort. The Arvados Leadership Team will not act on complaints regarding: + + - ‘Reverse’ -isms, including ‘reverse racism,’ ‘reverse sexism,’ and ‘cisphobia’ + - Reasonable communication of boundaries, such as “leave me alone,” “go away,” or +“I’m not discussing this with you.” + - Communicating in a [tone](http://geekfeminism.wikia.com/wiki/Tone_argument) +you don’t find congenial + +Reporting +--------- + +If you are being harassed by a member of the Arvados Project, notice that someone +else is being harassed, or have any other concerns, please contact the Arvados +Project Team at contact@arvados.org. If person who is harassing +you is on the team, they will recuse themselves from handling your incident. We +will respond as promptly as we can. + +This code of conduct applies to Arvados Project spaces, but if you are being +harassed by a member of Arvados Project outside our spaces, we still want to +know about it. We will take all good-faith reports of harassment by Arvados Project +members, especially the Arvados Team, seriously. This includes harassment +outside our spaces and harassment that took place at any point in time. The +abuse team reserves the right to exclude people from the Arvados Project based on +their past behavior, including behavior outside Arvados Project spaces and +behavior towards people who are not in the Arvados Project. + +In order to protect volunteers from abuse and burnout, we reserve the right to +reject any report we believe to have been made in bad faith. Reports intended +to silence legitimate criticism may be deleted without response. + +We will respect confidentiality requests for the purpose of protecting victims +of abuse. At our discretion, we may publicly name a person about whom we’ve +received harassment complaints, or privately warn third parties about them, if +we believe that doing so will increase the safety of Arvados Project members or +the general public. We will not name harassment victims without their +affirmative consent. + +Consequences +------------ + +Participants asked to stop any harassing behavior are expected to comply +immediately. + +If a participant engages in harassing behavior, the Arvados Team may +take any action they deem appropriate, up to and including expulsion from all +Arvados Project spaces and identification of the participant as a harasser to other +Arvados Project members or the general public. + +This anti-harassment policy is based on the [example policy from the Geek +Feminism wiki](http://geekfeminism.wikia.com/wiki/Community_anti-harassment/Policy), +created by the Geek Feminism community. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..459d7277a5 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,75 @@ +[comment]: # (Copyright © The Arvados Authors. All rights reserved.) +[comment]: # () +[comment]: # (SPDX-License-Identifier: CC-BY-SA-3.0) + +# Contributing + +Arvados is free software, which means it is free for all to use, learn +from, and improve. We encourage contributions from the community that +improve Arvados for everyone. Some examples of contributions are bug +reports, bug fixes, new features, and scripts or documentation that help +with using, administering, or installing Arvados. We also love to +hear about Arvados success stories. + +Those interested in contributing should begin by joining the [Arvados community +channel](https://gitter.im/arvados/community) and telling us about your interest. + +Contributers should also create an account at https://dev.arvados.org +to be able to create and comment on bug tracker issues. The +Arvados public bug tracker is located at +https://dev.arvados.org/projects/arvados/issues . + +Contributers may also be interested in the [development road map](https://dev.arvados.org/issues/gantt?utf8=%E2%9C%93&set_filter=1&gantt=1&f%5B%5D=project_id&op%5Bproject_id%5D=%3D&v%5Bproject_id%5D%5B%5D=49&f%5B%5D=&zoom=1). + +# Development + +Git repositories for primary development are located at +https://git.arvados.org/ and can also be browsed at +https://dev.arvados.org/projects/arvados/repository . Every push to +the master branch is also mirrored to Github at +https://github.com/arvados/arvados . + +Visit [Hacking Arvados](https://dev.arvados.org/projects/arvados/wiki/Hacking) for +detailed information about setting up an Arvados development +environment, development process, coding standards, and notes about specific components. + +If you wish to build the Arvados documentation from a local git clone, see +[doc/README.textile](doc/README.textile) for instructions. + +# Pull requests + +The preferred method for making contributions is through Github pull requests. + +This is the general contribution process: + +1. Fork the Arvados repository using the Github "Fork" button +2. Clone your fork, make your changes, commit to your fork. +3. Every commit message must have a DCO sign-off and every file must have a SPDX license (see below). +4. Add yourself to the [AUTHORS](AUTHORS) file +5. When your fork is ready, through Github, Create a Pull Request against `arvados:master` +6. Notify the core team about your pull request through the [Arvados development +channel](https://gitter.im/arvados/development) or by other means. +7. A member of the core team will review the pull request. They may have questions or comments, or request changes. +8. When the contribution is ready, a member of the core team will +merge the pull request into the master branch, which will +automatically resolve the pull request. + +The Arvados project does not require a contributor agreement in advance, but does require each commit message include a [Developer Certificate of Origin](https://dev.arvados.org/projects/arvados/wiki/Developer_Certificate_Of_Origin). Please ensure *every git commit message* includes `Arvados-DCO-1.1-Signed-off-by`. If you have already made commits without it, fix them with `git commit --amend` or `git rebase`. + +The Developer Certificate of Origin line looks like this: + +``` +Arvados-DCO-1.1-Signed-off-by: Joe Smith +``` + +New files must also include `SPDX-License-Identifier` at the top with one of the three Arvados open source licenses. See [COPYING](COPYING) for details. + +# Continuous integration + +Continuous integration is hosted at https://ci.arvados.org/ + +Currently, external contributers cannot trigger builds. We are investigating integration with Github pull requests for the future. + +[![Build Status](https://ci.arvados.org/buildStatus/icon?job=run-tests)](https://ci.arvados.org/job/run-tests/) + +[![Go Report Card](https://goreportcard.com/badge/github.com/arvados/arvados)](https://goreportcard.com/report/github.com/arvados/arvados) diff --git a/COPYING b/COPYING index 61c31397a0..c549d8a7bb 100644 --- a/COPYING +++ b/COPYING @@ -17,3 +17,7 @@ The full license text for each license is available in this directory: AGPL-3.0: agpl-3.0.txt Apache-2.0: apache-2.0.txt CC-BY-SA-3.0: cc-by-sa-3.0.txt + +As a general rule, code in the sdk/ directory is licensed Apache-2.0, +documentation in the doc/ directory is licensed CC-BY-SA-3.0, and +everything else is licensed AGPL-3.0. \ No newline at end of file diff --git a/README.md b/README.md index 08c102557c..fced2eb5b7 100644 --- a/README.md +++ b/README.md @@ -2,22 +2,47 @@ [comment]: # () [comment]: # (SPDX-License-Identifier: CC-BY-SA-3.0) -[Arvados](https://arvados.org) is a free software distributed computing platform -for bioinformatics, data science, and high throughput analysis of massive data -sets. Arvados supports a variety of cloud, cluster and HPC environments. +[![Join the chat at https://gitter.im/arvados/community](https://badges.gitter.im/arvados/community.svg)](https://gitter.im/arvados/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) | [Installing Arvados](https://doc.arvados.org/install/index.html) | [Installing Client SDKs](https://doc.arvados.org/sdk/index.html) | [Report a bug](https://dev.arvados.org/projects/arvados/issues/new) | [Development and Contributing](CONTRIBUTING.md) -Arvados consists of: + -* *Keep*: a petabyte-scale content-addressed distributed storage system for managing and - storing collections of files, accessible via HTTP and FUSE mount. +[Arvados](https://arvados.org) is an open source platform for +managing, processing, and sharing genomic and other large scientific +and biomedical data. With Arvados, bioinformaticians run and scale +compute-intensive workflows, developers create biomedical +applications, and IT administrators manage large compute and storage +resources. -* *Crunch*: a Docker-based cluster and HPC workflow engine designed providing - strong versioning, reproducibilty, and provenance of computations. +The key components of Arvados are: -* Related services and components including a web workbench for managing files - and compute jobs, REST APIs, SDKs, and other tools. +* *Keep*: Keep is the Arvados storage system for managing and storing large +collections of files. Keep combines content addressing and a +distributed storage architecture resulting in both high reliability +and high throughput. Every file stored in Keep can be accurately +verified every time it is retrieved. Keep supports the creation of +collections as a flexible way to define data sets without having to +re-organize or needlessly copy data. Keep works on a wide range of +underlying filesystems and object stores. -## Quick start +* *Crunch*: Crunch is the orchestration system for running [Common Workflow Language](https://www.commonwl.org) workflows. It is +designed to maintain data provenance and workflow +reproducibility. Crunch automatically tracks data inputs and outputs +through Keep and executes workflow processes in Docker containers. In +a cloud environment, Crunch optimizes costs by scaling compute on demand. + +* *Workbench*: The Workbench web application allows users to interactively access +Arvados functionality. It is especially helpful for querying and +browsing data, visualizing provenance, and tracking the progress of +workflows. + +* *Command Line tools*: The command line interface (CLI) provides convenient access to Arvados +functionality in the Arvados platform from the command line. + +* *API and SDKs*: Arvados is designed to be integrated with existing infrastructure. All +the services in Arvados are accessed through a RESTful API. SDKs are +available for Python, Go, R, Perl, Ruby, and Java. + +# Quick start To try out Arvados on your local workstation, you can use Arvbox, which provides Arvados components pre-installed in a Docker container (requires @@ -32,48 +57,40 @@ In this mode you will only be able to connect to Arvbox from the same host. To configure Arvbox to be accessible over a network and for other options see http://doc.arvados.org/install/arvbox.html for details. -## Documentation +# Documentation -Complete documentation, including a User Guide, Installation documentation and -API documentation is available at http://doc.arvados.org/ +Complete documentation, including the [User Guide](https://doc.arvados.org/user/index.html), [Installation documentation](https://doc.arvados.org/install/index.html), [Administrator documentation](https://doc.arvados.org/admin/index.html) and +[API documentation](https://doc.arvados.org/api/index.html) is available at http://doc.arvados.org/ If you wish to build the Arvados documentation from a local git clone, see -doc/README.textile for instructions. +[doc/README.textile](doc/README.textile) for instructions. -## Community +# Community [![Join the chat at https://gitter.im/arvados/community](https://badges.gitter.im/arvados/community.svg)](https://gitter.im/arvados/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) -The [arvados community channel](https://gitter.im/arvados/community) +The [Arvados community channel](https://gitter.im/arvados/community) channel at [gitter.im](https://gitter.im) is available for live discussion and support. -The -[Arvados user mailing list](http://lists.arvados.org/mailman/listinfo/arvados) -is a forum for general discussion, questions, and news about Arvados -development. The -[Arvados developer mailing list](http://lists.arvados.org/mailman/listinfo/arvados-dev) -is a forum for more technical discussion, intended for developers and -contributors to Arvados. +The [Arvados developement channel](https://gitter.im/arvados/development) +channel at [gitter.im](https://gitter.im) is used to coordinate development. -## Development +The [Arvados user mailing list](http://lists.arvados.org/mailman/listinfo/arvados) +is used to announce new versions and other news. -[![Build Status](https://ci.arvados.org/buildStatus/icon?job=run-tests)](https://ci.arvados.org/job/run-tests/) -[![Go Report Card](https://goreportcard.com/badge/github.com/arvados/arvados)](https://goreportcard.com/report/github.com/arvados/arvados) +All participants are expected to abide by the [Arvados Code of Conduct](CODE_OF_CONDUCT.md). -The Arvados public bug tracker is located at https://dev.arvados.org/projects/arvados/issues +# Reporting bugs -Continuous integration is hosted at https://ci.arvados.org/ +[Report a bug](https://dev.arvados.org/projects/arvados/issues/new) on [dev.arvados.org](https://dev.arvados.org). -Instructions for setting up a development environment and working on specific -components can be found on the -["Hacking Arvados" page of the Arvados wiki](https://dev.arvados.org/projects/arvados/wiki/Hacking). +# Development and Contributing -## Contributing +See [CONTRIBUTING](CONTRIBUTING.md) for information about Arvados development and how to contribute to the Arvados project. -When making a pull request, please ensure *every git commit message* includes a one-line [Developer Certificate of Origin](https://dev.arvados.org/projects/arvados/wiki/Developer_Certificate_Of_Origin). If you have already made commits without it, fix them with `git commit --amend` or `git rebase`. +The [development road map](https://dev.arvados.org/issues/gantt?utf8=%E2%9C%93&set_filter=1&gantt=1&f%5B%5D=project_id&op%5Bproject_id%5D=%3D&v%5Bproject_id%5D%5B%5D=49&f%5B%5D=&zoom=1) outlines some of the project priorities over the next twelve months. -## Licensing +# Licensing -Arvados is Free Software. See COPYING for information about Arvados Free -Software licenses. +Arvados is Free Software. See [COPYING](COPYING) for information about the open source licenses used in Arvados. diff --git a/build/package-build-dockerfiles/centos7/Dockerfile b/build/package-build-dockerfiles/centos7/Dockerfile index 3a61e64cf4..9c2660387a 100644 --- a/build/package-build-dockerfiles/centos7/Dockerfile +++ b/build/package-build-dockerfiles/centos7/Dockerfile @@ -50,7 +50,7 @@ RUN rpm -ivh epel-release-latest-7.noarch.rpm RUN git clone --depth 1 git://git.arvados.org/arvados.git /tmp/arvados && cd /tmp/arvados/services/api && /usr/local/rvm/bin/rvm-exec default bundle && cd /tmp/arvados/apps/workbench && /usr/local/rvm/bin/rvm-exec default bundle # The version of setuptools that comes with CentOS is way too old -RUN pip install --upgrade setuptools +RUN pip install --upgrade 'setuptools<45' ENV WORKSPACE /arvados CMD ["scl", "enable", "rh-python36", "/usr/local/rvm/bin/rvm-exec default bash /jenkins/run-build-packages.sh --target centos7"] diff --git a/build/package-build-dockerfiles/debian10/Dockerfile b/build/package-build-dockerfiles/debian10/Dockerfile index 20527dd45c..ff86262d38 100644 --- a/build/package-build-dockerfiles/debian10/Dockerfile +++ b/build/package-build-dockerfiles/debian10/Dockerfile @@ -12,7 +12,7 @@ ENV DEBIAN_FRONTEND noninteractive RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-setuptools python3-pip libcurl4-gnutls-dev curl git procps libattr1-dev libfuse-dev libgnutls28-dev libpq-dev python-pip unzip python3-venv python3-dev # Install virtualenv -RUN /usr/bin/pip install virtualenv +RUN /usr/bin/pip install 'virtualenv<20' # Install RVM ADD generated/mpapis.asc /tmp/ diff --git a/build/package-build-dockerfiles/debian9/Dockerfile b/build/package-build-dockerfiles/debian9/Dockerfile index 2aaccf6aa7..257523a72f 100644 --- a/build/package-build-dockerfiles/debian9/Dockerfile +++ b/build/package-build-dockerfiles/debian9/Dockerfile @@ -12,7 +12,7 @@ ENV DEBIAN_FRONTEND noninteractive RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-setuptools python3-pip libcurl4-gnutls-dev curl git procps libattr1-dev libfuse-dev libgnutls28-dev libpq-dev python-pip unzip python3-venv python3-dev # Install virtualenv -RUN /usr/bin/pip install virtualenv +RUN /usr/bin/pip install 'virtualenv<20' # Install RVM ADD generated/mpapis.asc /tmp/ diff --git a/build/package-build-dockerfiles/ubuntu1604/Dockerfile b/build/package-build-dockerfiles/ubuntu1604/Dockerfile index 23fcca07df..e046ae1690 100644 --- a/build/package-build-dockerfiles/ubuntu1604/Dockerfile +++ b/build/package-build-dockerfiles/ubuntu1604/Dockerfile @@ -11,7 +11,7 @@ ENV DEBIAN_FRONTEND noninteractive RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-setuptools python3-pip libcurl4-gnutls-dev libgnutls-dev curl git libattr1-dev libfuse-dev libpq-dev python-pip unzip tzdata python3-venv python3-dev # Install virtualenv -RUN /usr/bin/pip install virtualenv +RUN /usr/bin/pip install 'virtualenv<20' # Install RVM ADD generated/mpapis.asc /tmp/ diff --git a/build/package-build-dockerfiles/ubuntu1804/Dockerfile b/build/package-build-dockerfiles/ubuntu1804/Dockerfile index 06c21ca3fa..c652fe1c08 100644 --- a/build/package-build-dockerfiles/ubuntu1804/Dockerfile +++ b/build/package-build-dockerfiles/ubuntu1804/Dockerfile @@ -11,7 +11,7 @@ ENV DEBIAN_FRONTEND noninteractive RUN /usr/bin/apt-get update && /usr/bin/apt-get install -q -y python2.7-dev python3 python-setuptools python3-pip libcurl4-gnutls-dev libgnutls28-dev curl git libattr1-dev libfuse-dev libpq-dev python-pip unzip tzdata python3-venv python3-dev # Install virtualenv -RUN /usr/bin/pip install virtualenv +RUN /usr/bin/pip install 'virtualenv<20' # Install RVM ADD generated/mpapis.asc /tmp/ diff --git a/build/run-library.sh b/build/run-library.sh index 5b1dde46a9..ac5dc718be 100755 --- a/build/run-library.sh +++ b/build/run-library.sh @@ -475,9 +475,9 @@ fpm_build_virtualenv () { rm -rf dist/* # Get the latest setuptools - if ! $pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U setuptools; then + if ! $pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U 'setuptools<45'; then echo "Error, unable to upgrade setuptools with" - echo " $pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U setuptools" + echo " $pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U 'setuptools<45'" exit 1 fi # filter a useless warning (when building the cwltest package) from the stderr output @@ -532,9 +532,9 @@ fpm_build_virtualenv () { fi echo "pip version: `build/usr/share/$python/dist/$PYTHON_PKG/bin/$pip --version`" - if ! build/usr/share/$python/dist/$PYTHON_PKG/bin/$pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U setuptools; then + if ! build/usr/share/$python/dist/$PYTHON_PKG/bin/$pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U 'setuptools<45'; then echo "Error, unable to upgrade setuptools with" - echo " build/usr/share/$python/dist/$PYTHON_PKG/bin/$pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U setuptools" + echo " build/usr/share/$python/dist/$PYTHON_PKG/bin/$pip install $DASHQ_UNLESS_DEBUG $CACHE_FLAG -U 'setuptools<45'" exit 1 fi echo "setuptools version: `build/usr/share/$python/dist/$PYTHON_PKG/bin/$python -c 'import setuptools; print(setuptools.__version__)'`" diff --git a/doc/_includes/_navbar_top.liquid b/doc/_includes/_navbar_top.liquid index c69a2efbec..38c8e0a1db 100644 --- a/doc/_includes/_navbar_top.liquid +++ b/doc/_includes/_navbar_top.liquid @@ -57,10 +57,10 @@ SPDX-License-Identifier: CC-BY-SA-3.0