X-Git-Url: https://git.arvados.org/arvados.git/blobdiff_plain/0bf166a618c7dafcffa3cf0602dc7f09f521e3b7..093ec98e4a065acfc537ea22c08c337c115fe273:/README.md diff --git a/README.md b/README.md index 9ced1d11d1..fced2eb5b7 100644 --- a/README.md +++ b/README.md @@ -1,116 +1,96 @@ -# Arvados-in-a-box +[comment]: # (Copyright © The Arvados Authors. All rights reserved.) +[comment]: # () +[comment]: # (SPDX-License-Identifier: CC-BY-SA-3.0) -Self-contained development, demonstration and testing environment for Arvados. +[![Join the chat at https://gitter.im/arvados/community](https://badges.gitter.im/arvados/community.svg)](https://gitter.im/arvados/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) | [Installing Arvados](https://doc.arvados.org/install/index.html) | [Installing Client SDKs](https://doc.arvados.org/sdk/index.html) | [Report a bug](https://dev.arvados.org/projects/arvados/issues/new) | [Development and Contributing](CONTRIBUTING.md) -## Quick start + -``` -$ bin/arvbox reboot localdemo -``` +[Arvados](https://arvados.org) is an open source platform for +managing, processing, and sharing genomic and other large scientific +and biomedical data. With Arvados, bioinformaticians run and scale +compute-intensive workflows, developers create biomedical +applications, and IT administrators manage large compute and storage +resources. + +The key components of Arvados are: + +* *Keep*: Keep is the Arvados storage system for managing and storing large +collections of files. Keep combines content addressing and a +distributed storage architecture resulting in both high reliability +and high throughput. Every file stored in Keep can be accurately +verified every time it is retrieved. Keep supports the creation of +collections as a flexible way to define data sets without having to +re-organize or needlessly copy data. Keep works on a wide range of +underlying filesystems and object stores. + +* *Crunch*: Crunch is the orchestration system for running [Common Workflow Language](https://www.commonwl.org) workflows. It is +designed to maintain data provenance and workflow +reproducibility. Crunch automatically tracks data inputs and outputs +through Keep and executes workflow processes in Docker containers. In +a cloud environment, Crunch optimizes costs by scaling compute on demand. + +* *Workbench*: The Workbench web application allows users to interactively access +Arvados functionality. It is especially helpful for querying and +browsing data, visualizing provenance, and tracking the progress of +workflows. + +* *Command Line tools*: The command line interface (CLI) provides convenient access to Arvados +functionality in the Arvados platform from the command line. -## Usage +* *API and SDKs*: Arvados is designed to be integrated with existing infrastructure. All +the services in Arvados are accessed through a RESTful API. SDKs are +available for Python, Go, R, Perl, Ruby, and Java. + +# Quick start + +To try out Arvados on your local workstation, you can use Arvbox, which +provides Arvados components pre-installed in a Docker container (requires +Docker 1.9+). After cloning the Arvados git repository: ``` -Arvados-in-a-box - -arvbox (build|start|run|open|shell|ip|stop|reboot|reset|destroy|log|svrestart) - -build build arvbox Docker image -start|run start arvbox container -open open arvbox workbench in a web browser -shell enter arvbox shell -ip print arvbox ip address -stop stop arvbox container -restart stop, then run again -reboot stop, build arvbox Docker image, run -reset delete arvbox arvados data (be careful!) -destroy delete all arvbox code and data (be careful!) -log tail log of specified service -svrestart restart specified service inside arvbox -clone clone an arvbox +$ cd arvados/tools/arvbox/bin +$ ./arvbox start localdemo ``` -## Requirements - -* Linux 3.x+ and Docker 1.9+ -* Minimum of 3 GiB of RAM + additional memory to run jobs -* Minimum of 3 GiB of disk + storage for actual data - -## Configs - -### dev -Development configuration. Boots a complete Arvados environment inside the -container. The "arvados", "arvado-dev" and "sso-devise-omniauth-provider" code -directories along data directories "postgres", "var", "passenger" and "gems" -are bind mounted from the host file system for easy access and persistence -across container rebuilds. Services are bound to the Docker container's -network IP address and can only be accessed on the local host. - -### localdemo -Demo configuration. Boots a complete Arvados environment inside the container. -Unlike the development configuration, code directories are included in the demo -image, and data directories are stored in a separate data volume container. -Services are bound to the Docker container's network IP address and can only be -accessed on the local host. - -### test -Run the test suite. - -### publicdev -Publicly accessible development configuration. Similar to 'dev' except that -service ports are published to the host's IP address and can accessed by anyone -who can connect to the host system. WARNING! The public arvbox configuration -is NOT SECURE and must not be placed on a public IP address or used for -production work. - -### publicdemo -Publicly accessible development configuration. Similar to 'localdemo' except -that service ports are published to the host's IP address and can accessed by -anyone who can connect to the host system. WARNING! The public arvbox configuration -is NOT SECURE and must not be placed on a public IP address or used for -production work. - -## Environment variables - -### ARVBOX_DOCKER -The location of Dockerfile.base and associated files used by "arvbox build". -default: result of $(readlink -f $(dirname $0)/../lib/arvbox/docker) - -### ARVBOX_CONTAINER -The name of the Docker container to manipulate. -default: arvbox - -### ARVBOX_BASE -The base directory to store persistent data for arvbox containers. -default: $HOME/.arvbox - -### ARVBOX_DATA -The base directory to store persistent data for the current container. -default: $ARVBOX_BASE/$ARVBOX_CONTAINER - -### ARVADOS_ROOT -The root directory of the Arvados source tree -default: $ARVBOX_DATA/arvados - -### ARVADOS_DEV_ROOT -The root directory of the Arvados-dev source tree -default: $ARVBOX_DATA/arvados-dev - -### SSO_ROOT -The root directory of the SSO source tree -default: $ARVBOX_DATA/sso-devise-omniauth-provider - -### ARVBOX_PUBLISH_IP -The IP address on which to publish services when running in public -configuration. Overrides default detection of the host's IP address. - -## Notes - -Services are designed to install and auto-configure on start or restart. For -example, the service script for keepstore always compiles keepstore from source -and registers the daemon with the API server. - -Services are run with process supervision, so a service which exits will be -restarted. Dependencies between services are handled by repeatedly trying and -failing the service script until dependencies are fulfilled (by other service -scripts) enabling the service script to complete. +In this mode you will only be able to connect to Arvbox from the same host. To +configure Arvbox to be accessible over a network and for other options see +http://doc.arvados.org/install/arvbox.html for details. + +# Documentation + +Complete documentation, including the [User Guide](https://doc.arvados.org/user/index.html), [Installation documentation](https://doc.arvados.org/install/index.html), [Administrator documentation](https://doc.arvados.org/admin/index.html) and +[API documentation](https://doc.arvados.org/api/index.html) is available at http://doc.arvados.org/ + +If you wish to build the Arvados documentation from a local git clone, see +[doc/README.textile](doc/README.textile) for instructions. + +# Community + +[![Join the chat at https://gitter.im/arvados/community](https://badges.gitter.im/arvados/community.svg)](https://gitter.im/arvados/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) + +The [Arvados community channel](https://gitter.im/arvados/community) +channel at [gitter.im](https://gitter.im) is available for live +discussion and support. + +The [Arvados developement channel](https://gitter.im/arvados/development) +channel at [gitter.im](https://gitter.im) is used to coordinate development. + +The [Arvados user mailing list](http://lists.arvados.org/mailman/listinfo/arvados) +is used to announce new versions and other news. + +All participants are expected to abide by the [Arvados Code of Conduct](CODE_OF_CONDUCT.md). + +# Reporting bugs + +[Report a bug](https://dev.arvados.org/projects/arvados/issues/new) on [dev.arvados.org](https://dev.arvados.org). + +# Development and Contributing + +See [CONTRIBUTING](CONTRIBUTING.md) for information about Arvados development and how to contribute to the Arvados project. + +The [development road map](https://dev.arvados.org/issues/gantt?utf8=%E2%9C%93&set_filter=1&gantt=1&f%5B%5D=project_id&op%5Bproject_id%5D=%3D&v%5Bproject_id%5D%5B%5D=49&f%5B%5D=&zoom=1) outlines some of the project priorities over the next twelve months. + +# Licensing + +Arvados is Free Software. See [COPYING](COPYING) for information about the open source licenses used in Arvados.