arvados.git
10 years agoEnsure script_version matches before satisfying pipeline component with a job.
Tom Clegg [Tue, 19 Nov 2013 08:38:38 +0000 (00:38 -0800)]
Ensure script_version matches before satisfying pipeline component with a job.

10 years agoAdd /users/:uuid/event_stream action to subscribe to per-user channel
Tom Clegg [Tue, 19 Nov 2013 08:37:03 +0000 (00:37 -0800)]
Add /users/:uuid/event_stream action to subscribe to per-user channel

10 years agoSend job log messages to owner_uuid redis channel
Tom Clegg [Tue, 19 Nov 2013 08:16:22 +0000 (00:16 -0800)]
Send job log messages to owner_uuid redis channel

10 years agoRead pipeline template from local file if --template arg is not a uuid
Tom Clegg [Tue, 19 Nov 2013 00:40:40 +0000 (16:40 -0800)]
Read pipeline template from local file if --template arg is not a uuid

10 years agoun-comment ADD line for api.tar
Ward Vandewege [Tue, 19 Nov 2013 03:30:19 +0000 (22:30 -0500)]
un-comment ADD line for api.tar

10 years agoLet API server use SSO container.
Ward Vandewege [Tue, 19 Nov 2013 03:24:03 +0000 (22:24 -0500)]
Let API server use SSO container.

This doesn't work quite yet - need to sort out inter-container
communication. But it's very close now.

10 years agoFixes for SSO rails app.
Ward Vandewege [Tue, 19 Nov 2013 03:23:44 +0000 (22:23 -0500)]
Fixes for SSO rails app.

10 years agoGenerated files shouldn't have been committed.
Ward Vandewege [Tue, 19 Nov 2013 02:51:38 +0000 (21:51 -0500)]
Generated files shouldn't have been committed.

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Ward Vandewege [Mon, 18 Nov 2013 22:25:29 +0000 (17:25 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

Conflicts:
docker/build.sh
docker/config.yml.example

10 years agoMerge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into...
Tim Pierce [Mon, 18 Nov 2013 21:50:38 +0000 (16:50 -0500)]
Merge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into 1579-keep-server-in-docker

10 years agoInstall warehouse-apps into arvados/warehouse directly from github source.
Tim Pierce [Mon, 18 Nov 2013 20:06:21 +0000 (15:06 -0500)]
Install warehouse-apps into arvados/warehouse directly from github source.

10 years agoAdd prerequisites for Keep server described in
Tim Pierce [Fri, 15 Nov 2013 23:52:56 +0000 (18:52 -0500)]
Add prerequisites for Keep server described in
https://github.com/clinicalfuture/warehouse-apps/blob/master/libwarehouse-perl/server/INSTALL

10 years agoAdd supervisor to start keepd when container is launched.
Tim Pierce [Thu, 14 Nov 2013 22:54:55 +0000 (17:54 -0500)]
Add supervisor to start keepd when container is launched.
Update README with deploy instructions for Keep.

10 years agoBuild an arvados/warehouse container with locally checked out warehouse-apps
Tim Pierce [Thu, 14 Nov 2013 20:49:18 +0000 (15:49 -0500)]
Build an arvados/warehouse container with locally checked out warehouse-apps
code.

The warehouse-apps repo at git://github.com/clinicalfuture/warehouse-apps.git
must be checked out at the same level as the arvados repo.

10 years agoInitial checkin for building warehouse-apps.
Tim Pierce [Fri, 8 Nov 2013 22:30:59 +0000 (17:30 -0500)]
Initial checkin for building warehouse-apps.

10 years agoDeploy containers with working copy of sources.
Tim Pierce [Thu, 14 Nov 2013 07:10:05 +0000 (02:10 -0500)]
Deploy containers with working copy of sources.

10 years agoRevert anjlab-bootstrap-rails to version 2 because bootstrap-editable
Tom Clegg [Mon, 18 Nov 2013 20:49:13 +0000 (12:49 -0800)]
Revert anjlab-bootstrap-rails to version 2 because bootstrap-editable
is not ready for bootstrap 3 yet.

10 years agoInstall warehouse-apps into arvados/warehouse directly from github source.
Tim Pierce [Mon, 18 Nov 2013 20:06:21 +0000 (15:06 -0500)]
Install warehouse-apps into arvados/warehouse directly from github source.

10 years agoMove @redis.exists and .getrange outside .subscribe block. Otherwise
Tom Clegg [Mon, 18 Nov 2013 19:53:00 +0000 (11:53 -0800)]
Move @redis.exists and .getrange outside .subscribe block. Otherwise
they fail silently and the client does not get any of the buffered log
messages.

10 years agoAdd SSO docker container
Ward Vandewege [Mon, 18 Nov 2013 19:49:21 +0000 (14:49 -0500)]
Add SSO docker container

10 years agoBring the name of the workbench secret token config variable in line.
Ward Vandewege [Mon, 18 Nov 2013 19:47:35 +0000 (14:47 -0500)]
Bring the name of the workbench secret token config variable in line.

10 years agoAvoid Rails cache when polling database for latest node/job state
Tom Clegg [Mon, 18 Nov 2013 19:38:10 +0000 (11:38 -0800)]
Avoid Rails cache when polling database for latest node/job state

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Ward Vandewege [Mon, 18 Nov 2013 19:36:38 +0000 (14:36 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

10 years agoAlso auto-generate _SECRET config variables when they are unset, like we
Ward Vandewege [Mon, 18 Nov 2013 19:36:15 +0000 (14:36 -0500)]
Also auto-generate _SECRET config variables when they are unset, like we
do for _PW config variables.

10 years agoAdd Admin/Nodes page
Tom Clegg [Mon, 18 Nov 2013 19:21:30 +0000 (11:21 -0800)]
Add Admin/Nodes page

10 years agoInclude privileged attributes in node#ping response
Tom Clegg [Mon, 18 Nov 2013 19:09:44 +0000 (11:09 -0800)]
Include privileged attributes in node#ping response

10 years agoadd tests for node#ping, fix response when ping_secret is invalid
Tom Clegg [Mon, 18 Nov 2013 18:21:28 +0000 (10:21 -0800)]
add tests for node#ping, fix response when ping_secret is invalid

10 years agoFix crash building api response when user is nil
Tom Clegg [Mon, 18 Nov 2013 18:20:44 +0000 (10:20 -0800)]
Fix crash building api response when user is nil

10 years agoMerge branch '1600-native-keep-client'
Tom Clegg [Mon, 18 Nov 2013 04:58:23 +0000 (20:58 -0800)]
Merge branch '1600-native-keep-client'

10 years agoObey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
Tom Clegg [Fri, 15 Nov 2013 16:40:46 +0000 (08:40 -0800)]
Obey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
available (instead of skipping certificate validation because Python's
CA list does not include the one we need).

10 years agoAllow all active users to retrieve list of keep nodes.
Tom Clegg [Fri, 15 Nov 2013 16:31:54 +0000 (08:31 -0800)]
Allow all active users to retrieve list of keep nodes.
refs #1599

10 years agoturn on logging if ARVADOS_DEBUG environment variable exists
Tom Clegg [Mon, 18 Nov 2013 04:55:19 +0000 (20:55 -0800)]
turn on logging if ARVADOS_DEBUG environment variable exists

10 years agoUse hash to determine probe order
Tom Clegg [Mon, 18 Nov 2013 04:54:35 +0000 (20:54 -0800)]
Use hash to determine probe order

10 years agoAdd test case for reading and writing small Keep blocks
Tom Clegg [Mon, 18 Nov 2013 04:28:55 +0000 (20:28 -0800)]
Add test case for reading and writing small Keep blocks

10 years agoUse httplib2 to connect to Keep instead of executing whget/whput
Tom Clegg [Mon, 18 Nov 2013 04:26:23 +0000 (20:26 -0800)]
Use httplib2 to connect to Keep instead of executing whget/whput

10 years agoObey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
Tom Clegg [Fri, 15 Nov 2013 16:40:46 +0000 (08:40 -0800)]
Obey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
available (instead of skipping certificate validation because Python's
CA list does not include the one we need).

10 years agoAdd prerequisites for Keep server described in
Tim Pierce [Fri, 15 Nov 2013 23:52:56 +0000 (18:52 -0500)]
Add prerequisites for Keep server described in
https://github.com/clinicalfuture/warehouse-apps/blob/master/libwarehouse-perl/server/INSTALL

10 years agoImprove error handling for login operations.
Ward Vandewege [Fri, 15 Nov 2013 21:49:35 +0000 (16:49 -0500)]
Improve error handling for login operations.

10 years agoAllow all active users to retrieve list of keep nodes.
Tom Clegg [Fri, 15 Nov 2013 16:31:54 +0000 (08:31 -0800)]
Allow all active users to retrieve list of keep nodes.
refs #1599

10 years agofix error message scraping regexp
Tom Clegg [Fri, 15 Nov 2013 16:13:41 +0000 (08:13 -0800)]
fix error message scraping regexp

10 years agoUse more current slurm environment variable name in error message scraping
Tom Clegg [Fri, 15 Nov 2013 15:51:45 +0000 (07:51 -0800)]
Use more current slurm environment variable name in error message scraping

10 years agofix over-compensation for attempts counter when task fails on suspect node
Tom Clegg [Fri, 15 Nov 2013 15:51:02 +0000 (07:51 -0800)]
fix over-compensation for attempts counter when task fails on suspect node

10 years agoreduce default #attempts per job task from 9 to 2
Tom Clegg [Fri, 15 Nov 2013 15:45:20 +0000 (07:45 -0800)]
reduce default #attempts per job task from 9 to 2

10 years agofix loop label syntax error introduced in 7c4460b82b8abadc5144f758a9f36ce0f42893e8
Tom Clegg [Fri, 15 Nov 2013 08:15:57 +0000 (00:15 -0800)]
fix loop label syntax error introduced in 7c4460b82b8abadc5144f758a9f36ce0f42893e8

10 years agoAdd supervisor to start keepd when container is launched.
Tim Pierce [Thu, 14 Nov 2013 22:54:55 +0000 (17:54 -0500)]
Add supervisor to start keepd when container is launched.
Update README with deploy instructions for Keep.

10 years agoBuild an arvados/warehouse container with locally checked out warehouse-apps
Tim Pierce [Thu, 14 Nov 2013 20:49:18 +0000 (15:49 -0500)]
Build an arvados/warehouse container with locally checked out warehouse-apps
code.

The warehouse-apps repo at git://github.com/clinicalfuture/warehouse-apps.git
must be checked out at the same level as the arvados repo.

10 years agoMerge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into...
Tim Pierce [Thu, 14 Nov 2013 17:35:48 +0000 (12:35 -0500)]
Merge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into 1579-keep-server-in-docker

Rebasing 1579-keep-server-in-docker from 1578-api-server-in-docker
(which has much better Docker support to work from).

10 years agoadd discovery document test
Tom Clegg [Thu, 14 Nov 2013 16:50:13 +0000 (08:50 -0800)]
add discovery document test

10 years agodelete cached discovery document during boot, so doc matches running code
Tom Clegg [Thu, 14 Nov 2013 16:49:58 +0000 (08:49 -0800)]
delete cached discovery document during boot, so doc matches running code

10 years agoupdate discovery document generator for rails 3.2
Tom Clegg [Thu, 14 Nov 2013 16:49:02 +0000 (08:49 -0800)]
update discovery document generator for rails 3.2

10 years agoadd generatedAt field to discovery document
Tom Clegg [Thu, 14 Nov 2013 16:48:22 +0000 (08:48 -0800)]
add generatedAt field to discovery document

10 years agoDeploy containers with working copy of sources.
Tim Pierce [Thu, 14 Nov 2013 07:10:05 +0000 (02:10 -0500)]
Deploy containers with working copy of sources.

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Tim Pierce [Wed, 13 Nov 2013 18:31:57 +0000 (13:31 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

10 years agoBreak out deploying vs. installation instructions.
Tim Pierce [Wed, 13 Nov 2013 18:31:21 +0000 (13:31 -0500)]
Break out deploying vs. installation instructions.

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Ward Vandewege [Wed, 13 Nov 2013 17:59:02 +0000 (12:59 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

Conflicts:
docker/workbench/Dockerfile

10 years agoAdd workbench server to build.sh and run.sh
Ward Vandewege [Wed, 13 Nov 2013 17:43:20 +0000 (12:43 -0500)]
Add workbench server to build.sh and run.sh

10 years agodocserver container updates:
Ward Vandewege [Wed, 13 Nov 2013 17:42:23 +0000 (12:42 -0500)]
docserver container updates:
* use base arvados Debian image
* simplify Dockerfile

10 years agoFix Dockerfile comments.
Ward Vandewege [Wed, 13 Nov 2013 17:11:57 +0000 (12:11 -0500)]
Fix Dockerfile comments.

10 years agoAdd docker image with workbench server.
Ward Vandewege [Mon, 11 Nov 2013 22:33:12 +0000 (17:33 -0500)]
Add docker image with workbench server.

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Tim Pierce [Wed, 13 Nov 2013 15:04:03 +0000 (10:04 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

10 years agoPrecompile Rails assets.
Tim Pierce [Wed, 13 Nov 2013 15:03:05 +0000 (10:03 -0500)]
Precompile Rails assets.
Make sure mod_ssl is enabled in Apache.

10 years agoAdd docker image with workbench server.
Ward Vandewege [Mon, 11 Nov 2013 22:33:12 +0000 (17:33 -0500)]
Add docker image with workbench server.

10 years agoMerge branch '1578-api-server-in-docker'
Tim Pierce [Tue, 12 Nov 2013 01:52:16 +0000 (20:52 -0500)]
Merge branch '1578-api-server-in-docker'

10 years agoSet nameservers at config time.
Tim Pierce [Mon, 11 Nov 2013 23:11:45 +0000 (18:11 -0500)]
Set nameservers at config time.

10 years agoMerge branch '1578-api-server-in-docker' of git.clinicalfuture.com:arvados into 1578...
Tim Pierce [Mon, 11 Nov 2013 23:11:05 +0000 (18:11 -0500)]
Merge branch '1578-api-server-in-docker' of git.clinicalfuture.com:arvados into 1578-api-server-in-docker

Conflicts:
docker/config.rb

10 years agoCopied Apache vhost config from qr1hi.
Tim Pierce [Mon, 11 Nov 2013 23:05:19 +0000 (18:05 -0500)]
Copied Apache vhost config from qr1hi.

10 years agoWe want to be able to handle naked
Ward Vandewege [Mon, 11 Nov 2013 22:24:28 +0000 (17:24 -0500)]
We want to be able to handle naked

  false

in config.rb's variable substitution.

10 years agoAdd sass explicitly
Ward Vandewege [Mon, 11 Nov 2013 22:12:40 +0000 (17:12 -0500)]
Add sass explicitly

10 years agoStyle improvements -- rubocop
Ward Vandewege [Mon, 11 Nov 2013 21:41:39 +0000 (16:41 -0500)]
Style improvements -- rubocop

10 years agoAllow up to 10 levels of nesting for variable substitution in config.rb
Ward Vandewege [Mon, 11 Nov 2013 21:38:03 +0000 (16:38 -0500)]
Allow up to 10 levels of nesting for variable substitution in config.rb

10 years agoRename config.yml to config.yml.example (easier to keep real configuration
Tim Pierce [Mon, 11 Nov 2013 21:38:01 +0000 (16:38 -0500)]
Rename config.yml to config.yml.example (easier to keep real configuration
settings out of the repo)

10 years agoWe don't need the pg gem in workbench.
Ward Vandewege [Mon, 11 Nov 2013 19:03:17 +0000 (14:03 -0500)]
We don't need the pg gem in workbench.

10 years agoadd log_stream_href attribute to Job resource
Tom Clegg [Mon, 11 Nov 2013 18:52:16 +0000 (10:52 -0800)]
add log_stream_href attribute to Job resource

10 years agoFix deprecation warning:
Ward Vandewege [Mon, 11 Nov 2013 17:52:46 +0000 (12:52 -0500)]
Fix deprecation warning:

  do not use ActionController::UnknownAction

10 years agoWe don't use the bootstrap-rails gem, and it does not exist anymore.
Ward Vandewege [Mon, 11 Nov 2013 17:52:09 +0000 (12:52 -0500)]
We don't use the bootstrap-rails gem, and it does not exist anymore.

10 years agoFix deprecation warning:
Ward Vandewege [Mon, 11 Nov 2013 17:52:46 +0000 (12:52 -0500)]
Fix deprecation warning:

  do not use ActionController::UnknownAction

10 years agoWe don't use the bootstrap-rails gem, and it does not exist anymore.
Ward Vandewege [Mon, 11 Nov 2013 17:52:09 +0000 (12:52 -0500)]
We don't use the bootstrap-rails gem, and it does not exist anymore.

10 years agoUse supervisord to start postgres + apache2 when the api
Tim Pierce [Mon, 11 Nov 2013 17:32:45 +0000 (12:32 -0500)]
Use supervisord to start postgres + apache2 when the api
container is launched.

Move generated files to an api/generated directory for
easier ignoring.

10 years agoStart the API container with supervisor, running postgres & apache2.
Tim Pierce [Fri, 8 Nov 2013 22:36:15 +0000 (17:36 -0500)]
Start the API container with supervisor, running postgres & apache2.
Reduce commands in Dockerfiles to stay under the 42-layer limit for AUFS
  (https://github.com/dotcloud/docker/issues/1171)

10 years agoInitial checkin for building warehouse-apps.
Tim Pierce [Fri, 8 Nov 2013 22:30:59 +0000 (17:30 -0500)]
Initial checkin for building warehouse-apps.

10 years agoCatch AbstractController::ActionNotFound instead of deprecated
Tom Clegg [Fri, 8 Nov 2013 22:00:33 +0000 (14:00 -0800)]
Catch AbstractController::ActionNotFound instead of deprecated
ActionController::UnknownAction

See http://guides.rubyonrails.org/3_2_release_notes.html

10 years agoadd crunch_worker_state to Node api response. refs #1586
Tom Clegg [Fri, 8 Nov 2013 21:16:18 +0000 (13:16 -0800)]
add crunch_worker_state to Node api response. refs #1586

10 years agofix tests and behavior for nodes#list as called by inactive user. refs #1594
Tom Clegg [Fri, 8 Nov 2013 21:14:13 +0000 (13:14 -0800)]
fix tests and behavior for nodes#list as called by inactive user. refs #1594

10 years agoFix passenger filename.
Tim Pierce [Fri, 8 Nov 2013 16:12:14 +0000 (11:12 -0500)]
Fix passenger filename.

10 years agoConfigure Apache and Passenger.
Tim Pierce [Fri, 8 Nov 2013 06:45:55 +0000 (01:45 -0500)]
Configure Apache and Passenger.

10 years agoBug fixes.
Tim Pierce [Fri, 8 Nov 2013 04:26:30 +0000 (23:26 -0500)]
Bug fixes.

10 years agoSpecify all configuration settings in config.yml.
Tim Pierce [Thu, 7 Nov 2013 22:41:45 +0000 (17:41 -0500)]
Specify all configuration settings in config.yml.

* Added README with step-by-step build instructions.
* config.rb substitutes config.yml variables in *.in files.

10 years agoAdd docker arvados/docserver build.
Ward Vandewege [Thu, 7 Nov 2013 22:18:30 +0000 (17:18 -0500)]
Add docker arvados/docserver build.

This closes #1580

10 years agoUse current_user to select API response template. fixes #1594
Tom Clegg [Thu, 7 Nov 2013 17:15:01 +0000 (09:15 -0800)]
Use current_user to select API response template. fixes #1594

10 years agoPreload all models at startup. Otherwise, in non-production mode,
Tom Clegg [Thu, 7 Nov 2013 17:27:03 +0000 (09:27 -0800)]
Preload all models at startup. Otherwise, in non-production mode,
Rails crashes trying to instantiate cached objects before loading the
relevant classes.

http://aaronvb.com/articles/37-rails-caching-and-undefined-class-module

10 years agoAdd "not null" constraint to timestamp columns.
Tom Clegg [Thu, 7 Nov 2013 17:24:09 +0000 (09:24 -0800)]
Add "not null" constraint to timestamp columns.

"When the timestamps method creates the created_at and updated_at
columns, it makes them non-nullable by default."

http://guides.rubyonrails.org/3_2_release_notes.html

10 years agoRemove "test user needs createdb privilege" workaround.
Tom Clegg [Thu, 7 Nov 2013 17:20:51 +0000 (09:20 -0800)]
Remove "test user needs createdb privilege" workaround.

It doesn't seem to work any more in Rails 3.2. Just accept that your
test user needs to own the test db and have the createdb privilege.

ALTER ROLE arvados_test_user CREATEDB

10 years agoMerging the postgres container back into the api container (simpler
Tim Pierce [Wed, 6 Nov 2013 23:11:51 +0000 (18:11 -0500)]
Merging the postgres container back into the api container (simpler
and more consistent with best practices).

'docker build -t arvados/api api' now builds a container with postgres
and apache2, and installs the API Rails database.

To have this container start multiple services at run time
(i.e. postgres + apache2) the popular solution is to make supervisord
the entrypoint for the container.

10 years agoSplitting out containers into arvados/base, arvados/postgresql and arvados/api.
Tim Pierce [Wed, 6 Nov 2013 20:45:59 +0000 (15:45 -0500)]
Splitting out containers into arvados/base, arvados/postgresql and arvados/api.

10 years agoMerge branch 'master' of git.clinicalfuture.com:arvados
Tim Pierce [Wed, 6 Nov 2013 14:44:52 +0000 (09:44 -0500)]
Merge branch 'master' of git.clinicalfuture.com:arvados

10 years agoRename "cfi" docker images to "arvados"
Tim Pierce [Tue, 5 Nov 2013 21:35:28 +0000 (16:35 -0500)]
Rename "cfi" docker images to "arvados"

Some fixes for PostgreSQL and gem installation.

10 years agoupgrade to latest Rails 3.2
Tom Clegg [Tue, 5 Nov 2013 20:41:45 +0000 (12:41 -0800)]
upgrade to latest Rails 3.2

10 years agoPost-install configuration script for PostgreSQL.
Tim Pierce [Tue, 5 Nov 2013 18:47:53 +0000 (13:47 -0500)]
Post-install configuration script for PostgreSQL.

10 years agoUse PostgreSQL, not MySQL.
Tim Pierce [Tue, 5 Nov 2013 18:43:34 +0000 (13:43 -0500)]
Use PostgreSQL, not MySQL.

10 years agoInitial build files for Docker images with Arvados support.
Tim Pierce [Mon, 4 Nov 2013 22:58:39 +0000 (17:58 -0500)]
Initial build files for Docker images with Arvados support.

docker/build.sh will build two Docker images locally:
  cfi/debian - a base Debian Wheezy image
  cfi/base   - cfi/debian, rvm, rails, MySQL, Arvados source code