arvados.git
7 years agoSmall naming fix for the config variables.
Ward Vandewege [Sat, 8 Oct 2016 15:24:35 +0000 (11:24 -0400)]
Small naming fix for the config variables.

refs #10078

7 years ago10078: fix failing test due to new fixture addition
radhika [Fri, 7 Oct 2016 18:32:32 +0000 (14:32 -0400)]
10078: fix failing test due to new fixture addition

7 years ago10078: Since we are not fetching components for pipeline instances (at least until...
radhika [Fri, 7 Oct 2016 17:40:34 +0000 (13:40 -0400)]
10078: Since we are not fetching components for pipeline instances (at least until the performance issues are resolved),
we will no longer see the running and queued jobs listed within each pipeline instance row in dashboard. Hence, update
test to look for these for a container request row instead of for a pipeline instance row.

7 years ago10078: When fetching recent_processes, do not select 'components' which is not needed...
radhika [Fri, 7 Oct 2016 14:19:48 +0000 (10:19 -0400)]
10078: When fetching recent_processes, do not select 'components' which is not needed in this context. For very large pipelines, this will perform better.

7 years ago10078: Now that the recent_collections panel includes collections from Home directory...
radhika [Mon, 3 Oct 2016 22:58:59 +0000 (18:58 -0400)]
10078: Now that the recent_collections panel includes collections from Home directory as well
(self and other users' for admin user), suppress displaying confusing '(none)' in place of a Home directory.

7 years ago10078: add configuration parameter to suppress display of user notifications to impro...
radhika [Mon, 3 Oct 2016 21:07:43 +0000 (17:07 -0400)]
10078: add configuration parameter to suppress display of user notifications to improve dashboard performance

7 years ago10078: config parameter to hide / show recent_collections panel in dashboard.
radhika [Mon, 3 Oct 2016 18:29:00 +0000 (14:29 -0400)]
10078: config parameter to hide / show recent_collections panel in dashboard.

7 years ago10078: preload collections and links during dashboard display
radhika [Sat, 1 Oct 2016 02:54:09 +0000 (22:54 -0400)]
10078: preload collections and links during dashboard display

7 years ago10078: Restore master version
radhika [Fri, 30 Sep 2016 17:16:31 +0000 (13:16 -0400)]
10078: Restore master version

7 years agoMerge branch 'master' into 10078-dashboard-perf
radhika [Fri, 30 Sep 2016 17:12:07 +0000 (13:12 -0400)]
Merge branch 'master' into 10078-dashboard-perf

7 years agoMerge branch '10032-cwl-spinup' refs #10032
Peter Amstutz [Fri, 30 Sep 2016 17:21:56 +0000 (13:21 -0400)]
Merge branch '10032-cwl-spinup' refs #10032

7 years ago10032: Fix test to include --preserve-entire-environment
Peter Amstutz [Fri, 30 Sep 2016 15:58:49 +0000 (11:58 -0400)]
10032: Fix test to include --preserve-entire-environment

7 years ago10032: Update schema salad and cwltool packages to get document validation optimizations.
Peter Amstutz [Fri, 30 Sep 2016 15:24:53 +0000 (11:24 -0400)]
10032: Update schema salad and cwltool packages to get document validation optimizations.

7 years agoDrill down on metrics on job submission.
Peter Amstutz [Fri, 30 Sep 2016 15:24:15 +0000 (11:24 -0400)]
Drill down on metrics on job submission.

7 years ago10078: Update recent_collections so that the collection query it is not executed...
radhika [Fri, 30 Sep 2016 14:40:41 +0000 (10:40 -0400)]
10078: Update recent_collections so that the collection query it is not executed twice, once in this method and once in show_dashboard for r[:collections].each statement.

7 years ago10078: Update the Node.filter so that this query is not executed three times: once...
radhika [Fri, 30 Sep 2016 14:36:07 +0000 (10:36 -0400)]
10078: Update the Node.filter so that this query is not executed three times: once for nodes.sort_by and twice in compute_node_summary for nodes.select

7 years agoIncrease timeout and allow early success on download test. No issue #
Tom Clegg [Thu, 29 Sep 2016 16:11:45 +0000 (12:11 -0400)]
Increase timeout and allow early success on download test. No issue #

7 years agorefs #9498 Fix failing workbench integration test due to containers fixture update.
radhika [Thu, 29 Sep 2016 15:09:30 +0000 (11:09 -0400)]
refs #9498 Fix failing workbench integration test due to containers fixture update.

7 years agoMerge branch '10151-cwl-dir-listing' closes #10151
Peter Amstutz [Thu, 29 Sep 2016 14:31:08 +0000 (10:31 -0400)]
Merge branch '10151-cwl-dir-listing' closes #10151

7 years ago10032: Add more metrics
Peter Amstutz [Thu, 29 Sep 2016 14:30:30 +0000 (10:30 -0400)]
10032: Add more metrics

7 years ago10151: Ensure that job_order_object is normalized and Directory listings are enumerated.
Peter Amstutz [Wed, 28 Sep 2016 20:26:45 +0000 (16:26 -0400)]
10151: Ensure that job_order_object is normalized and Directory listings are enumerated.

7 years agoMerge branch '9848-copy-container-output' refs #9848
Tom Clegg [Wed, 28 Sep 2016 20:20:00 +0000 (16:20 -0400)]
Merge branch '9848-copy-container-output' refs #9848

7 years ago9848: Set expiry time on container output and log collections.
Tom Clegg [Tue, 27 Sep 2016 16:49:49 +0000 (12:49 -0400)]
9848: Set expiry time on container output and log collections.

7 years ago9848: Use pointer receiver for all arvadosclient.ArvadosClient methods.
Tom Clegg [Tue, 27 Sep 2016 16:48:16 +0000 (12:48 -0400)]
9848: Use pointer receiver for all arvadosclient.ArvadosClient methods.

This makes it possible for all methods to satisfy a single interface.

7 years ago9848: Copy the output and log collections (if any) when finalizing a container request.
Tom Clegg [Tue, 27 Sep 2016 03:18:24 +0000 (23:18 -0400)]
9848: Copy the output and log collections (if any) when finalizing a container request.

When reusing a container, ensure log and output are readable.

7 years ago9848: group some assertions into bigger test cases.
Tom Clegg [Tue, 27 Sep 2016 03:23:01 +0000 (23:23 -0400)]
9848: group some assertions into bigger test cases.

7 years agoMerge branch '9953-dockercleaner-config' closes #9953
Tom Clegg [Wed, 28 Sep 2016 20:19:03 +0000 (16:19 -0400)]
Merge branch '9953-dockercleaner-config' closes #9953

7 years ago9953: Add arvados-docker-cleaner.service to MANIFEST.in.
Tom Clegg [Wed, 28 Sep 2016 20:18:00 +0000 (16:18 -0400)]
9953: Add arvados-docker-cleaner.service to MANIFEST.in.

Fixes error during `setup.py install`:

Command "/tmp/tmp.lpB0zPHYWs/VENV3DIR/bin/python3 -u -c "import setuptools, tokenize;__file__=/tmp/pip-6gqvw70n-build/setup.py;exec(compile(getattr(tokenize, open, open)(__file__).read().replace(rn, n), __file__, exec))" install --record /tmp/pip-ljhdcczu-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/tmp.lpB0zPHYWs/VENV3DIR/include/site/python3.4/arvados-docker-cleaner" failed with error code 1 in /tmp/pip-6gqvw70n-build/

7 years ago9950: 9953: 9954: 9955: 9957: Add AssertPathExists= to unit file to avoid startup...
Tom Clegg [Wed, 28 Sep 2016 15:13:40 +0000 (11:13 -0400)]
9950: 9953: 9954: 9955: 9957: Add AssertPathExists= to unit file to avoid startup until config file exists.

7 years ago9950: Fix arvados-git-httpd executable path. Update config path to /etc/arvados/git...
Tom Clegg [Wed, 28 Sep 2016 15:12:11 +0000 (11:12 -0400)]
9950: Fix arvados-git-httpd executable path. Update config path to /etc/arvados/git-httpd/.

7 years ago9953: Update default config path.
Tom Clegg [Wed, 28 Sep 2016 14:47:36 +0000 (10:47 -0400)]
9953: Update default config path.

7 years ago9953: If config cannot be read/parsed, show just the error without the stack trace.
Tom Clegg [Wed, 28 Sep 2016 13:44:07 +0000 (09:44 -0400)]
9953: If config cannot be read/parsed, show just the error without the stack trace.

7 years ago9953: Install/remove systemd unit files wherever systemd exists, even if it is not...
Tom Clegg [Fri, 23 Sep 2016 20:47:29 +0000 (16:47 -0400)]
9953: Install/remove systemd unit files wherever systemd exists, even if it is not running.

7 years ago9953: Update build scripts to Go 1.7.1.
Tom Clegg [Fri, 23 Sep 2016 20:41:01 +0000 (16:41 -0400)]
9953: Update build scripts to Go 1.7.1.

7 years ago9953: De-duplicate Go and Python postinst/prerm scripts.
Tom Clegg [Fri, 23 Sep 2016 20:40:37 +0000 (16:40 -0400)]
9953: De-duplicate Go and Python postinst/prerm scripts.

7 years ago9953: Add systemd unit file for arvados-docker-cleaner.
Tom Clegg [Fri, 23 Sep 2016 20:08:37 +0000 (16:08 -0400)]
9953: Add systemd unit file for arvados-docker-cleaner.

7 years ago9953: Install python data files to /usr/share/... not /usr/data/share/...
Tom Clegg [Fri, 23 Sep 2016 20:09:03 +0000 (16:09 -0400)]
9953: Install python data files to /usr/share/... not /usr/data/share/...

7 years ago9953: PEP-8
Tom Clegg [Fri, 23 Sep 2016 14:26:10 +0000 (10:26 -0400)]
9953: PEP-8

7 years ago9953: Load config from file.
Tom Clegg [Fri, 23 Sep 2016 14:17:59 +0000 (10:17 -0400)]
9953: Load config from file.

7 years ago9953: Ignore non-container events (volume, network) and events with no status, instea...
Tom Clegg [Fri, 23 Sep 2016 03:58:30 +0000 (23:58 -0400)]
9953: Ignore non-container events (volume, network) and events with no status, instead of crashing.

7 years agoUse our cached copy of phantomjs-1.9.7 instead of pulling it from https
Ward Vandewege [Wed, 28 Sep 2016 17:35:56 +0000 (13:35 -0400)]
Use our cached copy of phantomjs-1.9.7 instead of pulling it from https
upstream with flakey connectivity.

closes #10130

7 years agoMerge branch '9990-workbench-client-timeout' closes #9990
Tom Clegg [Wed, 28 Sep 2016 13:59:36 +0000 (09:59 -0400)]
Merge branch '9990-workbench-client-timeout' closes #9990

7 years ago9990: Add configs for Workbench API client timeouts.
Tom Clegg [Thu, 8 Sep 2016 14:41:37 +0000 (10:41 -0400)]
9990: Add configs for Workbench API client timeouts.

7 years agoMerge branch '10129-cwl-remove-listing' closes #10129
Peter Amstutz [Tue, 27 Sep 2016 20:36:58 +0000 (16:36 -0400)]
Merge branch '10129-cwl-remove-listing' closes #10129

7 years ago10129: Rename del_listing to trim_listing and add docstring about what it is
Peter Amstutz [Tue, 27 Sep 2016 20:30:27 +0000 (16:30 -0400)]
10129: Rename del_listing to trim_listing and add docstring about what it is
for.

7 years agoMerge branch '9912-config-yaml' closes #9912
Tom Clegg [Tue, 27 Sep 2016 20:11:22 +0000 (16:11 -0400)]
Merge branch '9912-config-yaml' closes #9912

7 years ago10129: Ensure that directory literals get preserved when trimming keep directory...
Peter Amstutz [Tue, 27 Sep 2016 19:57:40 +0000 (15:57 -0400)]
10129: Ensure that directory literals get preserved when trimming keep directory listings.

7 years ago9912: Support YAML config files. Change default config file paths to /etc/arvados...
Tom Clegg [Tue, 27 Sep 2016 18:00:21 +0000 (14:00 -0400)]
9912: Support YAML config files. Change default config file paths to /etc/arvados/foo/foo.yml.

7 years agoFix #10133, use "resolved" (keep reference) instead of target (filesystem
Peter Amstutz [Tue, 27 Sep 2016 17:42:04 +0000 (13:42 -0400)]
Fix #10133, use "resolved" (keep reference) instead of target (filesystem
path).  I think this was a merge conflict mistake.

7 years ago10129: Trim listing from Directory objects that are keep references, because
Peter Amstutz [Mon, 26 Sep 2016 20:49:27 +0000 (16:49 -0400)]
10129: Trim listing from Directory objects that are keep references, because
they can get very large.

7 years agoMerge branch '10081-update-cwl-runner' refs #10081
Peter Amstutz [Mon, 26 Sep 2016 21:17:35 +0000 (17:17 -0400)]
Merge branch '10081-update-cwl-runner'  refs #10081

7 years ago10081: Fix renamed methods.
Peter Amstutz [Mon, 26 Sep 2016 21:04:29 +0000 (17:04 -0400)]
10081: Fix renamed methods.

7 years agocloses #9514
radhika [Mon, 26 Sep 2016 20:03:17 +0000 (16:03 -0400)]
closes #9514
Merge branch '9514-only-delete-old-container-logs'

7 years agoMerge branch '9988-cwl-arv-hints' closes #9988
Peter Amstutz [Mon, 26 Sep 2016 19:30:39 +0000 (15:30 -0400)]
Merge branch '9988-cwl-arv-hints' closes #9988

7 years agoFix typo.
Peter Amstutz [Mon, 26 Sep 2016 19:30:17 +0000 (15:30 -0400)]
Fix typo.

7 years agoMerge branch '9950-arv-git-httpd-config' closes #9950
Tom Clegg [Mon, 26 Sep 2016 17:44:16 +0000 (13:44 -0400)]
Merge branch '9950-arv-git-httpd-config' closes #9950

7 years ago9950: Remove uses of environment variables.
Tom Clegg [Mon, 26 Sep 2016 16:24:46 +0000 (12:24 -0400)]
9950: Remove uses of environment variables.

7 years ago9950: Propagate Client.Insecure config to arvadosclient library via ARVADOS_API_HOST_...
Tom Clegg [Wed, 21 Sep 2016 18:34:05 +0000 (14:34 -0400)]
9950: Propagate Client.Insecure config to arvadosclient library via ARVADOS_API_HOST_INSECURE env var.

7 years ago9950: Add systemd unit file arv-git-httpd.service.
Tom Clegg [Wed, 21 Sep 2016 01:28:00 +0000 (21:28 -0400)]
9950: Add systemd unit file arv-git-httpd.service.

7 years ago9950: Rename Root config to RepoRoot.
Tom Clegg [Wed, 21 Sep 2016 13:35:27 +0000 (09:35 -0400)]
9950: Rename Root config to RepoRoot.

7 years ago9950: Load config from /etc/arvados/arv-git-httpd/config.json.
Tom Clegg [Wed, 21 Sep 2016 01:27:32 +0000 (21:27 -0400)]
9950: Load config from /etc/arvados/arv-git-httpd/config.json.

7 years agorefs #10028
radhika [Mon, 26 Sep 2016 14:53:38 +0000 (10:53 -0400)]
refs #10028
Merge branch '10028-search-uuid-prefix'

7 years ago9514: update delete_old_job_logs task also to use the better performing sql.
radhika [Mon, 26 Sep 2016 14:15:33 +0000 (10:15 -0400)]
9514: update delete_old_job_logs task also to use the better performing sql.

7 years agoMerge branch '10081-cwl-run-same-job' closes #10081
Peter Amstutz [Mon, 26 Sep 2016 12:42:43 +0000 (08:42 -0400)]
Merge branch '10081-cwl-run-same-job' closes #10081

7 years ago9514: Use sql to issue delete sql without getting ids in batch.
radhika [Sat, 24 Sep 2016 23:31:09 +0000 (19:31 -0400)]
9514: Use sql to issue delete sql without getting ids in batch.

7 years agoMerge branch '10124-fix-sleepy-tests' closes #10124
Tom Clegg [Sat, 24 Sep 2016 02:31:54 +0000 (22:31 -0400)]
Merge branch '10124-fix-sleepy-tests' closes #10124

7 years ago10124: Replace "sleep 1, then assert" with "retry assertion for up to 10 seconds".
Tom Clegg [Fri, 23 Sep 2016 18:50:23 +0000 (14:50 -0400)]
10124: Replace "sleep 1, then assert" with "retry assertion for up to 10 seconds".

7 years agoBump cwltool version in fpm_build refs #10088
Peter Amstutz [Fri, 23 Sep 2016 20:22:10 +0000 (16:22 -0400)]
Bump cwltool version in fpm_build refs #10088

7 years agoMerge branch '10081-cwl-run-same-job' into 9988-cwl-arv-hints
Peter Amstutz [Fri, 23 Sep 2016 20:20:07 +0000 (16:20 -0400)]
Merge branch '10081-cwl-run-same-job' into 9988-cwl-arv-hints

7 years agoMerge branch 'master' into 10081-cwl-run-same-job
Peter Amstutz [Fri, 23 Sep 2016 20:18:39 +0000 (16:18 -0400)]
Merge branch 'master' into 10081-cwl-run-same-job

Conflicts:
sdk/cwl/arvados_cwl/runner.py

7 years agoMerge branch '10088-cwl-dedup-deps' refs #10088
Peter Amstutz [Fri, 23 Sep 2016 20:14:06 +0000 (16:14 -0400)]
Merge branch '10088-cwl-dedup-deps' refs #10088

7 years ago10081: Change logical identifier for Arvados schema to http://arvados.org/cwl.
Peter Amstutz [Fri, 23 Sep 2016 18:27:35 +0000 (14:27 -0400)]
10081: Change logical identifier for Arvados schema to arvados.org/cwl.
Improve exception text when checking that locations are keep references.

7 years ago9514: if logs are not found in logs table, do not show the log div for a completed...
radhika [Fri, 23 Sep 2016 18:09:20 +0000 (14:09 -0400)]
9514: if logs are not found in logs table, do not show the log div for a completed container.

7 years ago10088: Updated cwltool deduplicates dependencies to avoid creating separate
Peter Amstutz [Fri, 23 Sep 2016 02:38:06 +0000 (22:38 -0400)]
10088: Updated cwltool deduplicates dependencies to avoid creating separate
collections for subdirectories.  Don't try to update file literals that arn't
in scandeps.

7 years agoUse longer timeout for flaky test.
Tom Clegg [Fri, 23 Sep 2016 16:06:54 +0000 (12:06 -0400)]
Use longer timeout for flaky test.

No issue #

7 years agogofmt. refs #9955
Tom Clegg [Fri, 23 Sep 2016 15:34:14 +0000 (11:34 -0400)]
gofmt. refs #9955

7 years ago9514: fixture correction
radhika [Fri, 23 Sep 2016 15:31:00 +0000 (11:31 -0400)]
9514: fixture correction

7 years agoMerge branch '9955-keepproxy-config' closes #9955
Tom Clegg [Fri, 23 Sep 2016 15:00:05 +0000 (11:00 -0400)]
Merge branch '9955-keepproxy-config' closes #9955

7 years agoMerge branch 'master' into 9514-only-delete-old-container-logs
radhika [Fri, 23 Sep 2016 14:09:13 +0000 (10:09 -0400)]
Merge branch 'master' into 9514-only-delete-old-container-logs

7 years ago10028: replace _____-uuid-prefix with site_config_uuid_prefix-uuid-prefix
radhika [Thu, 22 Sep 2016 23:16:12 +0000 (19:16 -0400)]
10028: replace _____-uuid-prefix with site_config_uuid_prefix-uuid-prefix

7 years agocloses #10036
radhika [Thu, 22 Sep 2016 18:32:16 +0000 (14:32 -0400)]
closes #10036
Merge branch '10036-canceled-container-state'

7 years ago10036: Use status label "Ready" for a canceled container request (priority 0), when...
radhika [Thu, 22 Sep 2016 02:50:24 +0000 (22:50 -0400)]
10036: Use status label "Ready" for a canceled container request (priority 0), when it's container_uuid is in Queued or Locked states.

7 years agoMerge branch '10037-useless-progress-bar'
Lucas Di Pentima [Thu, 22 Sep 2016 17:02:34 +0000 (14:02 -0300)]
Merge branch '10037-useless-progress-bar'
Closes #10037

7 years ago10037: Merge branch 'master' into 10037-useless-progress-bar
Lucas Di Pentima [Thu, 22 Sep 2016 17:01:40 +0000 (14:01 -0300)]
10037: Merge branch 'master' into 10037-useless-progress-bar

7 years ago10037: Changed test name to something more appropiate.
Lucas Di Pentima [Thu, 22 Sep 2016 17:00:37 +0000 (14:00 -0300)]
10037: Changed test name to something more appropiate.

7 years ago9988: Add arvados-specfic hints RuntimeConstraints and APIRequirement.
Peter Amstutz [Thu, 22 Sep 2016 16:29:33 +0000 (12:29 -0400)]
9988: Add arvados-specfic hints RuntimeConstraints and APIRequirement.

7 years agoMerge branch '10117-fuse-refcount' refs #10117
Peter Amstutz [Thu, 22 Sep 2016 16:39:44 +0000 (12:39 -0400)]
Merge branch '10117-fuse-refcount' refs #10117

7 years ago10117: Revert mistaken change to Gemfile.lock.
Peter Amstutz [Thu, 22 Sep 2016 16:38:17 +0000 (12:38 -0400)]
10117: Revert mistaken change to Gemfile.lock.

7 years ago10081: Rename cwl.input.json to cwl.input.yml, fix test.
Peter Amstutz [Thu, 22 Sep 2016 16:32:32 +0000 (12:32 -0400)]
10081: Rename cwl.input.json to cwl.input.yml, fix test.

7 years ago10117: Tweak behavior of has_ref(). Fix naming of CacheTest.
Peter Amstutz [Thu, 22 Sep 2016 15:38:21 +0000 (11:38 -0400)]
10117: Tweak behavior of has_ref().  Fix naming of CacheTest.

7 years ago10037: Test enhancement to check for other statuses on the dashboard, also adding...
Lucas Di Pentima [Thu, 22 Sep 2016 15:31:23 +0000 (12:31 -0300)]
10037: Test enhancement to check for other statuses on the dashboard, also adding pipeline instances on those checks.

7 years agoMerge branch 'master' into 10081-cwl-run-same-job
Peter Amstutz [Thu, 22 Sep 2016 13:58:57 +0000 (09:58 -0400)]
Merge branch 'master' into 10081-cwl-run-same-job

7 years agoUpdate cwltool again for bugfix, refs #10088
Peter Amstutz [Thu, 22 Sep 2016 13:57:56 +0000 (09:57 -0400)]
Update cwltool again for bugfix, refs #10088

7 years ago10037: Fixed alignment issues on the dashboard
Lucas Di Pentima [Thu, 22 Sep 2016 13:20:32 +0000 (10:20 -0300)]
10037: Fixed alignment issues on the dashboard

7 years ago10117: The check needs to be between clear() and the deletion from entries.
Peter Amstutz [Wed, 21 Sep 2016 22:10:54 +0000 (18:10 -0400)]
10117: The check needs to be between clear() and the deletion from entries.

7 years ago10117: Check that inode is in entries before trying to remove it, along with
Peter Amstutz [Wed, 21 Sep 2016 21:58:29 +0000 (17:58 -0400)]
10117: Check that inode is in entries before trying to remove it, along with
explanation why it is necessary.

7 years agoMerge branch '10029-wb-send-request-id'
Lucas Di Pentima [Wed, 21 Sep 2016 21:37:07 +0000 (18:37 -0300)]
Merge branch '10029-wb-send-request-id'
Closes #10029

7 years ago10029: Merge branch 'master' into 10029-wb-send-request-id
Lucas Di Pentima [Wed, 21 Sep 2016 21:36:04 +0000 (18:36 -0300)]
10029: Merge branch 'master' into 10029-wb-send-request-id

7 years ago10037: Fixed test because of the fixture dates changes made to allow some containers...
Lucas Di Pentima [Wed, 21 Sep 2016 21:20:20 +0000 (18:20 -0300)]
10037: Fixed test because of the fixture dates changes made to allow some containers to appear on the recent pipelines and processes list on the dashboard

7 years ago10037: Added test to check for progress bar showing only on running containers
Lucas Di Pentima [Wed, 21 Sep 2016 21:19:23 +0000 (18:19 -0300)]
10037: Added test to check for progress bar showing only on running containers