Ward Vandewege [Sat, 8 Oct 2016 15:24:35 +0000 (11:24 -0400)]
Small naming fix for the config variables.
refs #10078
radhika [Fri, 7 Oct 2016 18:32:32 +0000 (14:32 -0400)]
10078: fix failing test due to new fixture addition
radhika [Fri, 7 Oct 2016 17:40:34 +0000 (13:40 -0400)]
10078: Since we are not fetching components for pipeline instances (at least until the performance issues are resolved),
we will no longer see the running and queued jobs listed within each pipeline instance row in dashboard. Hence, update
test to look for these for a container request row instead of for a pipeline instance row.
radhika [Fri, 7 Oct 2016 14:19:48 +0000 (10:19 -0400)]
10078: When fetching recent_processes, do not select 'components' which is not needed in this context. For very large pipelines, this will perform better.
radhika [Mon, 3 Oct 2016 22:58:59 +0000 (18:58 -0400)]
10078: Now that the recent_collections panel includes collections from Home directory as well
(self and other users' for admin user), suppress displaying confusing '(none)' in place of a Home directory.
radhika [Mon, 3 Oct 2016 21:07:43 +0000 (17:07 -0400)]
10078: add configuration parameter to suppress display of user notifications to improve dashboard performance
radhika [Mon, 3 Oct 2016 18:29:00 +0000 (14:29 -0400)]
10078: config parameter to hide / show recent_collections panel in dashboard.
radhika [Sat, 1 Oct 2016 02:54:09 +0000 (22:54 -0400)]
10078: preload collections and links during dashboard display
radhika [Fri, 30 Sep 2016 17:16:31 +0000 (13:16 -0400)]
10078: Restore master version
radhika [Fri, 30 Sep 2016 17:12:07 +0000 (13:12 -0400)]
Merge branch 'master' into 10078-dashboard-perf
Peter Amstutz [Fri, 30 Sep 2016 17:21:56 +0000 (13:21 -0400)]
Merge branch '10032-cwl-spinup' refs #10032
Peter Amstutz [Fri, 30 Sep 2016 15:58:49 +0000 (11:58 -0400)]
10032: Fix test to include --preserve-entire-environment
Peter Amstutz [Fri, 30 Sep 2016 15:24:53 +0000 (11:24 -0400)]
10032: Update schema salad and cwltool packages to get document validation optimizations.
Peter Amstutz [Fri, 30 Sep 2016 15:24:15 +0000 (11:24 -0400)]
Drill down on metrics on job submission.
radhika [Fri, 30 Sep 2016 14:40:41 +0000 (10:40 -0400)]
10078: Update recent_collections so that the collection query it is not executed twice, once in this method and once in show_dashboard for r[:collections].each statement.
radhika [Fri, 30 Sep 2016 14:36:07 +0000 (10:36 -0400)]
10078: Update the Node.filter so that this query is not executed three times: once for nodes.sort_by and twice in compute_node_summary for nodes.select
Tom Clegg [Thu, 29 Sep 2016 16:11:45 +0000 (12:11 -0400)]
Increase timeout and allow early success on download test. No issue #
radhika [Thu, 29 Sep 2016 15:09:30 +0000 (11:09 -0400)]
refs #9498 Fix failing workbench integration test due to containers fixture update.
Peter Amstutz [Thu, 29 Sep 2016 14:31:08 +0000 (10:31 -0400)]
Merge branch '10151-cwl-dir-listing' closes #10151
Peter Amstutz [Thu, 29 Sep 2016 14:30:30 +0000 (10:30 -0400)]
10032: Add more metrics
Peter Amstutz [Wed, 28 Sep 2016 20:26:45 +0000 (16:26 -0400)]
10151: Ensure that job_order_object is normalized and Directory listings are enumerated.
Tom Clegg [Wed, 28 Sep 2016 20:20:00 +0000 (16:20 -0400)]
Merge branch '9848-copy-container-output' refs #9848
Tom Clegg [Tue, 27 Sep 2016 16:49:49 +0000 (12:49 -0400)]
9848: Set expiry time on container output and log collections.
Tom Clegg [Tue, 27 Sep 2016 16:48:16 +0000 (12:48 -0400)]
9848: Use pointer receiver for all arvadosclient.ArvadosClient methods.
This makes it possible for all methods to satisfy a single interface.
Tom Clegg [Tue, 27 Sep 2016 03:18:24 +0000 (23:18 -0400)]
9848: Copy the output and log collections (if any) when finalizing a container request.
When reusing a container, ensure log and output are readable.
Tom Clegg [Tue, 27 Sep 2016 03:23:01 +0000 (23:23 -0400)]
9848: group some assertions into bigger test cases.
Tom Clegg [Wed, 28 Sep 2016 20:19:03 +0000 (16:19 -0400)]
Merge branch '9953-dockercleaner-config' closes #9953
Tom Clegg [Wed, 28 Sep 2016 20:18:00 +0000 (16:18 -0400)]
9953: Add arvados-docker-cleaner.service to MANIFEST.in.
Fixes error during `setup.py install`:
Command "/tmp/tmp.lpB0zPHYWs/VENV3DIR/bin/python3 -u -c "import setuptools, tokenize;__file__=/tmp/pip-6gqvw70n-build/setup.py;exec(compile(getattr(tokenize, open, open)(__file__).read().replace(rn, n), __file__, exec))" install --record /tmp/pip-ljhdcczu-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/tmp.lpB0zPHYWs/VENV3DIR/include/site/python3.4/arvados-docker-cleaner" failed with error code 1 in /tmp/pip-6gqvw70n-build/
Tom Clegg [Wed, 28 Sep 2016 15:13:40 +0000 (11:13 -0400)]
9950: 9953: 9954: 9955: 9957: Add AssertPathExists= to unit file to avoid startup until config file exists.
Tom Clegg [Wed, 28 Sep 2016 15:12:11 +0000 (11:12 -0400)]
9950: Fix arvados-git-httpd executable path. Update config path to /etc/arvados/git-httpd/.
Tom Clegg [Wed, 28 Sep 2016 14:47:36 +0000 (10:47 -0400)]
9953: Update default config path.
Tom Clegg [Wed, 28 Sep 2016 13:44:07 +0000 (09:44 -0400)]
9953: If config cannot be read/parsed, show just the error without the stack trace.
Tom Clegg [Fri, 23 Sep 2016 20:47:29 +0000 (16:47 -0400)]
9953: Install/remove systemd unit files wherever systemd exists, even if it is not running.
Tom Clegg [Fri, 23 Sep 2016 20:41:01 +0000 (16:41 -0400)]
9953: Update build scripts to Go 1.7.1.
Tom Clegg [Fri, 23 Sep 2016 20:40:37 +0000 (16:40 -0400)]
9953: De-duplicate Go and Python postinst/prerm scripts.
Tom Clegg [Fri, 23 Sep 2016 20:08:37 +0000 (16:08 -0400)]
9953: Add systemd unit file for arvados-docker-cleaner.
Tom Clegg [Fri, 23 Sep 2016 20:09:03 +0000 (16:09 -0400)]
9953: Install python data files to /usr/share/... not /usr/data/share/...
Tom Clegg [Fri, 23 Sep 2016 14:26:10 +0000 (10:26 -0400)]
9953: PEP-8
Tom Clegg [Fri, 23 Sep 2016 14:17:59 +0000 (10:17 -0400)]
9953: Load config from file.
Tom Clegg [Fri, 23 Sep 2016 03:58:30 +0000 (23:58 -0400)]
9953: Ignore non-container events (volume, network) and events with no status, instead of crashing.
Ward Vandewege [Wed, 28 Sep 2016 17:35:56 +0000 (13:35 -0400)]
Use our cached copy of phantomjs-1.9.7 instead of pulling it from https
upstream with flakey connectivity.
closes #10130
Tom Clegg [Wed, 28 Sep 2016 13:59:36 +0000 (09:59 -0400)]
Merge branch '9990-workbench-client-timeout' closes #9990
Tom Clegg [Thu, 8 Sep 2016 14:41:37 +0000 (10:41 -0400)]
9990: Add configs for Workbench API client timeouts.
Peter Amstutz [Tue, 27 Sep 2016 20:36:58 +0000 (16:36 -0400)]
Merge branch '10129-cwl-remove-listing' closes #10129
Peter Amstutz [Tue, 27 Sep 2016 20:30:27 +0000 (16:30 -0400)]
10129: Rename del_listing to trim_listing and add docstring about what it is
for.
Tom Clegg [Tue, 27 Sep 2016 20:11:22 +0000 (16:11 -0400)]
Merge branch '9912-config-yaml' closes #9912
Peter Amstutz [Tue, 27 Sep 2016 19:57:40 +0000 (15:57 -0400)]
10129: Ensure that directory literals get preserved when trimming keep directory listings.
Tom Clegg [Tue, 27 Sep 2016 18:00:21 +0000 (14:00 -0400)]
9912: Support YAML config files. Change default config file paths to /etc/arvados/foo/foo.yml.
Peter Amstutz [Tue, 27 Sep 2016 17:42:04 +0000 (13:42 -0400)]
Fix #10133, use "resolved" (keep reference) instead of target (filesystem
path). I think this was a merge conflict mistake.
Peter Amstutz [Mon, 26 Sep 2016 20:49:27 +0000 (16:49 -0400)]
10129: Trim listing from Directory objects that are keep references, because
they can get very large.
Peter Amstutz [Mon, 26 Sep 2016 21:17:35 +0000 (17:17 -0400)]
Merge branch '10081-update-cwl-runner' refs #10081
Peter Amstutz [Mon, 26 Sep 2016 21:04:29 +0000 (17:04 -0400)]
10081: Fix renamed methods.
radhika [Mon, 26 Sep 2016 20:03:17 +0000 (16:03 -0400)]
closes #9514
Merge branch '9514-only-delete-old-container-logs'
Peter Amstutz [Mon, 26 Sep 2016 19:30:39 +0000 (15:30 -0400)]
Merge branch '9988-cwl-arv-hints' closes #9988
Peter Amstutz [Mon, 26 Sep 2016 19:30:17 +0000 (15:30 -0400)]
Fix typo.
Tom Clegg [Mon, 26 Sep 2016 17:44:16 +0000 (13:44 -0400)]
Merge branch '9950-arv-git-httpd-config' closes #9950
Tom Clegg [Mon, 26 Sep 2016 16:24:46 +0000 (12:24 -0400)]
9950: Remove uses of environment variables.
Tom Clegg [Wed, 21 Sep 2016 18:34:05 +0000 (14:34 -0400)]
9950: Propagate Client.Insecure config to arvadosclient library via ARVADOS_API_HOST_INSECURE env var.
Tom Clegg [Wed, 21 Sep 2016 01:28:00 +0000 (21:28 -0400)]
9950: Add systemd unit file arv-git-httpd.service.
Tom Clegg [Wed, 21 Sep 2016 13:35:27 +0000 (09:35 -0400)]
9950: Rename Root config to RepoRoot.
Tom Clegg [Wed, 21 Sep 2016 01:27:32 +0000 (21:27 -0400)]
9950: Load config from /etc/arvados/arv-git-httpd/config.json.
radhika [Mon, 26 Sep 2016 14:53:38 +0000 (10:53 -0400)]
refs #10028
Merge branch '10028-search-uuid-prefix'
radhika [Mon, 26 Sep 2016 14:15:33 +0000 (10:15 -0400)]
9514: update delete_old_job_logs task also to use the better performing sql.
Peter Amstutz [Mon, 26 Sep 2016 12:42:43 +0000 (08:42 -0400)]
Merge branch '10081-cwl-run-same-job' closes #10081
radhika [Sat, 24 Sep 2016 23:31:09 +0000 (19:31 -0400)]
9514: Use sql to issue delete sql without getting ids in batch.
Tom Clegg [Sat, 24 Sep 2016 02:31:54 +0000 (22:31 -0400)]
Merge branch '10124-fix-sleepy-tests' closes #10124
Tom Clegg [Fri, 23 Sep 2016 18:50:23 +0000 (14:50 -0400)]
10124: Replace "sleep 1, then assert" with "retry assertion for up to 10 seconds".
Peter Amstutz [Fri, 23 Sep 2016 20:22:10 +0000 (16:22 -0400)]
Bump cwltool version in fpm_build refs #10088
Peter Amstutz [Fri, 23 Sep 2016 20:20:07 +0000 (16:20 -0400)]
Merge branch '10081-cwl-run-same-job' into 9988-cwl-arv-hints
Peter Amstutz [Fri, 23 Sep 2016 20:18:39 +0000 (16:18 -0400)]
Merge branch 'master' into 10081-cwl-run-same-job
Conflicts:
sdk/cwl/arvados_cwl/runner.py
Peter Amstutz [Fri, 23 Sep 2016 20:14:06 +0000 (16:14 -0400)]
Merge branch '10088-cwl-dedup-deps' refs #10088
Peter Amstutz [Fri, 23 Sep 2016 18:27:35 +0000 (14:27 -0400)]
10081: Change logical identifier for Arvados schema to arvados.org/cwl.
Improve exception text when checking that locations are keep references.
radhika [Fri, 23 Sep 2016 18:09:20 +0000 (14:09 -0400)]
9514: if logs are not found in logs table, do not show the log div for a completed container.
Peter Amstutz [Fri, 23 Sep 2016 02:38:06 +0000 (22:38 -0400)]
10088: Updated cwltool deduplicates dependencies to avoid creating separate
collections for subdirectories. Don't try to update file literals that arn't
in scandeps.
Tom Clegg [Fri, 23 Sep 2016 16:06:54 +0000 (12:06 -0400)]
Use longer timeout for flaky test.
No issue #
Tom Clegg [Fri, 23 Sep 2016 15:34:14 +0000 (11:34 -0400)]
gofmt. refs #9955
radhika [Fri, 23 Sep 2016 15:31:00 +0000 (11:31 -0400)]
9514: fixture correction
Tom Clegg [Fri, 23 Sep 2016 15:00:05 +0000 (11:00 -0400)]
Merge branch '9955-keepproxy-config' closes #9955
radhika [Fri, 23 Sep 2016 14:09:13 +0000 (10:09 -0400)]
Merge branch 'master' into 9514-only-delete-old-container-logs
radhika [Thu, 22 Sep 2016 23:16:12 +0000 (19:16 -0400)]
10028: replace _____-uuid-prefix with site_config_uuid_prefix-uuid-prefix
radhika [Thu, 22 Sep 2016 18:32:16 +0000 (14:32 -0400)]
closes #10036
Merge branch '10036-canceled-container-state'
radhika [Thu, 22 Sep 2016 02:50:24 +0000 (22:50 -0400)]
10036: Use status label "Ready" for a canceled container request (priority 0), when it's container_uuid is in Queued or Locked states.
Lucas Di Pentima [Thu, 22 Sep 2016 17:02:34 +0000 (14:02 -0300)]
Merge branch '10037-useless-progress-bar'
Closes #10037
Lucas Di Pentima [Thu, 22 Sep 2016 17:01:40 +0000 (14:01 -0300)]
10037: Merge branch 'master' into 10037-useless-progress-bar
Lucas Di Pentima [Thu, 22 Sep 2016 17:00:37 +0000 (14:00 -0300)]
10037: Changed test name to something more appropiate.
Peter Amstutz [Thu, 22 Sep 2016 16:29:33 +0000 (12:29 -0400)]
9988: Add arvados-specfic hints RuntimeConstraints and APIRequirement.
Peter Amstutz [Thu, 22 Sep 2016 16:39:44 +0000 (12:39 -0400)]
Merge branch '10117-fuse-refcount' refs #10117
Peter Amstutz [Thu, 22 Sep 2016 16:38:17 +0000 (12:38 -0400)]
10117: Revert mistaken change to Gemfile.lock.
Peter Amstutz [Thu, 22 Sep 2016 16:32:32 +0000 (12:32 -0400)]
10081: Rename cwl.input.json to cwl.input.yml, fix test.
Peter Amstutz [Thu, 22 Sep 2016 15:38:21 +0000 (11:38 -0400)]
10117: Tweak behavior of has_ref(). Fix naming of CacheTest.
Lucas Di Pentima [Thu, 22 Sep 2016 15:31:23 +0000 (12:31 -0300)]
10037: Test enhancement to check for other statuses on the dashboard, also adding pipeline instances on those checks.
Peter Amstutz [Thu, 22 Sep 2016 13:58:57 +0000 (09:58 -0400)]
Merge branch 'master' into 10081-cwl-run-same-job
Peter Amstutz [Thu, 22 Sep 2016 13:57:56 +0000 (09:57 -0400)]
Update cwltool again for bugfix, refs #10088
Lucas Di Pentima [Thu, 22 Sep 2016 13:20:32 +0000 (10:20 -0300)]
10037: Fixed alignment issues on the dashboard
Peter Amstutz [Wed, 21 Sep 2016 22:10:54 +0000 (18:10 -0400)]
10117: The check needs to be between clear() and the deletion from entries.
Peter Amstutz [Wed, 21 Sep 2016 21:58:29 +0000 (17:58 -0400)]
10117: Check that inode is in entries before trying to remove it, along with
explanation why it is necessary.
Lucas Di Pentima [Wed, 21 Sep 2016 21:37:07 +0000 (18:37 -0300)]
Merge branch '10029-wb-send-request-id'
Closes #10029
Lucas Di Pentima [Wed, 21 Sep 2016 21:36:04 +0000 (18:36 -0300)]
10029: Merge branch 'master' into 10029-wb-send-request-id
Lucas Di Pentima [Wed, 21 Sep 2016 21:20:20 +0000 (18:20 -0300)]
10037: Fixed test because of the fixture dates changes made to allow some containers to appear on the recent pipelines and processes list on the dashboard
Lucas Di Pentima [Wed, 21 Sep 2016 21:19:23 +0000 (18:19 -0300)]
10037: Added test to check for progress bar showing only on running containers