arvados.git
7 years ago10764: Test v0 session.
Tom Clegg [Wed, 1 Mar 2017 07:36:18 +0000 (02:36 -0500)]
10764: Test v0 session.

7 years ago10764: Permission tests. Support PDH permission check.
Tom Clegg [Tue, 28 Feb 2017 02:49:42 +0000 (21:49 -0500)]
10764: Permission tests. Support PDH permission check.

7 years ago10764: Add unit tests
Tom Clegg [Mon, 27 Feb 2017 22:51:51 +0000 (17:51 -0500)]
10764: Add unit tests

7 years ago10979: Add missing SqueueChecker initialization.
Tom Clegg [Mon, 27 Feb 2017 16:54:40 +0000 (11:54 -0500)]
10979: Add missing SqueueChecker initialization.

refs #10979

7 years agoMerge branch '11002-arvput-crash-fix'
Lucas Di Pentima [Mon, 27 Feb 2017 13:37:07 +0000 (10:37 -0300)]
Merge branch '11002-arvput-crash-fix'
Closes #11002

7 years ago11002: Merge branch 'master' into 11002-arvput-crash-fix
Lucas Di Pentima [Mon, 27 Feb 2017 12:42:49 +0000 (09:42 -0300)]
11002: Merge branch 'master' into 11002-arvput-crash-fix

7 years ago11002: Added note explaining why we're expecting a SystemExit to catch a SIGINT ...
Lucas Di Pentima [Mon, 27 Feb 2017 12:41:17 +0000 (09:41 -0300)]
11002: Added note explaining why we're expecting a SystemExit to catch a SIGINT (KeyboardInterrupt)

7 years agorun-tests.sh exit non-zero if gofmt fails
Tom Clegg [Fri, 24 Feb 2017 22:46:38 +0000 (17:46 -0500)]
run-tests.sh exit non-zero if gofmt fails

No issue #

7 years ago10979: Check for orphans only once at startup. Add missing Lock() in
Tom Clegg [Fri, 24 Feb 2017 21:46:46 +0000 (16:46 -0500)]
10979: Check for orphans only once at startup. Add missing Lock() in
squeue checker. Avoid holding mtx while waiting for API response.
Ensure RunContainer actually gets called in test case.

refs #10979

7 years agobuild improvement: really include apps/workbench_functionals when
Ward Vandewege [Fri, 24 Feb 2017 20:56:32 +0000 (15:56 -0500)]
build improvement: really include apps/workbench_functionals when
apps/workbench is specified.

No issue #

7 years agoMerge branch '6347-log-timestamps'
Tom Clegg [Fri, 24 Feb 2017 19:43:33 +0000 (14:43 -0500)]
Merge branch '6347-log-timestamps'

closes #6347

7 years agoMerge branch '10629-fuse-listing-perf' closes #10629
Peter Amstutz [Fri, 24 Feb 2017 19:24:18 +0000 (14:24 -0500)]
Merge branch '10629-fuse-listing-perf' closes #10629

7 years agoMerge branch '9277-container-output' closes #9277
Peter Amstutz [Fri, 24 Feb 2017 19:21:52 +0000 (14:21 -0500)]
Merge branch '9277-container-output' closes #9277

7 years ago9277: Add test that setting trashed, unreable collection is disallowed.
Peter Amstutz [Fri, 24 Feb 2017 19:20:15 +0000 (14:20 -0500)]
9277: Add test that setting trashed, unreable collection is disallowed.

7 years ago6347: Use RFC3339Nano to render timestamps loaded from serialized fields.
Tom Clegg [Fri, 24 Feb 2017 18:45:12 +0000 (13:45 -0500)]
6347: Use RFC3339Nano to render timestamps loaded from serialized fields.

Psych (YAML) serializes timestamps as ISO8601-with-space-separators,
and safe_load deserializes them to Time even with
whitelist_classes=[].

Psych.dump(Time.now.utc)
=> "--- 2017-02-22 21:33:22.845133778 Z\n...\n"

Psych.safe_load('2017-02-22 21:33:22.845133778 Z').class
=> Time

Psych.safe_load('2017-02-31 21:33:22.845133778 Z').class
=> String

Before:

Psych.safe_load('2017-02-22 21:33:22.845133778 Z').as_json
=> "2017-02-22T21:33:22Z"

After:

Psych.safe_load('2017-02-22 21:33:22.845133778 Z').as_json
=> "2017-02-22T21:33:22.845133778Z"

7 years agorefs #10979
radhika [Fri, 24 Feb 2017 18:17:15 +0000 (13:17 -0500)]
refs #10979
Merge branch '10979-cancelled-job-nodes'

7 years ago10979: fix failing test
radhika [Fri, 24 Feb 2017 18:16:31 +0000 (13:16 -0500)]
10979: fix failing test

7 years ago9277: arvados-cwl-runner sets "is_trashed" when directly setting output of container.
Peter Amstutz [Fri, 24 Feb 2017 16:01:07 +0000 (11:01 -0500)]
9277: arvados-cwl-runner sets "is_trashed" when directly setting output of container.

7 years ago9277: Container output check must be unscoped to include trashed collections.
Peter Amstutz [Fri, 24 Feb 2017 15:53:28 +0000 (10:53 -0500)]
9277: Container output check must be unscoped to include trashed collections.

7 years agocloses #10979
radhika [Fri, 24 Feb 2017 05:54:47 +0000 (00:54 -0500)]
closes #10979
Merge branch '10979-cancelled-job-nodes'

7 years agoMerge branch 'master' into 10979-cancelled-job-nodes
radhika [Fri, 24 Feb 2017 05:54:24 +0000 (00:54 -0500)]
Merge branch 'master' into 10979-cancelled-job-nodes

7 years ago10979: ruby way of doing it
radhika [Fri, 24 Feb 2017 05:52:18 +0000 (00:52 -0500)]
10979: ruby way of doing it

7 years ago11002: Don't save the state and log the stack trace before quitting upon catching...
Lucas Di Pentima [Thu, 23 Feb 2017 20:56:24 +0000 (17:56 -0300)]
11002: Don't save the state and log the stack trace before quitting upon catching an exception. Also, when receiving SIGINT (KeyboardInterrupt), just quit without any logging.
Updated tests to reflect this new behaviour.

7 years ago11156: Fix infinite loop condition.
Tom Clegg [Thu, 23 Feb 2017 19:04:42 +0000 (14:04 -0500)]
11156: Fix infinite loop condition.

closes #11156

7 years agoMerge branch '7995-keep-balance-docs'
Tom Clegg [Wed, 22 Feb 2017 21:36:50 +0000 (16:36 -0500)]
Merge branch '7995-keep-balance-docs'

closes #7995

7 years agoRemove pidfiles after shutting down test servers.
Tom Clegg [Wed, 22 Feb 2017 21:33:35 +0000 (16:33 -0500)]
Remove pidfiles after shutting down test servers.

No issue #

7 years ago10629: Don't flush dirhandles.
Peter Amstutz [Wed, 22 Feb 2017 21:28:22 +0000 (16:28 -0500)]
10629: Don't flush dirhandles.

7 years ago10629: Make tracking and dirtying of _committed flag efficient.
Peter Amstutz [Wed, 22 Feb 2017 21:08:41 +0000 (16:08 -0500)]
10629: Make tracking and dirtying of _committed flag efficient.

7 years ago10629: improve debug logging
Peter Amstutz [Wed, 22 Feb 2017 20:40:22 +0000 (20:40 +0000)]
10629: improve debug logging

--debug includes Keep logging.
--logfile includes timestamps.

7 years agoMerge branch 'master' into 10979-cancelled-job-nodes
radhika [Wed, 22 Feb 2017 19:49:37 +0000 (14:49 -0500)]
Merge branch 'master' into 10979-cancelled-job-nodes

7 years ago10979: refactor squeue invocations
radhika [Wed, 22 Feb 2017 19:48:47 +0000 (14:48 -0500)]
10979: refactor squeue invocations

7 years ago11002: Track this specific error with its own exception class, for future-proofing.
Lucas Di Pentima [Wed, 22 Feb 2017 18:26:16 +0000 (15:26 -0300)]
11002: Track this specific error with its own exception class, for future-proofing.

7 years agoMerge branch '11097-reuse-impure'
Tom Clegg [Wed, 22 Feb 2017 18:19:09 +0000 (13:19 -0500)]
Merge branch '11097-reuse-impure'

closes #11097

7 years ago7995: Add note about one keep-balance process at a time.
Tom Clegg [Wed, 22 Feb 2017 16:45:35 +0000 (11:45 -0500)]
7995: Add note about one keep-balance process at a time.

7 years ago7995: Fix up dry-run instructions.
Tom Clegg [Wed, 22 Feb 2017 16:29:48 +0000 (11:29 -0500)]
7995: Fix up dry-run instructions.

7 years ago7995: Fix up inconsistent "e.g." vs. "e.g.,".
Tom Clegg [Wed, 22 Feb 2017 16:24:13 +0000 (11:24 -0500)]
7995: Fix up inconsistent "e.g." vs. "e.g.,".

7 years ago7995: Copy edits.
Tom Clegg [Wed, 22 Feb 2017 16:02:32 +0000 (11:02 -0500)]
7995: Copy edits.

7 years agoFix dispatch panic when processing an update after tracker has been closed/deleted.
Tom Clegg [Wed, 22 Feb 2017 15:29:31 +0000 (10:29 -0500)]
Fix dispatch panic when processing an update after tracker has been closed/deleted.

refs #11151

7 years agoA few copy edits
Tom Morris [Tue, 21 Feb 2017 21:35:00 +0000 (16:35 -0500)]
A few copy edits

7 years agoDocument epydoc dependency
Tom Morris [Tue, 21 Feb 2017 21:34:07 +0000 (16:34 -0500)]
Document epydoc dependency

7 years ago11097: Clarify reuse query.
Tom Clegg [Tue, 21 Feb 2017 20:27:54 +0000 (15:27 -0500)]
11097: Clarify reuse query.

7 years ago11097: Update docs to reflect new container reuse behavior.
Tom Clegg [Tue, 21 Feb 2017 20:27:37 +0000 (15:27 -0500)]
11097: Update docs to reflect new container reuse behavior.

7 years ago10979: scancel orphaned job nodes in crunch1.
radhika [Tue, 21 Feb 2017 18:29:58 +0000 (13:29 -0500)]
10979: scancel orphaned job nodes in crunch1.

7 years ago11002: Do not try to save internal state when receiving a KeyboardInterrupt exception.
Lucas Di Pentima [Tue, 21 Feb 2017 17:41:58 +0000 (14:41 -0300)]
11002: Do not try to save internal state when receiving a KeyboardInterrupt exception.
Updated test accordingly.

7 years ago11002: Merge branch 'master' into 11002-arvput-crash-fix
Lucas Di Pentima [Tue, 21 Feb 2017 16:22:07 +0000 (13:22 -0300)]
11002: Merge branch 'master' into 11002-arvput-crash-fix

7 years ago11002: When trying to save the cache's state before quitting, if an exception
Lucas Di Pentima [Tue, 21 Feb 2017 13:00:19 +0000 (10:00 -0300)]
11002: When trying to save the cache's state before quitting, if an exception
is caught because of a BlockManager problem induced by an interruption,
print a warning message and quit without saving the last checkpoint.

7 years ago11002: Added missing assertion to test.
Lucas Di Pentima [Tue, 21 Feb 2017 12:54:07 +0000 (09:54 -0300)]
11002: Added missing assertion to test.

7 years ago11002: Improved test mocking a more suitable method and catching the specific excepti...
Lucas Di Pentima [Tue, 21 Feb 2017 11:41:07 +0000 (08:41 -0300)]
11002: Improved test mocking a more suitable method and catching the specific exception type.

7 years ago7995: Add "dry run" note.
Tom Clegg [Mon, 20 Feb 2017 22:03:10 +0000 (17:03 -0500)]
7995: Add "dry run" note.

7 years agoMerge branch '6520-pending-reason' refs #6520
Peter Amstutz [Mon, 20 Feb 2017 21:18:44 +0000 (16:18 -0500)]
Merge branch '6520-pending-reason' refs #6520

7 years ago6520: Add ReqNodeNotAvail to list of reasons (along with "Resources") to boot a new...
Peter Amstutz [Mon, 20 Feb 2017 18:54:47 +0000 (13:54 -0500)]
6520: Add ReqNodeNotAvail to list of reasons (along with "Resources") to boot a new node.

7 years ago11097: Update test to match new behavior.
Tom Clegg [Mon, 20 Feb 2017 20:45:40 +0000 (15:45 -0500)]
11097: Update test to match new behavior.

7 years ago11097: Merge branch 'master' into 11097-reuse-impure
Tom Clegg [Mon, 20 Feb 2017 20:34:15 +0000 (15:34 -0500)]
11097: Merge branch 'master' into 11097-reuse-impure

7 years ago11002: Added test to make the bug happen.
Lucas Di Pentima [Mon, 20 Feb 2017 20:25:05 +0000 (17:25 -0300)]
11002: Added test to make the bug happen.

7 years ago7995: Add keep-balance to install guide.
Tom Clegg [Mon, 20 Feb 2017 15:41:52 +0000 (10:41 -0500)]
7995: Add keep-balance to install guide.

7 years agoMerge branch '11127-delete-trash-with-links'
Tom Clegg [Fri, 17 Feb 2017 22:32:53 +0000 (17:32 -0500)]
Merge branch '11127-delete-trash-with-links'

refs #11127

7 years agoMerge branch '11121-crunch-output-collection-owner'
Lucas Di Pentima [Fri, 17 Feb 2017 22:18:54 +0000 (19:18 -0300)]
Merge branch '11121-crunch-output-collection-owner'
Closes #11121

7 years ago11121: Merge branch 'master' into 11121-crunch-output-collection-owner
Lucas Di Pentima [Fri, 17 Feb 2017 22:17:40 +0000 (19:17 -0300)]
11121: Merge branch 'master' into 11121-crunch-output-collection-owner

7 years ago11127: Delete dependent links too when emptying trash.
Tom Clegg [Fri, 17 Feb 2017 21:17:26 +0000 (16:17 -0500)]
11127: Delete dependent links too when emptying trash.

7 years ago10979: TrackContainer implementation and test
radhika [Fri, 17 Feb 2017 20:15:14 +0000 (15:15 -0500)]
10979: TrackContainer implementation and test

7 years ago11127: Do not crash server if SweepTrashedCollections thread has an exception.
Tom Clegg [Fri, 17 Feb 2017 19:49:13 +0000 (14:49 -0500)]
11127: Do not crash server if SweepTrashedCollections thread has an exception.

7 years agoMerge branch 'master' into 10979-cancelled-job-nodes
radhika [Thu, 16 Feb 2017 23:22:07 +0000 (18:22 -0500)]
Merge branch 'master' into 10979-cancelled-job-nodes

7 years agoMerge branch '6520-nodemanager-crunchv2' refs #6520
Peter Amstutz [Thu, 16 Feb 2017 21:44:22 +0000 (16:44 -0500)]
Merge branch '6520-nodemanager-crunchv2' refs #6520

7 years ago6520: Crunch v1 recognizes 'mix' and 'drng' as busy states for
Peter Amstutz [Thu, 16 Feb 2017 21:43:06 +0000 (16:43 -0500)]
6520: Crunch v1 recognizes 'mix' and 'drng' as busy states for
consistency (won't affect node manager behavior going forward, however.)

7 years agoMerge branch '6520-skip-compute0' refs #6520
Peter Amstutz [Thu, 16 Feb 2017 21:34:00 +0000 (16:34 -0500)]
Merge branch '6520-skip-compute0' refs #6520

7 years ago6520: Expand recognized 'busy' states.
Peter Amstutz [Thu, 16 Feb 2017 21:31:43 +0000 (16:31 -0500)]
6520: Expand recognized 'busy' states.

7 years ago11121: Added test.
Lucas Di Pentima [Thu, 16 Feb 2017 21:11:09 +0000 (18:11 -0300)]
11121: Added test.

7 years agoMerge branch 'master' into 10979-cancelled-job-nodes
radhika [Thu, 16 Feb 2017 21:07:02 +0000 (16:07 -0500)]
Merge branch 'master' into 10979-cancelled-job-nodes

7 years agoMerge branch '9277-trash-container-outputs' closes #9277
Peter Amstutz [Thu, 16 Feb 2017 20:13:40 +0000 (15:13 -0500)]
Merge branch '9277-trash-container-outputs' closes #9277

7 years ago6520: Explicitly mark node as 'down' before shutting down.
Peter Amstutz [Thu, 16 Feb 2017 20:10:34 +0000 (15:10 -0500)]
6520: Explicitly mark node as 'down' before shutting down.

7 years agocloses #11104
radhika [Thu, 16 Feb 2017 20:04:39 +0000 (15:04 -0500)]
closes #11104
Merge branch '11104-work-unit-expand-on-caret'

7 years ago11104: add fa-caret-down class to hyperlink rather than adding a span with icon.
radhika [Thu, 16 Feb 2017 19:08:40 +0000 (14:08 -0500)]
11104: add fa-caret-down class to hyperlink rather than adding a span with icon.

7 years ago9277: Include trashed collections when creating copy collection for container
Peter Amstutz [Thu, 16 Feb 2017 17:51:38 +0000 (12:51 -0500)]
9277: Include trashed collections when creating copy collection for container
request.

7 years agocloses #11103
radhika [Thu, 16 Feb 2017 17:49:09 +0000 (12:49 -0500)]
closes #11103
Merge branch '11103-fts-crs'

7 years ago11103: ensure the index exists before attempting to drop it.
radhika [Thu, 16 Feb 2017 17:48:11 +0000 (12:48 -0500)]
11103: ensure the index exists before attempting to drop it.

7 years ago9277: crunch-run Log and output collections set is_trashed to true instead of
Peter Amstutz [Thu, 16 Feb 2017 17:00:30 +0000 (12:00 -0500)]
9277: crunch-run Log and output collections set is_trashed to true instead of
setting trash_at.

7 years ago11103: Omit mounts from container_requests and definition from workflows for full...
radhika [Thu, 16 Feb 2017 17:27:51 +0000 (12:27 -0500)]
11103: Omit mounts from container_requests and definition from workflows for full text search index.

7 years ago11121: Add owner_uuid to output collection on crunch-job
Lucas Di Pentima [Thu, 16 Feb 2017 17:00:59 +0000 (14:00 -0300)]
11121: Add owner_uuid to output collection on crunch-job

7 years agoMerge branch '11017-docker-migration'
Tom Clegg [Thu, 16 Feb 2017 15:17:34 +0000 (10:17 -0500)]
Merge branch '11017-docker-migration'

closes #11017

7 years agoMerge branch '10797-ruby-2.3'
Tom Clegg [Wed, 15 Feb 2017 22:41:21 +0000 (17:41 -0500)]
Merge branch '10797-ruby-2.3'

closes #10797

7 years ago10797: Merge branch 'master' into 10797-ruby-2.3
Tom Clegg [Wed, 15 Feb 2017 22:40:40 +0000 (17:40 -0500)]
10797: Merge branch 'master' into 10797-ruby-2.3

7 years ago10797: Ruby 2.3.3
Tom Clegg [Wed, 15 Feb 2017 22:40:16 +0000 (17:40 -0500)]
10797: Ruby 2.3.3

7 years agoUpdate golang version to 1.7.5. Also make the equivalent changes to the
Ward Vandewege [Wed, 15 Feb 2017 22:35:40 +0000 (17:35 -0500)]
Update golang version to 1.7.5. Also make the equivalent changes to the
(new) ubuntu1604 build scripts.

refs #10797

7 years agoMerge branch 'master' into 10797-ruby-2.3
Ward Vandewege [Wed, 15 Feb 2017 22:25:24 +0000 (17:25 -0500)]
Merge branch 'master' into 10797-ruby-2.3

refs #10797

7 years ago6520: Don't ever assign compute0, start at slot 1 when assigning hostname.
Peter Amstutz [Wed, 15 Feb 2017 20:15:09 +0000 (15:15 -0500)]
6520: Don't ever assign compute0, start at slot 1 when assigning hostname.

7 years ago6520: Check that hostname is set before setting weight.
Peter Amstutz [Wed, 15 Feb 2017 20:08:39 +0000 (15:08 -0500)]
6520: Check that hostname is set before setting weight.

7 years ago6520: Apply node size fixup in CloudNodeListMonitorActor
Peter Amstutz [Wed, 15 Feb 2017 19:56:16 +0000 (14:56 -0500)]
6520: Apply node size fixup in CloudNodeListMonitorActor

7 years ago11097: Drop check for conflicting outputs when reusing completed containers.
Tom Clegg [Wed, 15 Feb 2017 19:47:57 +0000 (14:47 -0500)]
11097: Drop check for conflicting outputs when reusing completed containers.

7 years ago6520: Update cloud_node.size to match server calculator configuration.
Peter Amstutz [Wed, 15 Feb 2017 17:28:36 +0000 (12:28 -0500)]
6520: Update cloud_node.size to match server calculator configuration.

7 years ago11017: Add arv-migrate-docker19 to package.
Tom Clegg [Wed, 15 Feb 2017 16:30:29 +0000 (11:30 -0500)]
11017: Add arv-migrate-docker19 to package.

7 years ago6520: Assign weight to node based on price to preferentially schedule on
Peter Amstutz [Wed, 15 Feb 2017 16:29:25 +0000 (11:29 -0500)]
6520: Assign weight to node based on price to preferentially schedule on
cheaper nodes.

7 years ago10979: trackContainers func in crunch-dispatch-slurm.go
radhika [Tue, 14 Feb 2017 21:26:39 +0000 (16:26 -0500)]
10979: trackContainers func in crunch-dispatch-slurm.go

7 years ago6520: Add testcase using sinfo to set crunch_worker_state
Peter Amstutz [Tue, 14 Feb 2017 21:22:09 +0000 (16:22 -0500)]
6520: Add testcase using sinfo to set crunch_worker_state

7 years ago6520: Add testcase getting wishlist from squeue.
Peter Amstutz [Tue, 14 Feb 2017 21:01:21 +0000 (16:01 -0500)]
6520: Add testcase getting wishlist from squeue.

7 years ago6520: Call squeue to include slurm jobs in wishlist. Call sinfo to update
Peter Amstutz [Mon, 6 Feb 2017 19:00:53 +0000 (14:00 -0500)]
6520: Call squeue to include slurm jobs in wishlist.  Call sinfo to update
crunch_worker_state in arvados node records.

7 years ago11017: When compute nodes use image format v2, prefer migrated docker images.
Tom Clegg [Tue, 14 Feb 2017 20:16:22 +0000 (15:16 -0500)]
11017: When compute nodes use image format v2, prefer migrated docker images.

7 years agoMerge branch 'arvbox-docker-imgfmt' no issue #
Peter Amstutz [Tue, 14 Feb 2017 20:02:49 +0000 (15:02 -0500)]
Merge branch 'arvbox-docker-imgfmt' no issue #

7 years agoUpdate test_with_arvbox script.
Peter Amstutz [Tue, 14 Feb 2017 20:02:26 +0000 (15:02 -0500)]
Update test_with_arvbox script.

7 years agoSet docker_image_formats to v1.
Peter Amstutz [Tue, 14 Feb 2017 19:11:20 +0000 (14:11 -0500)]
Set docker_image_formats to v1.

7 years agoMerge branch 'fix-arvbox' no issue #
Peter Amstutz [Tue, 14 Feb 2017 17:09:13 +0000 (12:09 -0500)]
Merge branch 'fix-arvbox' no issue #