Peter Amstutz [Fri, 15 Jun 2018 02:38:57 +0000 (22:38 -0400)]
13627: a-c-r migration to cwltool loading/runtimeContext API
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 14 Jun 2018 21:56:13 +0000 (17:56 -0400)]
API fixup
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Fuad Muhic [Mon, 18 Jun 2018 14:47:23 +0000 (16:47 +0200)]
Merge branch '13429-cwl-runner-storage-classes-support'
closes #13429
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Fuad Muhic [Mon, 18 Jun 2018 09:03:33 +0000 (11:03 +0200)]
Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Fuad Muhic [Mon, 18 Jun 2018 08:58:33 +0000 (10:58 +0200)]
Add storage_classes tests for arvados_cwl_runner
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Lucas Di Pentima [Fri, 15 Jun 2018 19:58:29 +0000 (16:58 -0300)]
Merge branch '13581-cr-not-satisfiable-report'
Closes #13581
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 14 Jun 2018 17:16:37 +0000 (13:16 -0400)]
Merge branch '13407-volume-replication'
refs #13407
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Thu, 14 Jun 2018 17:09:34 +0000 (14:09 -0300)]
Merge branch '13617-anm-instance-type-logging'
Closes #13617
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 14 Jun 2018 15:39:08 +0000 (12:39 -0300)]
13617: Replaces all logging about size names with size ids.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 14 Jun 2018 15:04:22 +0000 (12:04 -0300)]
13581: Adds log message when no instance types are configured.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 13 Jun 2018 20:04:15 +0000 (17:04 -0300)]
Merge branch '7478-anm-libcloud-deps-fix'
Refs #7478
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 13 Jun 2018 19:29:17 +0000 (16:29 -0300)]
7478: Fix nodemanager's libcloud install dependency.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 13 Jun 2018 13:04:07 +0000 (10:04 -0300)]
Merge branch '7478-invalid-size-not-defined'
Refs #7478
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 13 Jun 2018 01:35:21 +0000 (22:35 -0300)]
7478: Fixes InvalidCloudSize creation. Adds wishlist related node info to logs.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 12 Jun 2018 18:24:41 +0000 (15:24 -0300)]
Merge branch '7478-anm-spot-instances'
Closes #7478
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 12 Jun 2018 17:44:24 +0000 (14:44 -0300)]
7478: Moves invalid cloud size node's shutdown decision to proper method.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Thu, 7 Jun 2018 17:11:00 +0000 (14:11 -0300)]
13581: Available types reported to the user when CR is not satisfiable.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Tue, 12 Jun 2018 14:46:01 +0000 (10:46 -0400)]
13407: Test stats reporting for volumes with replication>1.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Fuad Muhic [Tue, 12 Jun 2018 13:10:23 +0000 (15:10 +0200)]
Update storage_classes support for arvados_cwl_runner to work correctly
when arvados_cwl_runner is run in submit mode.
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Lucas Di Pentima [Mon, 11 Jun 2018 22:49:21 +0000 (19:49 -0300)]
7478: Fixes GCE driver's arvados_node_size tag handling.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Mon, 11 Jun 2018 21:01:38 +0000 (17:01 -0400)]
13407: Fix stats reporting for mounts with replication>1.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Mon, 11 Jun 2018 20:51:38 +0000 (17:51 -0300)]
7478: Adds test to check that state is 'down' with 'invalid' size.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Mon, 11 Jun 2018 19:47:50 +0000 (16:47 -0300)]
7478: Adds tests on driver's create_node calls.
* Check that arvados_node_size tag is passed
* On EC2 driver, check that ex_spot_market=True is used when requesting
spot instances.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Mon, 11 Jun 2018 19:36:21 +0000 (15:36 -0400)]
13407: Take keepstore-reported volume replication into account.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Mon, 11 Jun 2018 19:32:22 +0000 (15:32 -0400)]
Merge branch '13562-group-properties' closes #13562
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Mon, 11 Jun 2018 19:31:56 +0000 (15:31 -0400)]
Merge branch 'master' into 13562-group-properties
Peter Amstutz [Mon, 11 Jun 2018 19:30:05 +0000 (15:30 -0400)]
Merge branch '12960-migrate-properties' closes #12960
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Mon, 11 Jun 2018 16:59:31 +0000 (13:59 -0300)]
7478: Config load fix & test updates about instance type decoupling
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Mon, 11 Jun 2018 15:34:22 +0000 (12:34 -0300)]
Merge branch 'master' into 7478-anm-spot-instances
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Mon, 11 Jun 2018 15:10:44 +0000 (12:10 -0300)]
7478: Assign invalid cloud sizes to nodes with stale arvados_node_size tags
When retrieving the node's list from the cloud provider, if the tag
refers to a non existant arvados cloud size, assign an invalid size
and return 'down' state when asked so that it can be properly shut down.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Fuad Muhic [Mon, 11 Jun 2018 13:50:27 +0000 (15:50 +0200)]
Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Tom Clegg [Fri, 8 Jun 2018 19:05:12 +0000 (15:05 -0400)]
13594: Remove table lock.
refs #13594
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Fri, 8 Jun 2018 18:59:41 +0000 (14:59 -0400)]
12960: Fix test due to slightly different serialization from jsonb
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 8 Jun 2018 18:44:26 +0000 (14:44 -0400)]
13562: Migration needs to update full text search index on groups
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 8 Jun 2018 14:13:03 +0000 (10:13 -0400)]
12960: Update documentation.
Also allow "properties" to be updated on committed container
requests (bugfix implementation to match documented behavior).
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 8 Jun 2018 13:15:34 +0000 (09:15 -0400)]
13562: Update documentation
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Fri, 8 Jun 2018 12:59:56 +0000 (08:59 -0400)]
13562: Add "properties" field to groups, supports subproperty query.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Fuad Muhic [Fri, 8 Jun 2018 09:05:22 +0000 (11:05 +0200)]
Change multiple storage classes error handling in cwl_runner
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Peter Amstutz [Thu, 7 Jun 2018 20:58:57 +0000 (16:58 -0400)]
Merge branch '11907-acr-stable-pdh' refs #11907
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Lucas Di Pentima [Thu, 7 Jun 2018 19:52:42 +0000 (16:52 -0300)]
Merge branch '13480-anm-tagless-azure-node'
Closes #13480
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Peter Amstutz [Thu, 7 Jun 2018 19:26:36 +0000 (15:26 -0400)]
11907: Fix timezone handling for HTTP timestamps.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Thu, 7 Jun 2018 19:24:04 +0000 (15:24 -0400)]
Merge branch '13427-multiple-mounts'
refs #13427
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Thu, 7 Jun 2018 18:33:03 +0000 (14:33 -0400)]
12960: Migrate properties fields to jsonb to support subproperty filters.
Specifically: links.properties and container_requests.properties,
nodes.properties, and nodes.info.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 7 Jun 2018 17:13:53 +0000 (13:13 -0400)]
11907: Fix tests
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Fuad Muhic [Thu, 7 Jun 2018 15:20:43 +0000 (17:20 +0200)]
Merge branch '13330-collection-save'
refs #13330
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Peter Amstutz [Thu, 7 Jun 2018 13:54:47 +0000 (09:54 -0400)]
11907: Make collections containing dependencies have predictable PDHs
Removes optimization that tries to avoid redundant file
uploads (decided complexity tradeoff isn't worth it.) Collections
created from local file inputs longer use block packing.
Likely to invalidate job reuse for jobs submitted by past versions of
a-c-r, but will have more stable job reuse going forward.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 7 Jun 2018 14:38:41 +0000 (10:38 -0400)]
Merge branch '11907-uploadfile-no-pack' refs #11907
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Peter Amstutz [Thu, 7 Jun 2018 13:44:16 +0000 (09:44 -0400)]
11907: Add option to uploadfiles to disable packing.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Fuad Muhic [Thu, 7 Jun 2018 09:39:46 +0000 (11:39 +0200)]
Fix ciso8601 issue in collections tests.
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Tom Clegg [Wed, 6 Jun 2018 21:01:53 +0000 (17:01 -0400)]
13427: Use same index result N times if a device is mounted N times.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 6 Jun 2018 19:25:45 +0000 (15:25 -0400)]
13427: More symlink hack
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 6 Jun 2018 19:19:35 +0000 (15:19 -0400)]
13427: Fix replication stats reporting for multiple-mounted devices.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Wed, 6 Jun 2018 18:37:04 +0000 (15:37 -0300)]
13480: Ignore nodes with no tags instead of crashing.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 6 Jun 2018 18:36:07 +0000 (15:36 -0300)]
13480: Adds test making the bug show itself.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Wed, 6 Jun 2018 17:14:26 +0000 (13:14 -0400)]
13427: Handle same backend device mounted RW in multiple places.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 5 Jun 2018 02:10:37 +0000 (22:10 -0400)]
13427: Ignore readonly devices mounted read-write elsewhere.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 5 Jun 2018 19:56:57 +0000 (15:56 -0400)]
Merge branch '13456-phantomjs-died'
refs #13456
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Tue, 5 Jun 2018 19:11:38 +0000 (15:11 -0400)]
More gopath symlink hacks.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Tue, 5 Jun 2018 19:10:11 +0000 (16:10 -0300)]
Merge branch '13431-keepproxy-storage-classes-headers'
Closes #13431
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Tue, 5 Jun 2018 19:08:42 +0000 (15:08 -0400)]
13456: Restart driver after each test.
Shot in the dark from https://github.com/teampoltergeist/poltergeist/issues/232
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Tue, 5 Jun 2018 18:43:40 +0000 (15:43 -0300)]
13431: Removes commented code
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 5 Jun 2018 18:37:05 +0000 (15:37 -0300)]
13431: Fixes unrelated test
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Tue, 5 Jun 2018 18:22:18 +0000 (15:22 -0300)]
13431: Adds test confirming that keepproxy propagates X-Keep-Storage-Classes
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Tue, 5 Jun 2018 16:57:23 +0000 (12:57 -0400)]
Merge branch '13164-cr-locking'
refs #13164
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Fuad Muhic [Tue, 5 Jun 2018 15:30:05 +0000 (17:30 +0200)]
Add check for multiple storage classes in arvados-cwl-runner
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Tom Clegg [Tue, 5 Jun 2018 14:56:22 +0000 (10:56 -0400)]
13164: Remove locking. Clean up after races in background instead.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Tue, 5 Jun 2018 14:38:12 +0000 (11:38 -0300)]
7478: Fixes child CR default preemptable setting logic.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Fuad Muhic [Tue, 5 Jun 2018 12:03:38 +0000 (14:03 +0200)]
Add getters for properties and trash_at attributes and small bugfix for
save and save_new methods in Collection class.
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Lucas Di Pentima [Tue, 5 Jun 2018 00:39:18 +0000 (21:39 -0300)]
13431: Make keepproxy propagate X-Keep-Storage-Classes headers
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Mon, 4 Jun 2018 20:24:57 +0000 (17:24 -0300)]
13431: Adds storage class support to Keepclient.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Mon, 4 Jun 2018 13:22:13 +0000 (09:22 -0400)]
Merge branch '12061-nm-integration-tests'
refs #12061
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Mon, 4 Jun 2018 13:21:26 +0000 (10:21 -0300)]
Merge branch 'master' into 7478-anm-spot-instances
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Mon, 4 Jun 2018 13:20:29 +0000 (10:20 -0300)]
Merge branch '13559-ciso8601-pin'
Closes #13559
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Fri, 1 Jun 2018 21:12:43 +0000 (18:12 -0300)]
13559: Pin ciso8601 to version <2.0.0 due to API changes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Fri, 1 Jun 2018 17:35:10 +0000 (13:35 -0400)]
12061: Disable nodemanager integration tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 1 Jun 2018 17:36:43 +0000 (13:36 -0400)]
12061: Remove unreliable check.
"Shutdown because state=down" is possible during a race, when
nodemanager logs "Shutdown success" and polls sinfo again before the
test suite has finished updating the sinfo stub.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 1 Jun 2018 16:50:02 +0000 (12:50 -0400)]
12061: Reduce per-test timeout.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Fri, 1 Jun 2018 16:49:42 +0000 (12:49 -0400)]
12061: Fix nodes marked busy while expecting shutdown decision.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Fuad Muhic [Fri, 1 Jun 2018 16:06:49 +0000 (18:06 +0200)]
Add storage classes supper to arvados cwl runner
Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>
Lucas Di Pentima [Fri, 1 Jun 2018 15:41:44 +0000 (12:41 -0300)]
7478: Fixes integration tests.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 21:17:33 +0000 (17:17 -0400)]
12061: Use shorter retry-after sleeps in integration test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 21:07:13 +0000 (17:07 -0400)]
12061: Don't wait around for test daemon to shutdown gracefully.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 21:06:32 +0000 (17:06 -0400)]
12061: Poll aggressively in integration tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 21:01:15 +0000 (17:01 -0400)]
12061: Allow sub-second poll times and sub-minute shutdown windows.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Thu, 31 May 2018 19:24:07 +0000 (16:24 -0300)]
Merge branch 'master' into 7478-anm-spot-instances
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 19:05:47 +0000 (15:05 -0400)]
Merge branch '13517-buffer-leak'
refs #13517
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 18:39:41 +0000 (14:39 -0400)]
Merge branch '12061-flaky-tests'
refs #12061
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 18:26:59 +0000 (14:26 -0400)]
12061: Explicitly join actor thread after stopping.
Fixes the "nonfatal" test case the same way the "fatal" test case was
fixed in
6706c6d46c6cfb080371b147b60fcaa191bd55a6.
Since the nonfatal case just tests that kill_mock did *not* get
called, the race condition used to make the test ineffectual
occasionally, instead of failing occasionally.
The race condition became visible only after the status reporting test
was added.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Thu, 31 May 2018 17:44:18 +0000 (13:44 -0400)]
13517: Fix leaked goroutine and buffer on failed S3 write.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Lucas Di Pentima [Thu, 31 May 2018 16:42:45 +0000 (13:42 -0300)]
7478: Unify arvados node size access across different drivers
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 30 May 2018 21:08:34 +0000 (18:08 -0300)]
7478: Test fixes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 30 May 2018 21:02:30 +0000 (18:02 -0300)]
7478: Set arvados-node-size-id tag on node creation.
This tag has the Arvados assigned node size id, not the cloud size.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 30 May 2018 21:00:29 +0000 (18:00 -0300)]
7478: Override CloudSizeWrapper id with config Size name.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Lucas Di Pentima [Wed, 30 May 2018 20:55:59 +0000 (17:55 -0300)]
7478: Ignore redundant instance_type field on CloudSizeWrapper
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>
Tom Clegg [Wed, 30 May 2018 20:32:20 +0000 (16:32 -0400)]
12061: Don't round down to 1s in timestamp parser.
ComputeNodeMonitorActor.offer_arvados_pair() pairs nodes only if
first_ping_time >= cloud_node_start_time
However, first_ping_time is passed through arvados_timestamp() before
this comparison, which was truncating the subsecond part -- so the
comparison was effectively
floor(first_ping_time) >= cloud_node_start_time
When FPT and CNST differed only in the subsecond part, this comparison
failed, and the nodes could never be paired. This caused sporadic
failures in tests, where the two values are often separated by less
than a second.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Tom Clegg [Wed, 30 May 2018 20:30:45 +0000 (16:30 -0400)]
12061: Change busywait approach to preserve assertion messages.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Wed, 30 May 2018 11:39:30 +0000 (07:39 -0400)]
Merge branch '11162-wes-support' refs #11162
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 29 May 2018 20:05:27 +0000 (16:05 -0400)]
Merge branch '13513-balance-deadlock'
fixes #13513
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
Peter Amstutz [Tue, 29 May 2018 17:16:55 +0000 (13:16 -0400)]
11162: Remove debugging check
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>
Tom Clegg [Tue, 29 May 2018 13:46:04 +0000 (09:46 -0400)]
13513: Fix deadlocked workers.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>
projects / arvados.git / log