arvados.git
5 years ago14484: Updates the migration to use the sdk for manifest parsing
Eric Biagiotti [Wed, 27 Mar 2019 17:27:06 +0000 (13:27 -0400)]
14484: Updates the migration to use the sdk for manifest parsing

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Adds file count and size callback to the collection model
Eric Biagiotti [Tue, 26 Mar 2019 21:02:56 +0000 (17:02 -0400)]
14484: Adds file count and size callback to the collection model

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Fixes grouping bug by simplifying nested pdhs arrays
Eric Biagiotti [Tue, 26 Mar 2019 18:50:18 +0000 (14:50 -0400)]
14484: Fixes grouping bug by simplifying nested pdhs arrays

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Changes file token to match the one used in collection.rb model
Eric Biagiotti [Tue, 26 Mar 2019 17:56:10 +0000 (13:56 -0400)]
14484: Changes file token to match the one used in collection.rb model

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Adds migration for collection file count and total size
Eric Biagiotti [Tue, 26 Mar 2019 16:47:06 +0000 (12:47 -0400)]
14484: Adds migration for collection file count and total size

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Fixes bug in pdh transaction grouping
Eric Biagiotti [Tue, 26 Mar 2019 16:46:15 +0000 (12:46 -0400)]
14484: Fixes bug in pdh transaction grouping

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Adds functionality and test for pdh grouping in the container model
Eric Biagiotti [Mon, 25 Mar 2019 22:30:40 +0000 (18:30 -0400)]
14484: Adds functionality and test for pdh grouping in the container model

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Updates the db structure after db:migrate
Eric Biagiotti [Mon, 25 Mar 2019 20:17:37 +0000 (16:17 -0400)]
14484: Updates the db structure after db:migrate

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14484: Adds migration file for collection file count and size
Eric Biagiotti [Fri, 22 Mar 2019 17:55:15 +0000 (13:55 -0400)]
14484: Adds migration file for collection file count and size

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '15007-azure-panic'
Tom Clegg [Thu, 21 Mar 2019 20:55:00 +0000 (16:55 -0400)]
Merge branch '15007-azure-panic'

fixes #15007

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago15007: Fix crash when Azure nic has IP config but no IP address.
Tom Clegg [Thu, 21 Mar 2019 20:54:49 +0000 (16:54 -0400)]
15007: Fix crash when Azure nic has IP config but no IP address.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14807-prod-blockers'
Tom Clegg [Thu, 21 Mar 2019 20:50:32 +0000 (16:50 -0400)]
Merge branch '14807-prod-blockers'

refs #14807

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14451-crunchstat-summary-cache-ram'
Tom Morris [Thu, 21 Mar 2019 20:02:20 +0000 (16:02 -0400)]
Merge branch '14451-crunchstat-summary-cache-ram'

Fix three small crunchstat-summary issues:
- Use default of 1 if no cores requested. fixes #12026
- Don't recommend RAM or CPU changes based on zero usage. fixes #10570
- Remove extra MB (2**20) factor in Keep cache recommendation.  Fixes #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoImprove wording with direction of recommendation change.
Tom Morris [Fri, 15 Mar 2019 05:11:59 +0000 (01:11 -0400)]
Improve wording with direction of recommendation change.

No issue.

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoUse default of 1 if no cores requested. refs #12026
Tom Morris [Fri, 15 Mar 2019 04:47:38 +0000 (00:47 -0400)]
Use default of 1 if no cores requested. refs #12026

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoDon't recommend RAM or CPU changes based on zero usage. refs #10570
Tom Morris [Fri, 15 Mar 2019 04:10:06 +0000 (00:10 -0400)]
Don't recommend RAM or CPU changes based on zero usage. refs #10570

Never recommend 0 MB RAM. If CPU usage is zero, don't make any
CPU recommendations.

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoFix Keep cache recommendation calculation.
Tom Morris [Fri, 15 Mar 2019 04:03:59 +0000 (00:03 -0400)]
Fix Keep cache recommendation calculation.

Remove extra MB (2**20) factor. Refs #14451

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoSpell MB consistently.
Tom Morris [Fri, 15 Mar 2019 02:11:45 +0000 (22:11 -0400)]
Spell MB consistently.

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoMerge branch '14645-fuse-operations-reporting'
Eric Biagiotti [Thu, 21 Mar 2019 19:52:26 +0000 (15:52 -0400)]
Merge branch '14645-fuse-operations-reporting'

refs #14645

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '14939-crunchstat-summary-python3'
Tom Morris [Thu, 21 Mar 2019 19:47:54 +0000 (15:47 -0400)]
Merge branch '14939-crunchstat-summary-python3'

Fixes #14939

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoResolve deprecation warnings
Tom Morris [Thu, 14 Mar 2019 22:47:20 +0000 (18:47 -0400)]
Resolve deprecation warnings

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoFix resource leak warned by Python 3
Tom Morris [Thu, 14 Mar 2019 22:30:52 +0000 (18:30 -0400)]
Fix resource leak warned by Python 3

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoAdd crunchstat-summary to Python 3 tests. refs #14939
Tom Morris [Wed, 13 Mar 2019 23:33:03 +0000 (19:33 -0400)]
Add crunchstat-summary to Python 3 tests. refs #14939

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoFix test Python 3 integer divide difference.
Tom Morris [Wed, 13 Mar 2019 23:26:00 +0000 (19:26 -0400)]
Fix test Python 3 integer divide difference.

Use float for both Python 2 & 3 like the rest of the metrics.
refs #14393

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years agoUpdate crunchstat-summary to Python 3. Refs #14939
Tom Morris [Fri, 8 Mar 2019 23:40:24 +0000 (18:40 -0500)]
Update crunchstat-summary to Python 3. Refs #14939

Arvados-DCO-1.1-Signed-off-by: Tom Morris <tfmorris@veritasgenetics.com>

5 years ago14807: Allow some fake nodes to remain unbroken.
Tom Clegg [Thu, 21 Mar 2019 18:12:54 +0000 (14:12 -0400)]
14807: Allow some fake nodes to remain unbroken.

If every fake node hits a fake failure mode in 200-400 ms, throughput
can fall below 20 containers per second and fail the test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge remote-tracking branch 'origin/master' into 14645-fuse-operations-reporting
Eric Biagiotti [Thu, 21 Mar 2019 18:03:16 +0000 (14:03 -0400)]
Merge remote-tracking branch 'origin/master' into 14645-fuse-operations-reporting

refs #14645

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14645: Updates crunchstat format for fuse ops
Eric Biagiotti [Thu, 21 Mar 2019 14:49:31 +0000 (10:49 -0400)]
14645: Updates crunchstat format for fuse ops

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoMerge branch '14873-google-api-client-update'
Lucas Di Pentima [Wed, 20 Mar 2019 20:17:07 +0000 (17:17 -0300)]
Merge branch '14873-google-api-client-update'
Refs #14873

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14807: Give up if initial keep services list isn't loaded in 60s.
Tom Clegg [Wed, 20 Mar 2019 05:50:10 +0000 (01:50 -0400)]
14807: Give up if initial keep services list isn't loaded in 60s.

Otherwise, MakeKeepClient blocks forever when ARVADOS_* env vars are
bad.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Report queue metrics.
Tom Clegg [Tue, 19 Mar 2019 21:02:05 +0000 (17:02 -0400)]
14807: Report queue metrics.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Add "kill instance" management API.
Tom Clegg [Tue, 19 Mar 2019 17:40:10 +0000 (13:40 -0400)]
14807: Add "kill instance" management API.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Configurable rate limit for cloud provider API calls.
Tom Clegg [Mon, 18 Mar 2019 21:27:40 +0000 (17:27 -0400)]
14807: Configurable rate limit for cloud provider API calls.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Drain instances that crunch-run reports broken.
Tom Clegg [Mon, 18 Mar 2019 20:32:58 +0000 (16:32 -0400)]
14807: Drain instances that crunch-run reports broken.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14807-escalate-sigterm'
Tom Clegg [Wed, 20 Mar 2019 19:24:02 +0000 (15:24 -0400)]
Merge branch '14807-escalate-sigterm'

refs #14807

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Merge branch 'master'
Tom Clegg [Wed, 20 Mar 2019 19:20:57 +0000 (15:20 -0400)]
14807: Merge branch 'master'

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Tweak log message.
Tom Clegg [Wed, 20 Mar 2019 19:20:38 +0000 (15:20 -0400)]
14807: Tweak log message.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14947-run-tests-interactive'
Tom Clegg [Wed, 20 Mar 2019 18:07:06 +0000 (14:07 -0400)]
Merge branch '14947-run-tests-interactive'

closes #14947

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Ignore --only / --skip args in interactive mode.
Tom Clegg [Wed, 20 Mar 2019 17:41:35 +0000 (13:41 -0400)]
14947: Ignore --only / --skip args in interactive mode.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14873: Updates arvados & arvados-cli gems dependency on google-api-client.
Lucas Di Pentima [Wed, 20 Mar 2019 17:39:03 +0000 (14:39 -0300)]
14873: Updates arvados & arvados-cli gems dependency on google-api-client.

Start using our own fork arvados-google-api-client that ask for activesupport
version <5.1 so that it doesn't block the rails 5 upgrade.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14947: Fix controller install in deps.
Tom Clegg [Wed, 20 Mar 2019 17:37:28 +0000 (13:37 -0400)]
14947: Fix controller install in deps.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Accept test suite options in interactive mode.
Tom Clegg [Wed, 20 Mar 2019 13:28:24 +0000 (09:28 -0400)]
14947: Accept test suite options in interactive mode.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14645: Adds op counts to reporting
Eric Biagiotti [Tue, 19 Mar 2019 20:27:00 +0000 (16:27 -0400)]
14645: Adds op counts to reporting

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14947: Install controller (cmd/arvados-server) for integration tests.
Tom Clegg [Tue, 19 Mar 2019 20:23:55 +0000 (16:23 -0400)]
14947: Install controller (cmd/arvados-server) for integration tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Don't send SIGKILL to crunch-run processes.
Tom Clegg [Tue, 19 Mar 2019 17:28:39 +0000 (13:28 -0400)]
14807: Don't send SIGKILL to crunch-run processes.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14645: Adds individual ops time output
Eric Biagiotti [Tue, 19 Mar 2019 17:26:26 +0000 (13:26 -0400)]
14645: Adds individual ops time output

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14807: Add comments to "error indicates success" code.
Tom Clegg [Tue, 19 Mar 2019 15:57:32 +0000 (11:57 -0400)]
14807: Add comments to "error indicates success" code.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14947: Suggest "install deps" as first interactive command.
Tom Clegg [Tue, 19 Mar 2019 14:30:36 +0000 (10:30 -0400)]
14947: Suggest "install deps" as first interactive command.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMove back arvbox version of nodejs to fix composer compile issue
Peter Amstutz [Mon, 18 Mar 2019 20:13:35 +0000 (16:13 -0400)]
Move back arvbox version of nodejs to fix composer compile issue

no issue #

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14807: Update tests to new worker/pool interface.
Tom Clegg [Mon, 18 Mar 2019 19:16:25 +0000 (15:16 -0400)]
14807: Update tests to new worker/pool interface.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Lengthen kill delays in stub driver to test term-kill-drain.
Tom Clegg [Mon, 18 Mar 2019 19:08:19 +0000 (15:08 -0400)]
14807: Lengthen kill delays in stub driver to test term-kill-drain.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14322-cwl-uuid-input' refs #14322
Peter Amstutz [Mon, 18 Mar 2019 18:43:31 +0000 (14:43 -0400)]
Merge branch '14322-cwl-uuid-input' refs #14322

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak JSON_KEEP_LOCATOR_REGEXP and comment
Peter Amstutz [Mon, 18 Mar 2019 18:04:19 +0000 (14:04 -0400)]
14322: Tweak JSON_KEEP_LOCATOR_REGEXP and comment

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14807: Escalate sigterm->sigkill->drain instance.
Tom Clegg [Mon, 18 Mar 2019 15:09:09 +0000 (11:09 -0400)]
14807: Escalate sigterm->sigkill->drain instance.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Fail test if a container gets run twice.
Tom Clegg [Mon, 18 Mar 2019 15:44:19 +0000 (11:44 -0400)]
14807: Fail test if a container gets run twice.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14807: Fix unreliable test.
Tom Clegg [Mon, 18 Mar 2019 15:44:07 +0000 (11:44 -0400)]
14807: Fix unreliable test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14977-kill-if-not-in-queue'
Tom Clegg [Mon, 18 Mar 2019 15:11:56 +0000 (11:11 -0400)]
Merge branch '14977-kill-if-not-in-queue'

fixes #14977

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Re-order statements for clarity.
Tom Clegg [Mon, 18 Mar 2019 13:42:27 +0000 (09:42 -0400)]
14977: Re-order statements for clarity.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Don't try fixStaleLocks until worker pool state is loaded.
Tom Clegg [Fri, 15 Mar 2019 20:19:51 +0000 (16:19 -0400)]
14977: Don't try fixStaleLocks until worker pool state is loaded.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Kill containers that don't exist according to controller.
Tom Clegg [Fri, 15 Mar 2019 20:11:48 +0000 (16:11 -0400)]
14977: Kill containers that don't exist according to controller.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14977: Drop containers when the controller says they don't exist.
Tom Clegg [Fri, 15 Mar 2019 19:21:39 +0000 (15:21 -0400)]
14977: Drop containers when the controller says they don't exist.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14322: Fix noisy unicode logging error in tests, add comments
Peter Amstutz [Fri, 15 Mar 2019 14:54:59 +0000 (10:54 -0400)]
14322: Fix noisy unicode logging error in tests, add comments

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Add missing test files
Peter Amstutz [Wed, 13 Mar 2019 21:30:57 +0000 (17:30 -0400)]
14322: Add missing test files

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Workbench links collection uuids in workflow inputs
Peter Amstutz [Wed, 13 Mar 2019 21:26:07 +0000 (17:26 -0400)]
14322: Workbench links collection uuids in workflow inputs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Update comment on bwa-mem-input-mixed.yml
Peter Amstutz [Wed, 13 Mar 2019 21:20:22 +0000 (17:20 -0400)]
14322: Update comment on bwa-mem-input-mixed.yml

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Add additional example input files
Peter Amstutz [Wed, 13 Mar 2019 20:40:33 +0000 (16:40 -0400)]
14322: Add additional example input files

* with uuids
* with both portable data hashes and uuids

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak workbench support from manual testing
Peter Amstutz [Wed, 13 Mar 2019 20:19:37 +0000 (16:19 -0400)]
14322: Tweak workbench support from manual testing

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Update workbench to use new collectionUUID
Peter Amstutz [Wed, 13 Mar 2019 19:52:20 +0000 (15:52 -0400)]
14322: Update workbench to use new collectionUUID

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Fix tests
Peter Amstutz [Wed, 13 Mar 2019 19:25:33 +0000 (15:25 -0400)]
14322: Fix tests

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tweak documentation to mention uuid keep URI form
Peter Amstutz [Wed, 13 Mar 2019 18:57:53 +0000 (14:57 -0400)]
14322: Tweak documentation to mention uuid keep URI form

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: CollectionFsAccess accepts UUIDs
Peter Amstutz [Wed, 13 Mar 2019 18:57:20 +0000 (14:57 -0400)]
14322: CollectionFsAccess accepts UUIDs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Tests for edge cases
Peter Amstutz [Tue, 12 Mar 2019 20:39:12 +0000 (16:39 -0400)]
14322: Tests for edge cases

Report unknown uuids and mismatches between current collection
PDH (from API server lookup) and location PDH.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Propagate collection UUID to container mount
Peter Amstutz [Mon, 11 Mar 2019 21:42:09 +0000 (17:42 -0400)]
14322: Propagate collection UUID to container mount

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14322: Convert uuids on input to portable data hashes
Peter Amstutz [Mon, 11 Mar 2019 21:02:14 +0000 (17:02 -0400)]
14322: Convert uuids on input to portable data hashes

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years agoFix a-c-r printing logs on error refs #14870
Peter Amstutz [Fri, 15 Mar 2019 14:10:15 +0000 (10:10 -0400)]
Fix a-c-r printing logs on error refs #14870

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14947: run-tests: add --interactive, skip deps on --skip-install.
Tom Clegg [Thu, 14 Mar 2019 20:24:18 +0000 (16:24 -0400)]
14947: run-tests: add --interactive, skip deps on --skip-install.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoMerge branch '14966-api-test-hang-fix'
Lucas Di Pentima [Thu, 14 Mar 2019 19:42:18 +0000 (16:42 -0300)]
Merge branch '14966-api-test-hang-fix'
Closes #14966

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoArvbox updates, no issue #
Peter Amstutz [Thu, 7 Feb 2019 22:07:08 +0000 (17:07 -0500)]
Arvbox updates, no issue #

* Bump Go version, add openjdk.

* Bump node version.  Make sure node is in path.

* Fix $HOME in setup scripts.

* Add "arvbox devenv" command which creates and enters a container
  environment suitable for interactively running development tools

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

5 years ago14966: Implements cleanup on test case teardown.
Lucas Di Pentima [Thu, 14 Mar 2019 18:56:00 +0000 (15:56 -0300)]
14966: Implements cleanup on test case teardown.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago14645: Adds prometheus_client dependency
Eric Biagiotti [Thu, 14 Mar 2019 13:26:29 +0000 (09:26 -0400)]
14645: Adds prometheus_client dependency

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14966: Fixes test.
Lucas Di Pentima [Wed, 13 Mar 2019 23:52:25 +0000 (20:52 -0300)]
14966: Fixes test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge remote-tracking branch 'origin/master' into 14645-fuse-operations-reporting
Eric Biagiotti [Wed, 13 Mar 2019 17:49:48 +0000 (13:49 -0400)]
Merge remote-tracking branch 'origin/master' into 14645-fuse-operations-reporting

refs #14645

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years ago14645: Adds FUSE ops time to crunchstat output
Eric Biagiotti [Wed, 13 Mar 2019 17:46:26 +0000 (13:46 -0400)]
14645: Adds FUSE ops time to crunchstat output

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>
Makes stat writing more generic. Now can accommodate different stat counts and types.

5 years ago14645: Adds FUSE ops timing using prometheus
Eric Biagiotti [Wed, 13 Mar 2019 17:43:03 +0000 (13:43 -0400)]
14645: Adds FUSE ops timing using prometheus

Arvados-DCO-1.1-Signed-off-by: Eric Biagiotti <ebiagiotti@veritasgenetics.com>

5 years agoSwitch to stretch-slim as the base image for arvados/jobs, this results in a
Ward Vandewege [Wed, 13 Mar 2019 15:42:30 +0000 (11:42 -0400)]
Switch to stretch-slim as the base image for arvados/jobs, this results in a
smaller image.

arvados/jobs         1.3.1.20190308210512   2a6c5e4993e7        4 days ago          284MB
arvados/jobs         latest                 0cfff29120dd        20 minutes ago      232MB

refs #14819

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '13593-async-perm-graph-update'
Lucas Di Pentima [Wed, 13 Mar 2019 15:09:14 +0000 (12:09 -0300)]
Merge branch '13593-async-perm-graph-update'
Closes #13593

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoMerge branch '13937-keepstore-prometheus'
Lucas Di Pentima [Tue, 12 Mar 2019 20:32:43 +0000 (17:32 -0300)]
Merge branch '13937-keepstore-prometheus'
Closes #13937

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Moves & enhances group creation with async=true test.
Lucas Di Pentima [Tue, 12 Mar 2019 18:58:57 +0000 (15:58 -0300)]
13593: Moves & enhances group creation with async=true test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13937: Removes the 'any' operation type for a better way of testing driver labels
Lucas Di Pentima [Tue, 12 Mar 2019 18:19:36 +0000 (15:19 -0300)]
13937: Removes the 'any' operation type for a better way of testing driver labels

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Cleans up old experimental code.
Lucas Di Pentima [Tue, 12 Mar 2019 15:04:49 +0000 (12:04 -0300)]
13593: Cleans up old experimental code.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Changes default async permission update interval. Clears cache.
Lucas Di Pentima [Tue, 12 Mar 2019 14:38:09 +0000 (11:38 -0300)]
13593: Changes default async permission update interval. Clears cache.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoSome build script housekeeping: fix the detection logic for the existence of
Ward Vandewege [Tue, 12 Mar 2019 14:32:05 +0000 (10:32 -0400)]
Some build script housekeeping: fix the detection logic for the existence of
Debian/Ubuntu packages on our apt repositories.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoBuild the cwltest package, which lives out of tree.
Ward Vandewege [Tue, 12 Mar 2019 14:31:13 +0000 (10:31 -0400)]
Build the cwltest package, which lives out of tree.

refs #9945

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years ago13937: Removes unnecessary data passing on tests.
Lucas Di Pentima [Tue, 12 Mar 2019 14:10:51 +0000 (11:10 -0300)]
13937: Removes unnecessary data passing on tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Updates API documentation.
Lucas Di Pentima [Tue, 12 Mar 2019 12:51:11 +0000 (09:51 -0300)]
13593: Updates API documentation.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years ago13593: Adds 'async' parameter to groups create & update calls for perms updates.
Lucas Di Pentima [Tue, 12 Mar 2019 12:33:30 +0000 (09:33 -0300)]
13593: Adds 'async' parameter to groups create & update calls for perms updates.

When async=true, the permission graph update will be deferred by the configured
amount of seconds at 'async_permissions_update_interval' (default: 60)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

5 years agoFix error message.
Tom Clegg [Mon, 11 Mar 2019 20:21:26 +0000 (16:21 -0400)]
Fix error message.

No issue #.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years agoExplicitly print the version numbers of virtualenv, pip, setuptools and wheel
Ward Vandewege [Mon, 11 Mar 2019 15:04:44 +0000 (11:04 -0400)]
Explicitly print the version numbers of virtualenv, pip, setuptools and wheel
when building Python packages.

refs #14936

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

5 years agoMerge branch '14938-unreliable-test'
Tom Clegg [Mon, 11 Mar 2019 04:34:33 +0000 (00:34 -0400)]
Merge branch '14938-unreliable-test'

fixes #14938

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

5 years ago14938: Fix unreliable test.
Tom Clegg [Mon, 11 Mar 2019 04:32:22 +0000 (00:32 -0400)]
14938: Fix unreliable test.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>