arvados.git
6 years agoMerge branch 'wtsi/13093-crunch-dispatch-slurm-add-mem' refs #13093
Peter Amstutz [Fri, 23 Feb 2018 18:26:59 +0000 (13:26 -0500)]
Merge branch 'wtsi/13093-crunch-dispatch-slurm-add-mem' refs #13093

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13064-arv-mount-perf' closes #13064
Peter Amstutz [Fri, 23 Feb 2018 16:15:44 +0000 (11:15 -0500)]
Merge branch '13064-arv-mount-perf' closes #13064

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13064: Add comment about has_ref() and in_use() checks
Peter Amstutz [Fri, 23 Feb 2018 16:15:03 +0000 (11:15 -0500)]
13064: Add comment about has_ref() and in_use() checks

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13064: Explicitly precompile regexes used in manifest parsing
Peter Amstutz [Thu, 22 Feb 2018 20:58:56 +0000 (15:58 -0500)]
13064: Explicitly precompile regexes used in manifest parsing

Seems to improve parsing time roughly 5-10%

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13064: Add __slots__ to ArvadosFile and FuseArvadosFile
Peter Amstutz [Thu, 22 Feb 2018 19:47:42 +0000 (14:47 -0500)]
13064: Add __slots__ to ArvadosFile and FuseArvadosFile

More than 50% decrease in memory footprint for large collections!!!

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13064: Check has_ref before checking in_use
Peter Amstutz [Thu, 22 Feb 2018 19:09:33 +0000 (14:09 -0500)]
13064: Check has_ref before checking in_use

Kernel behavior seems to be that if a file is referenced, its parents
remain referenced too.  This means has_ref() exits early when a
collection is not candidate for eviction.

By contrast, in_use() doesn't increment references on parents, so it
requires a full tree walk to determine if a collection is a candidate
for eviction.

Switching the order of the checks retains correctness but greatly
improves real-world performance.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoadd ReserveExtraRAM config option to crunch-dispatch-slurm
Joshua C. Randall [Thu, 15 Feb 2018 18:50:19 +0000 (18:50 +0000)]
add ReserveExtraRAM config option to crunch-dispatch-slurm

Arvados-DCO-1.1-Signed-off-by: Joshua C. Randall <jcrandall@alum.mit.edu>

6 years agoMerge branch '11645-keepstore-storageclasses' closes #11645
Peter Amstutz [Wed, 21 Feb 2018 20:46:32 +0000 (15:46 -0500)]
Merge branch '11645-keepstore-storageclasses' closes #11645

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11645: Add tests that StorageClasses is loaded from config
Peter Amstutz [Wed, 21 Feb 2018 20:42:48 +0000 (15:42 -0500)]
11645: Add tests that StorageClasses is loaded from config

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix typos/grammar.
Ward Vandewege [Wed, 21 Feb 2018 20:13:02 +0000 (15:13 -0500)]
Fix typos/grammar.

No issue #

Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege@veritasgenetics.com>

6 years agoMerge branch '13106-keepclient-slot' closes #13106
Peter Amstutz [Wed, 21 Feb 2018 17:06:04 +0000 (12:06 -0500)]
Merge branch '13106-keepclient-slot' closes #13106

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13106: Add note about the test.
Peter Amstutz [Wed, 21 Feb 2018 17:05:36 +0000 (12:05 -0500)]
13106: Add note about the test.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11645: Add support for StorageClasses in volume config.
Peter Amstutz [Wed, 21 Feb 2018 16:56:39 +0000 (11:56 -0500)]
11645: Add support for StorageClasses in volume config.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13106: Add test confirming deadlock
Peter Amstutz [Wed, 21 Feb 2018 01:32:44 +0000 (20:32 -0500)]
13106: Add test confirming deadlock

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13106: Ensure cache slot is reset on error
Peter Amstutz [Tue, 20 Feb 2018 21:29:22 +0000 (16:29 -0500)]
13106: Ensure cache slot is reset on error

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13098-assign-hostname'
Tom Clegg [Fri, 16 Feb 2018 19:09:30 +0000 (14:09 -0500)]
Merge branch '13098-assign-hostname'

refs #13098
refs #12199

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13098: 12199: Fix hostname assigned but not saved.
Tom Clegg [Fri, 16 Feb 2018 18:55:24 +0000 (13:55 -0500)]
13098: 12199: Fix hostname assigned but not saved.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13063-update-azure-sdk'
Tom Clegg [Fri, 16 Feb 2018 17:48:34 +0000 (12:48 -0500)]
Merge branch '13063-update-azure-sdk'

refs #13063

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13063: Use upstream azure-sdk-for-go.
Tom Clegg [Fri, 16 Feb 2018 17:41:48 +0000 (12:41 -0500)]
13063: Use upstream azure-sdk-for-go.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13090-eval-timeout' closes #13090
Peter Amstutz [Fri, 16 Feb 2018 16:41:01 +0000 (11:41 -0500)]
Merge branch '13090-eval-timeout'  closes #13090

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago13090: Pass through --eval-timeout to container command line.
Peter Amstutz [Fri, 16 Feb 2018 15:53:49 +0000 (10:53 -0500)]
13090: Pass through --eval-timeout to container command line.

Also adds a check so that --ignore-docker-for-reuse can't be used with
containers, refs #12666

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoAdd missing parameter to update call.
Tom Clegg [Thu, 15 Feb 2018 20:31:33 +0000 (15:31 -0500)]
Add missing parameter to update call.

fixes #12199

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11876-R-deps' refs #11876
Peter Amstutz [Thu, 15 Feb 2018 19:51:43 +0000 (14:51 -0500)]
Merge branch '11876-R-deps' refs #11876

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Pin R deps that are installed from gitub.
Peter Amstutz [Thu, 15 Feb 2018 18:34:11 +0000 (13:34 -0500)]
11876: Pin R deps that are installed from gitub.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoMerge branch '13063-update-azure-sdk'
Tom Clegg [Thu, 15 Feb 2018 18:38:55 +0000 (13:38 -0500)]
Merge branch '13063-update-azure-sdk'

closes #13063

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11454-remove-button-removal'
Lucas Di Pentima [Thu, 15 Feb 2018 16:53:26 +0000 (13:53 -0300)]
Merge branch '11454-remove-button-removal'
Refs #11454

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoR SDK documentation link fix refs #11876
Peter Amstutz [Thu, 15 Feb 2018 16:07:24 +0000 (11:07 -0500)]
R SDK documentation link fix refs #11876

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11454: Code readability enhancement.
Lucas Di Pentima [Thu, 15 Feb 2018 16:00:57 +0000 (13:00 -0300)]
11454: Code readability enhancement.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13063: Don't use azure default delay-and-retry loop.
Tom Clegg [Thu, 15 Feb 2018 15:55:37 +0000 (10:55 -0500)]
13063: Don't use azure default delay-and-retry loop.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11454: Don't show 'Remove' session button when listed on remoteHosts.
Lucas Di Pentima [Thu, 15 Feb 2018 14:57:31 +0000 (11:57 -0300)]
11454: Don't show 'Remove' session button when listed on remoteHosts.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13063: Update azure storage client library.
Tom Clegg [Wed, 14 Feb 2018 21:16:12 +0000 (16:16 -0500)]
13063: Update azure storage client library.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12863-wb-cr-status'
Lucas Di Pentima [Wed, 14 Feb 2018 16:53:04 +0000 (13:53 -0300)]
Merge branch '12863-wb-cr-status'
Closes #12863

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '13023-wb-log-newlines'
Lucas Di Pentima [Wed, 14 Feb 2018 16:08:05 +0000 (13:08 -0300)]
Merge branch '13023-wb-log-newlines'
Closes #13023

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12199-dispatch-to-node-type'
Tom Clegg [Wed, 14 Feb 2018 15:33:49 +0000 (10:33 -0500)]
Merge branch '12199-dispatch-to-node-type'

refs #12199

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13025-keepstore-metrics'
Tom Clegg [Wed, 14 Feb 2018 15:20:02 +0000 (10:20 -0500)]
Merge branch '13025-keepstore-metrics'

refs #13025

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Fix job_uuid missing from nodes#update response.
Tom Clegg [Wed, 14 Feb 2018 15:10:24 +0000 (10:10 -0500)]
12199: Fix job_uuid missing from nodes#update response.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Ensure scratch space for tmp mounts when choosing node type.
Tom Clegg [Tue, 13 Feb 2018 20:10:40 +0000 (15:10 -0500)]
12199: Ensure scratch space for tmp mounts when choosing node type.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13023: Workaround firefox copy&paste bug on jobs & pipeline log viewers.
Lucas Di Pentima [Tue, 13 Feb 2018 19:45:59 +0000 (16:45 -0300)]
13023: Workaround firefox copy&paste bug on jobs & pipeline log viewers.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13023: Fix integration test.
Lucas Di Pentima [Tue, 13 Feb 2018 19:13:39 +0000 (16:13 -0300)]
13023: Fix integration test.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12199: Move services/dispatchcloud to lib/dispatchcloud.
Tom Clegg [Tue, 13 Feb 2018 16:40:18 +0000 (11:40 -0500)]
12199: Move services/dispatchcloud to lib/dispatchcloud.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13023: Fixes related tests.
Lucas Di Pentima [Tue, 13 Feb 2018 16:19:47 +0000 (13:19 -0300)]
13023: Fixes related tests.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13023: Replaces div with pre tags on 'Recent logs' viewer.
Lucas Di Pentima [Tue, 13 Feb 2018 16:01:30 +0000 (13:01 -0300)]
13023: Replaces div with pre tags on 'Recent logs' viewer.

This is to workaround a Firefox bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1174452

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12863: Adds CR's state label on description cell.
Lucas Di Pentima [Tue, 13 Feb 2018 13:56:44 +0000 (10:56 -0300)]
12863: Adds CR's state label on description cell.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12863: Adds column showing process status on project pages.
Lucas Di Pentima [Mon, 12 Feb 2018 22:41:13 +0000 (19:41 -0300)]
12863: Adds column showing process status on project pages.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12199: Merge branch 'master' into 12199-dispatch-to-node-type
Tom Clegg [Mon, 12 Feb 2018 22:28:22 +0000 (17:28 -0500)]
12199: Merge branch 'master' into 12199-dispatch-to-node-type

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Improve race handling in busywait.
Tom Clegg [Mon, 12 Feb 2018 22:06:13 +0000 (17:06 -0500)]
12199: Improve race handling in busywait.

The previous implementation was "call f() until it's truthy, then
assert f()". The assertion fails if f() returns truthy but then
returns falsy on the next call.

The new implementation doesn't call f() again after it returns truthy
once.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Fix errant uses of size.name that should be size.id.
Tom Clegg [Mon, 12 Feb 2018 21:48:15 +0000 (16:48 -0500)]
12199: Fix errant uses of size.name that should be size.id.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Update integration tests.
Tom Clegg [Mon, 12 Feb 2018 21:39:23 +0000 (16:39 -0500)]
12199: Update integration tests.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13025: Merge branch 'master' into 13025-keepstore-metrics
Tom Clegg [Mon, 12 Feb 2018 19:53:50 +0000 (14:53 -0500)]
13025: Merge branch 'master' into 13025-keepstore-metrics

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago13025: Revert returning WroteStatus()==200 if no header was written.
Tom Clegg [Mon, 12 Feb 2018 19:51:17 +0000 (14:51 -0500)]
13025: Revert returning WroteStatus()==200 if no header was written.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11454-wb-federated-search'
Lucas Di Pentima [Mon, 12 Feb 2018 18:41:44 +0000 (15:41 -0300)]
Merge branch '11454-wb-federated-search'
Closes #11454
Refs #12959

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: Avoid instantiating new SessionDB before autoRedirectToHomeCluster()
Lucas Di Pentima [Mon, 12 Feb 2018 17:15:11 +0000 (14:15 -0300)]
11454: Avoid instantiating new SessionDB before autoRedirectToHomeCluster()

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: A couple of fixes/enhancements:
Lucas Di Pentima [Mon, 12 Feb 2018 16:40:11 +0000 (13:40 -0300)]
11454: A couple of fixes/enhancements:

* Only try to migrate non federated sessions on /sessions page.
* autoLoadRemoteHosts only loads missing remotes, without reactivating
disabled sessions.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13025: Lint.
Tom Clegg [Mon, 12 Feb 2018 16:30:43 +0000 (11:30 -0500)]
13025: Lint.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11454: Removes workbench links on /sessions page.
Lucas Di Pentima [Mon, 12 Feb 2018 16:12:07 +0000 (13:12 -0300)]
11454: Removes workbench links on /sessions page.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12840-cancelled-pipeline'
Tom Clegg [Mon, 12 Feb 2018 02:27:32 +0000 (21:27 -0500)]
Merge branch '12840-cancelled-pipeline'

fixes #12840

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '12991-docker-memory-limit'
Tom Clegg [Mon, 12 Feb 2018 02:24:48 +0000 (21:24 -0500)]
Merge branch '12991-docker-memory-limit'

fixes #12991

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '13022-tmp-cleanup'
Tom Clegg [Mon, 12 Feb 2018 02:23:40 +0000 (21:23 -0500)]
Merge branch '13022-tmp-cleanup'

fixes #13022

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11876-r-sdk' documentation, refs #11876
Peter Amstutz [Fri, 9 Feb 2018 19:20:32 +0000 (14:20 -0500)]
Merge branch '11876-r-sdk' documentation, refs #11876

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Improve formatting
Peter Amstutz [Fri, 9 Feb 2018 19:10:43 +0000 (14:10 -0500)]
11876: Improve formatting

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Update installation instructions
Peter Amstutz [Fri, 9 Feb 2018 18:42:42 +0000 (13:42 -0500)]
11876: Update installation instructions

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Update install instructions to r.arvados.org
Peter Amstutz [Fri, 9 Feb 2018 14:36:34 +0000 (09:36 -0500)]
11876: Update install instructions to r.arvados.org

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Arvbox support for building R docs
Peter Amstutz [Fri, 9 Feb 2018 14:11:18 +0000 (09:11 -0500)]
11876: Arvbox support for building R docs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11876: Build R SDK reference pages for doc site
Peter Amstutz [Fri, 9 Feb 2018 14:14:51 +0000 (09:14 -0500)]
11876: Build R SDK reference pages for doc site

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11454: Replaced javascript events with proper form values
Lucas Di Pentima [Fri, 9 Feb 2018 02:26:57 +0000 (23:26 -0300)]
11454: Replaced javascript events with proper form values
on the search results form buttons.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: Code cleanup. Removed local wb link on sessions page.
Lucas Di Pentima [Thu, 8 Feb 2018 22:11:44 +0000 (19:11 -0300)]
11454: Code cleanup. Removed local wb link on sessions page.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: Replace link buttons with form buttons on search results,
Lucas Di Pentima [Thu, 8 Feb 2018 21:49:48 +0000 (18:49 -0300)]
11454: Replace link buttons with form buttons on search results,
so that embedded api tokens aren't shared accidentally.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '11454-fix-syntax-errors' 1.1.3
Lucas Di Pentima [Thu, 8 Feb 2018 20:22:02 +0000 (17:22 -0300)]
Merge branch '11454-fix-syntax-errors'
Refs #11454

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: Syntax errors fixes.
Lucas Di Pentima [Thu, 8 Feb 2018 19:49:42 +0000 (16:49 -0300)]
11454: Syntax errors fixes.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12840: Report pipeline state=Failed on job creation failure.
Tom Clegg [Thu, 8 Feb 2018 19:04:37 +0000 (14:04 -0500)]
12840: Report pipeline state=Failed on job creation failure.

There's no failed job in this case, because there's no job, but
a-r-p-i does set components_summary.failed > 0.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11454-wb-federated-search'
Lucas Di Pentima [Thu, 8 Feb 2018 16:59:10 +0000 (13:59 -0300)]
Merge branch '11454-wb-federated-search'
Refs #11454

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoAdded documentation for print classes
Fuad Muhic [Thu, 8 Feb 2018 11:45:34 +0000 (12:45 +0100)]
Added documentation for print classes

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoAdded documentation to each public class of SDK.
Fuad Muhic [Wed, 7 Feb 2018 16:42:56 +0000 (17:42 +0100)]
Added documentation to each public class of SDK.

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoReplaced README with README.Rmd
Fuad Muhic [Wed, 7 Feb 2018 11:19:25 +0000 (12:19 +0100)]
Replaced README with README.Rmd

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago13025: Add request time metrics at /metrics and /metrics.json.
Tom Clegg [Tue, 6 Feb 2018 22:52:18 +0000 (17:52 -0500)]
13025: Add request time metrics at /metrics and /metrics.json.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years agoMerge branch '11876-r-sdk' refs #11876
Peter Amstutz [Tue, 6 Feb 2018 18:15:17 +0000 (13:15 -0500)]
Merge branch '11876-r-sdk'  refs #11876

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years agoFix 5 failing tests
Fuad Muhic [Tue, 6 Feb 2018 15:06:39 +0000 (16:06 +0100)]
Fix 5 failing tests

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch 'master' of git.curoverse.com:arvados into 11876-r-sdk
Fuad Muhic [Tue, 6 Feb 2018 09:46:30 +0000 (10:46 +0100)]
Merge branch 'master' of git.curoverse.com:arvados into 11876-r-sdk

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years ago12991: Propagate memory and CPU limits to docker container.
Tom Clegg [Mon, 5 Feb 2018 22:00:27 +0000 (17:00 -0500)]
12991: Propagate memory and CPU limits to docker container.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11454: Remotes listed on remoteHosts show "enable/disable" action button
Lucas Di Pentima [Mon, 5 Feb 2018 21:27:07 +0000 (18:27 -0300)]
11454: Remotes listed on remoteHosts show "enable/disable" action button
instead of "Log in/Log out".
When a listed remote rejects the salted token, the session shows as
disabled.
When a user entered (ie: not listed) remote rejects the salted token,
classic login is attempted.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: When adding a uuid prefix on the add session box, append
Lucas Di Pentima [Mon, 5 Feb 2018 19:33:40 +0000 (16:33 -0300)]
11454: When adding a uuid prefix on the add session box, append
'.arvadosapi.com' before trying to get the API server url.
When a salted token is rejected, save the the session as logged out.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago13022: Don't abandon cleanup tasks on SIGTERM.
Tom Clegg [Mon, 5 Feb 2018 16:34:55 +0000 (11:34 -0500)]
13022: Don't abandon cleanup tasks on SIGTERM.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Make operator precedence more obvious.
Tom Clegg [Mon, 5 Feb 2018 14:52:18 +0000 (09:52 -0500)]
12199: Make operator precedence more obvious.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago12199: Track slurm node features, avoid redundant updates.
Tom Clegg [Fri, 2 Feb 2018 21:46:32 +0000 (16:46 -0500)]
12199: Track slurm node features, avoid redundant updates.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11876: Add sdk/R to run-tests.sh
Peter Amstutz [Sat, 3 Feb 2018 04:11:46 +0000 (23:11 -0500)]
11876: Add sdk/R to run-tests.sh

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago11454: On /search & /sessions page, check if the current user is
Lucas Di Pentima [Fri, 2 Feb 2018 22:29:37 +0000 (19:29 -0300)]
11454: On /search & /sessions page, check if the current user is
from a remote cluster, and if that's the case, redirect to the
proper home cluster workbench's page.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12840: Differentiate "Cancelled" from "Failed" pipeline instances.
Tom Clegg [Thu, 1 Feb 2018 15:32:53 +0000 (10:32 -0500)]
12840: Differentiate "Cancelled" from "Failed" pipeline instances.

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11454: Add the salted token on search result links from federated
Lucas Di Pentima [Fri, 2 Feb 2018 20:49:00 +0000 (17:49 -0300)]
11454: Add the salted token on search result links from federated
remote hosts to get auto-logged in when clicking through.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12268-libcloud-2-3-0'
Lucas Di Pentima [Fri, 2 Feb 2018 19:23:58 +0000 (16:23 -0300)]
Merge branch '12268-libcloud-2-3-0'
Refs #12268

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago12268: Update the libcloud dependency to our forked version 2.2.2.dev4
Lucas Di Pentima [Fri, 2 Feb 2018 18:01:53 +0000 (15:01 -0300)]
12268: Update the libcloud dependency to our forked version 2.2.2.dev4
that's really 2.3.0 release candidate from the libcloud project, to
be able to do some stress testing.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years ago11454: Removed unnecessary call on migrateNonFederatedSessions()
Lucas Di Pentima [Fri, 2 Feb 2018 14:53:48 +0000 (11:53 -0300)]
11454: Removed unnecessary call on migrateNonFederatedSessions()
Added auto loading of remote hosts listed on home cluster's api discovery doc. (WIP)

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch 'master' of git.curoverse.com:arvados into 11876-r-sdk
Fuad Muhic [Fri, 2 Feb 2018 14:45:41 +0000 (15:45 +0100)]
Merge branch 'master' of git.curoverse.com:arvados into 11876-r-sdk

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoArvados class now accepts additional parameter which specifies max
Fuad Muhic [Fri, 2 Feb 2018 14:41:44 +0000 (15:41 +0100)]
Arvados class now accepts additional parameter which specifies max
number of failed connection retries

Arvados-DCO-1.1-Signed-off-by: Fuad Muhic <fmuhic@capeannenterprises.com>

6 years agoMerge branch '12902-queued-to-cancelled'
Tom Clegg [Thu, 1 Feb 2018 16:51:40 +0000 (11:51 -0500)]
Merge branch '12902-queued-to-cancelled'

refs #12902

Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg@veritasgenetics.com>

6 years ago11454: Show a link to the remote workbench on the sessions page.
Lucas Di Pentima [Thu, 1 Feb 2018 16:40:17 +0000 (13:40 -0300)]
11454: Show a link to the remote workbench on the sessions page.
On page load, detect those non-federated active sessions and try to
migrate them to federated logins just in case they're stale sessions.
If the federated login doesn't succeed, keep the session's previous state.
Removed workbench remote_hosts config.

Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima@veritasgenetics.com>

6 years agoMerge branch '12764-secure-tmp-parent' refs #12764
Peter Amstutz [Thu, 1 Feb 2018 16:35:30 +0000 (11:35 -0500)]
Merge branch '12764-secure-tmp-parent' refs #12764

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12764: Set mode on copied single files. Fix tests.
Peter Amstutz [Thu, 1 Feb 2018 16:32:33 +0000 (11:32 -0500)]
12764: Set mode on copied single files.  Fix tests.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12764: Explicitly set desired permissions on copied files/dirs
Peter Amstutz [Thu, 1 Feb 2018 15:59:53 +0000 (10:59 -0500)]
12764: Explicitly set desired permissions on copied files/dirs

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12764: Remove CleanupTempDir. Better name for parent temp.
Peter Amstutz [Wed, 31 Jan 2018 21:59:50 +0000 (16:59 -0500)]
12764: Remove CleanupTempDir.  Better name for parent temp.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>

6 years ago12764: crunch-run temporary directories have common parent
Peter Amstutz [Wed, 31 Jan 2018 21:22:31 +0000 (16:22 -0500)]
12764: crunch-run temporary directories have common parent

Some directories created by crunch-run need to be world
readable/writable.  Apply path-based security using parent directory
with mode 0700.  Enables crunch-run to bind into the container while
avoiding making the directory trivially accessible to anyone on the
host.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz@veritasgenetics.com>