arvados.git
7 years ago11507: Put bufferblocks into DELETED state when deleted so they don't get
Peter Amstutz [Thu, 20 Apr 2017 21:45:37 +0000 (17:45 -0400)]
11507: Put bufferblocks into DELETED state when deleted so they don't get
reused accidentally if there are dangling references.  When packing small
blocks, update segments with the block hash instead of the bufferblock id to
avoid dangling references.

7 years ago11507: Test cases that reproduce bug
Peter Amstutz [Thu, 20 Apr 2017 21:10:30 +0000 (17:10 -0400)]
11507: Test cases that reproduce bug

7 years agoMerge branch '11510-sdk-extend-files' refs #11510
Peter Amstutz [Thu, 20 Apr 2017 18:18:44 +0000 (14:18 -0400)]
Merge branch '11510-sdk-extend-files' refs #11510

7 years ago11510: Tweak interface to better match Pythons "io.IOBase" class.
Peter Amstutz [Thu, 20 Apr 2017 18:15:49 +0000 (14:15 -0400)]
11510: Tweak interface to better match Pythons "io.IOBase" class.

7 years agoMerge branch '11470-update-task-fields'
Tom Clegg [Thu, 20 Apr 2017 18:10:50 +0000 (14:10 -0400)]
Merge branch '11470-update-task-fields'

refs #11470

7 years ago11510: Fix truncate() to not change file pointer. Additional tests.
Peter Amstutz [Thu, 20 Apr 2017 17:25:56 +0000 (13:25 -0400)]
11510: Fix truncate() to not change file pointer.  Additional tests.

7 years ago11510: Repack writes any time there's more than one segment referencing the same...
Peter Amstutz [Thu, 20 Apr 2017 14:41:05 +0000 (10:41 -0400)]
11510: Repack writes any time there's more than one segment referencing the same bufferblock.

7 years ago11510: Fix longstanding bug in replace_range() when appending data from a buffer...
Peter Amstutz [Tue, 18 Apr 2017 21:54:57 +0000 (17:54 -0400)]
11510: Fix longstanding bug in replace_range() when appending data from a buffer that was written out of order.

7 years ago11470: Fix identifiers to conventional Go.
Tom Clegg [Thu, 20 Apr 2017 02:12:42 +0000 (22:12 -0400)]
11470: Fix identifiers to conventional Go.

7 years ago11470: Fix accidentally cleared fields during job_task update.
Tom Clegg [Thu, 20 Apr 2017 02:09:45 +0000 (22:09 -0400)]
11470: Fix accidentally cleared fields during job_task update.

Updating with a Task struct had the effect of clearing job_uuid,
created_by_job_task_uuid, and parameters fields, and resetting
sequence to 0.

7 years agoMerge branch '11502-unstripped-manifest-fix'
Lucas Di Pentima [Wed, 19 Apr 2017 20:42:26 +0000 (17:42 -0300)]
Merge branch '11502-unstripped-manifest-fix'
Closes #11502, #11511

7 years ago11502: Merge branch 'master' into 11502-unstripped-manifest-fix
Lucas Di Pentima [Wed, 19 Apr 2017 19:38:30 +0000 (16:38 -0300)]
11502: Merge branch 'master' into 11502-unstripped-manifest-fix

7 years ago11502: Instead of asking for a stripped manifest, replace the tokens with some consta...
Lucas Di Pentima [Wed, 19 Apr 2017 19:28:26 +0000 (16:28 -0300)]
11502: Instead of asking for a stripped manifest, replace the tokens with some constant string to do the comparison.

7 years agoMerge branch '11517-indexes'
Tom Clegg [Wed, 19 Apr 2017 19:04:20 +0000 (15:04 -0400)]
Merge branch '11517-indexes'

refs #11517

7 years ago11517: Add indexes to help slow queries.
Tom Clegg [Wed, 19 Apr 2017 18:00:01 +0000 (14:00 -0400)]
11517: Add indexes to help slow queries.

7 years agocloses #11450
radhika [Wed, 19 Apr 2017 15:27:33 +0000 (11:27 -0400)]
closes #11450
Merge branch '11450-workflows-page'

7 years ago11450: workflows index page
radhika [Fri, 14 Apr 2017 17:29:54 +0000 (13:29 -0400)]
11450: workflows index page

7 years ago11510: Add support for sparse writes to ArvadosFile.
Peter Amstutz [Tue, 18 Apr 2017 20:40:51 +0000 (16:40 -0400)]
11510: Add support for sparse writes to ArvadosFile.

Allow seeking past end of file.  Allow using truncate() to extend file size.  Add zero padding to fill in gaps.

7 years ago8019: expose crunch log throttling config parameters via discovery document
radhika [Tue, 18 Apr 2017 20:35:08 +0000 (16:35 -0400)]
8019: expose crunch log throttling config parameters via discovery document

7 years ago11459: Move RAILS_PACKAGE_ITERATION to run-library.
Tom Clegg [Tue, 18 Apr 2017 18:01:47 +0000 (14:01 -0400)]
11459: Move RAILS_PACKAGE_ITERATION to run-library.

Otherwise, the SSO build script cannot see it.

refs #11459

7 years agoMerge branch '11509-keep-web-cors-range'
Tom Clegg [Tue, 18 Apr 2017 17:01:01 +0000 (13:01 -0400)]
Merge branch '11509-keep-web-cors-range'

closes #11509

7 years ago11509: Add comments.
Tom Clegg [Tue, 18 Apr 2017 17:00:27 +0000 (13:00 -0400)]
11509: Add comments.

7 years ago11502: Use os.path.join instead of formatting path-like strings on tests.
Lucas Di Pentima [Tue, 18 Apr 2017 16:45:39 +0000 (13:45 -0300)]
11502: Use os.path.join instead of formatting path-like strings on tests.

7 years ago11502: Avoid races on test by getting the unstripped manifest version before the...
Lucas Di Pentima [Tue, 18 Apr 2017 16:36:44 +0000 (13:36 -0300)]
11502: Avoid races on test by getting the unstripped manifest version before the comparison.

7 years ago11502: Fix regression by writing a collection manifest with its access tokens in...
Lucas Di Pentima [Tue, 18 Apr 2017 15:31:48 +0000 (12:31 -0300)]
11502: Fix regression by writing a collection manifest with its access tokens in place when asked by PDH & UUID.
Added --strip-manifest argument to allow writing manifests without tokens.
Updated tests accordingly.

7 years agoFix ignored symlink failure.
Tom Clegg [Tue, 18 Apr 2017 15:25:15 +0000 (11:25 -0400)]
Fix ignored symlink failure.

No issue #

7 years agorefs #11465
radhika [Tue, 18 Apr 2017 13:57:55 +0000 (09:57 -0400)]
refs #11465
Merge branch '11465-disabled-collection-file-tooltips'

7 years ago11465: tooltip on disabled pencil icon
radhika [Tue, 18 Apr 2017 13:56:48 +0000 (09:56 -0400)]
11465: tooltip on disabled pencil icon

7 years ago11509: Allow cross-origin requests with Range headers.
Tom Clegg [Mon, 17 Apr 2017 23:19:10 +0000 (19:19 -0400)]
11509: Allow cross-origin requests with Range headers.

7 years agocloses #11465
radhika [Mon, 17 Apr 2017 19:17:13 +0000 (15:17 -0400)]
closes #11465
Merge branch '11465-disabled-collection-file-tooltips'

7 years ago11465: correct disabled tooltip behavior on image types
radhika [Mon, 17 Apr 2017 17:34:44 +0000 (13:34 -0400)]
11465: correct disabled tooltip behavior on image types

7 years agoFix crunch-run tests.
Tom Clegg [Mon, 17 Apr 2017 14:01:06 +0000 (10:01 -0400)]
Fix crunch-run tests.

refs #11209

7 years ago11465: add tooltips to disabled collection file controls
radhika [Mon, 17 Apr 2017 01:20:11 +0000 (21:20 -0400)]
11465: add tooltips to disabled collection file controls

7 years agoMerge branch '11209-crunch-unmount-all'
Tom Clegg [Fri, 14 Apr 2017 20:42:12 +0000 (16:42 -0400)]
Merge branch '11209-crunch-unmount-all'

refs #11209

7 years agoMerge branch '11502-arvget-flaky-test'
Lucas Di Pentima [Fri, 14 Apr 2017 20:26:29 +0000 (17:26 -0300)]
Merge branch '11502-arvget-flaky-test'
Closes #11502

7 years ago11502: When getting a manifest using a PDH, the manifest is stripped.
Lucas Di Pentima [Fri, 14 Apr 2017 20:04:35 +0000 (17:04 -0300)]
11502: When getting a manifest using a PDH, the manifest is stripped.
Corrected test so that when comparing manifests, it does so by comparing stripped ones so that it doesn't intermittently fail.

7 years ago11209: Use arv-mount --unmount-all instead of mount|awk|xargs script to clean up...
Tom Clegg [Fri, 14 Apr 2017 17:19:16 +0000 (13:19 -0400)]
11209: Use arv-mount --unmount-all instead of mount|awk|xargs script to clean up stale mounts from previous jobs.

7 years agoMerge branch '11462-cwl-cache-collectionreader' refs #11462
Peter Amstutz [Fri, 14 Apr 2017 15:17:46 +0000 (11:17 -0400)]
Merge branch '11462-cwl-cache-collectionreader' refs #11462

7 years ago11462: Log ArgumentError in make_output_collection. Add test for CollectionCache.
Peter Amstutz [Fri, 14 Apr 2017 15:08:01 +0000 (11:08 -0400)]
11462: Log ArgumentError in make_output_collection.  Add test for CollectionCache.

7 years agoFix regression, restore a line that shouldn't have been removed. refs #10401
Peter Amstutz [Fri, 14 Apr 2017 14:21:33 +0000 (10:21 -0400)]
Fix regression, restore a line that shouldn't have been removed. refs #10401

7 years ago11462: Fix crunch script for fetcher_constructor constructor change.
Peter Amstutz [Thu, 13 Apr 2017 21:32:44 +0000 (17:32 -0400)]
11462: Fix crunch script for fetcher_constructor constructor change.

7 years agoAdd missing test file. refs #10401
Peter Amstutz [Thu, 13 Apr 2017 19:48:50 +0000 (15:48 -0400)]
Add missing test file. refs #10401

7 years ago11462: Runner asks for 3000 MB of RAM by default instead of 1024.
Peter Amstutz [Thu, 13 Apr 2017 19:54:15 +0000 (15:54 -0400)]
11462: Runner asks for 3000 MB of RAM by default instead of 1024.

7 years ago11462: Add missing test file.
Peter Amstutz [Thu, 13 Apr 2017 19:48:50 +0000 (15:48 -0400)]
11462: Add missing test file.

7 years ago11462: Bump cwltool version for mergedirs() fix
Peter Amstutz [Thu, 13 Apr 2017 19:46:05 +0000 (15:46 -0400)]
11462: Bump cwltool version for mergedirs() fix

7 years ago11462: Store CollectionReader objects in a central cache to avoid redundant
Peter Amstutz [Thu, 13 Apr 2017 19:11:39 +0000 (15:11 -0400)]
11462: Store CollectionReader objects in a central cache to avoid redundant
fetches & storing multiple copies.  On pipeline with large inputs, reduced
memory usage by up to 70%.

7 years ago11465: add tooltips to disabled collection file controls
radhika [Thu, 13 Apr 2017 18:20:32 +0000 (14:20 -0400)]
11465: add tooltips to disabled collection file controls

7 years agoMerge branch '11493-cwl-six' refs #11493
Peter Amstutz [Thu, 13 Apr 2017 15:40:27 +0000 (11:40 -0400)]
Merge branch '11493-cwl-six' refs #11493

7 years ago11493: Bump cwltool version for relaxed dependency on 'six'
Peter Amstutz [Thu, 13 Apr 2017 15:13:41 +0000 (11:13 -0400)]
11493: Bump cwltool version for relaxed dependency on 'six'

7 years agoMerge branch '8567-moar-docker' refs #8567
Peter Amstutz [Wed, 12 Apr 2017 20:10:13 +0000 (16:10 -0400)]
Merge branch '8567-moar-docker' refs #8567

7 years agoMerge branch '10401-limit-dir-expansion' closes #10401
Peter Amstutz [Wed, 12 Apr 2017 18:45:24 +0000 (14:45 -0400)]
Merge branch '10401-limit-dir-expansion' closes #10401

7 years ago8567: Migrate unnamed images
Peter Amstutz [Wed, 12 Apr 2017 18:38:33 +0000 (18:38 +0000)]
8567: Migrate unnamed images

7 years agoMerge branch '11341-arvput-resume-error'
Lucas Di Pentima [Wed, 12 Apr 2017 18:37:03 +0000 (15:37 -0300)]
Merge branch '11341-arvput-resume-error'
Closes #11341

7 years agoMerge branch '8465-cwl-containers-stdin-stderr' closes #8465
Peter Amstutz [Wed, 12 Apr 2017 18:27:15 +0000 (14:27 -0400)]
Merge branch '8465-cwl-containers-stdin-stderr' closes #8465

7 years agocloses #11365
radhika [Wed, 12 Apr 2017 16:42:16 +0000 (12:42 -0400)]
closes #11365
Merge branch '11365-collection-lock-button'

7 years ago11341: Fixed log message
Lucas Di Pentima [Wed, 12 Apr 2017 16:34:49 +0000 (13:34 -0300)]
11341: Fixed log message

7 years ago11341: Added additional information log for cases when the user ask to not use any...
Lucas Di Pentima [Wed, 12 Apr 2017 16:32:49 +0000 (13:32 -0300)]
11341: Added additional information log for cases when the user ask to not use any cache.

7 years ago10401: Document LoadListingRequirement in Arvados extensions.
Peter Amstutz [Wed, 12 Apr 2017 16:22:25 +0000 (12:22 -0400)]
10401: Document LoadListingRequirement in Arvados extensions.

7 years ago11365: "lock" collection to prevent user from deleting / renaming files until user...
radhika [Fri, 7 Apr 2017 21:08:14 +0000 (17:08 -0400)]
11365: "lock" collection to prevent user from deleting / renaming files until user explicitly unlocks by clicking the button.

7 years ago10401: Add integration tests for directory listing behavior.
Peter Amstutz [Wed, 12 Apr 2017 16:08:52 +0000 (12:08 -0400)]
10401: Add integration tests for directory listing behavior.

7 years ago11341: Use NamedTemporaryFile instead of mkstemp for cache saving.
Lucas Di Pentima [Wed, 12 Apr 2017 16:07:05 +0000 (13:07 -0300)]
11341: Use NamedTemporaryFile instead of mkstemp for cache saving.
Added logging messages to inform the user what's happening with the cache file.

7 years ago11349: Fix section name in example configs.
Tom Clegg [Wed, 12 Apr 2017 15:37:54 +0000 (11:37 -0400)]
11349: Fix section name in example configs.

refs #11349

7 years ago8465: Add unit test for generating container request with std* redirection.
Peter Amstutz [Wed, 12 Apr 2017 14:33:25 +0000 (10:33 -0400)]
8465: Add unit test for generating container request with std* redirection.

7 years agoMerge branch 'master' into 10401-limit-dir-expansion
Peter Amstutz [Wed, 12 Apr 2017 14:12:06 +0000 (10:12 -0400)]
Merge branch 'master' into 10401-limit-dir-expansion

7 years agocwl/test_with_arvbox.sh pulls arvados/jobs:latest refs #11457
Peter Amstutz [Tue, 11 Apr 2017 23:03:52 +0000 (19:03 -0400)]
cwl/test_with_arvbox.sh pulls arvados/jobs:latest refs #11457

7 years agoBuild SSO packages for Ubuntu 1604. refs #11457
Peter Amstutz [Tue, 11 Apr 2017 22:00:25 +0000 (18:00 -0400)]
Build SSO packages for Ubuntu 1604. refs #11457

7 years ago10401: Get structure.sql from master.
Peter Amstutz [Tue, 11 Apr 2017 19:56:56 +0000 (15:56 -0400)]
10401: Get structure.sql from master.

7 years agoMerge branch 'master' into 10401-limit-dir-expansion
Peter Amstutz [Tue, 11 Apr 2017 19:55:30 +0000 (15:55 -0400)]
Merge branch 'master' into 10401-limit-dir-expansion

7 years agoBuild packages for python "future" module.
Tom Clegg [Tue, 11 Apr 2017 18:31:42 +0000 (14:31 -0400)]
Build packages for python "future" module.

refs #11349
refs #11308

7 years agorefs #8465
radhika [Tue, 11 Apr 2017 18:34:46 +0000 (14:34 -0400)]
refs #8465
Merge branch '8465-stderr-redirection'

7 years agoMerge branch 'master' into 8465-stderr-redirection
radhika [Tue, 11 Apr 2017 18:08:21 +0000 (14:08 -0400)]
Merge branch 'master' into 8465-stderr-redirection

7 years ago8465: arvados-cwl-runner support for stdin and stderr redirection with containers.
Peter Amstutz [Tue, 11 Apr 2017 17:40:30 +0000 (13:40 -0400)]
8465: arvados-cwl-runner support for stdin and stderr redirection with containers.

7 years agoMerge branch '11349-nodemanager-status-api'
Tom Clegg [Tue, 11 Apr 2017 15:33:17 +0000 (11:33 -0400)]
Merge branch '11349-nodemanager-status-api'

refs #11349

7 years ago11349: Move assertion outside loop for clarity.
Tom Clegg [Tue, 11 Apr 2017 15:17:05 +0000 (11:17 -0400)]
11349: Move assertion outside loop for clarity.

7 years ago11349: Warn if management/status server is not enabled.
Tom Clegg [Tue, 11 Apr 2017 15:16:50 +0000 (11:16 -0400)]
11349: Warn if management/status server is not enabled.

7 years ago7709: Clear deliveries before each test. Fixes flaky test.
Tom Clegg [Mon, 10 Apr 2017 20:56:37 +0000 (16:56 -0400)]
7709: Clear deliveries before each test. Fixes flaky test.

7 years ago7709: Remove Ruby websocket server.
Tom Clegg [Mon, 10 Apr 2017 18:57:56 +0000 (14:57 -0400)]
7709: Remove Ruby websocket server.

7 years agoMerge branch '6304-dns-update-fix'
Lucas Di Pentima [Tue, 11 Apr 2017 13:33:24 +0000 (10:33 -0300)]
Merge branch '6304-dns-update-fix'
Closes #6304

7 years ago6304: Change assertion type for better failure message.
Lucas Di Pentima [Tue, 11 Apr 2017 13:32:10 +0000 (10:32 -0300)]
6304: Change assertion type for better failure message.

7 years agoSecure link to google search.
Tom Clegg [Mon, 10 Apr 2017 19:45:40 +0000 (15:45 -0400)]
Secure link to google search.

No issue #

7 years ago6304: Merge branch 'master' into 6304-dns-update-fix
Lucas Di Pentima [Mon, 10 Apr 2017 17:58:08 +0000 (14:58 -0300)]
6304: Merge branch 'master' into 6304-dns-update-fix

7 years ago6304: Corrected test setup to make it pass.
Lucas Di Pentima [Mon, 10 Apr 2017 17:56:04 +0000 (14:56 -0300)]
6304: Corrected test setup to make it pass.
Added a separator to the temp file so that its name is more readable.

7 years agoMerge branch '8465-stdin-redirection' into 8465-stderr-redirection
radhika [Mon, 10 Apr 2017 17:33:11 +0000 (13:33 -0400)]
Merge branch '8465-stdin-redirection' into 8465-stderr-redirection

7 years ago8465: test cleanup
radhika [Mon, 10 Apr 2017 17:32:22 +0000 (13:32 -0400)]
8465: test cleanup

7 years ago7709: Fix ActiveRecord::ImmutableRelation.
Tom Clegg [Mon, 10 Apr 2017 17:07:07 +0000 (13:07 -0400)]
7709: Fix ActiveRecord::ImmutableRelation.

7 years ago6304: Clean up temp files that could exist in case of writing errors.
Lucas Di Pentima [Mon, 10 Apr 2017 16:51:28 +0000 (13:51 -0300)]
6304: Clean up temp files that could exist in case of writing errors.
Also added a test for this case, not working properly atm.

7 years ago7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg [Mon, 10 Apr 2017 16:37:42 +0000 (12:37 -0400)]
7709: Merge branch 'master' into 7709-api-rails4

Conflicts:
services/api/app/controllers/arvados/v1/nodes_controller.rb
services/api/app/models/container_request.rb
services/api/db/structure.sql

7 years agoMerge branch '11332-cwl-url-quoting' refs #11332
Peter Amstutz [Mon, 10 Apr 2017 14:59:05 +0000 (10:59 -0400)]
Merge branch '11332-cwl-url-quoting' refs #11332

7 years ago11332: Add @ to quote safelist.
Peter Amstutz [Mon, 10 Apr 2017 14:58:33 +0000 (10:58 -0400)]
11332: Add @ to quote safelist.

7 years ago11349: Report wishlist size in status["nodes_wish"].
Tom Clegg [Mon, 10 Apr 2017 14:20:31 +0000 (10:20 -0400)]
11349: Report wishlist size in status["nodes_wish"].

7 years ago11332: Use urllib quote/unquote to handle special chars (e.g. '#')
Peter Amstutz [Mon, 10 Apr 2017 14:07:23 +0000 (10:07 -0400)]
11332: Use urllib quote/unquote to handle special chars (e.g. '#')

7 years ago11349: Add management server with /status.json
Tom Clegg [Fri, 7 Apr 2017 19:20:04 +0000 (15:20 -0400)]
11349: Add management server with /status.json

7 years ago8465: Set proper container flags to provide data on stdin.
Peter Amstutz [Fri, 7 Apr 2017 21:06:01 +0000 (17:06 -0400)]
8465: Set proper container flags to provide data on stdin.

7 years agoMerge branch '11237-google-api-python-client' refs #11237
Peter Amstutz [Fri, 7 Apr 2017 19:26:54 +0000 (15:26 -0400)]
Merge branch '11237-google-api-python-client' refs #11237

7 years ago11237: Tweak version dependencies.
Peter Amstutz [Fri, 7 Apr 2017 18:31:21 +0000 (14:31 -0400)]
11237: Tweak version dependencies.

7 years ago11237: Don't build oauth2client on ubuntu1604
Peter Amstutz [Fri, 7 Apr 2017 18:21:35 +0000 (14:21 -0400)]
11237: Don't build oauth2client on ubuntu1604

7 years ago11237: Bump google-api-python-client to 1.6.2 for better compatability across
Peter Amstutz [Fri, 7 Apr 2017 18:18:57 +0000 (14:18 -0400)]
11237: Bump google-api-python-client to 1.6.2 for better compatability across
oauth2client and Python versions.

7 years agoMerge branch '11209-unmount-subtype'
Tom Clegg [Fri, 7 Apr 2017 17:23:57 +0000 (13:23 -0400)]
Merge branch '11209-unmount-subtype'

refs #11209

7 years agoIncrease ping delay in WatchdogActorTest to try and reduce spurious test failures...
Peter Amstutz [Fri, 7 Apr 2017 15:43:56 +0000 (11:43 -0400)]
Increase ping delay in WatchdogActorTest to try and reduce spurious test failures.  no issue #

7 years agoAdd check that gnutls.h is available so pycurl can compile. no issue #
Peter Amstutz [Fri, 7 Apr 2017 15:36:30 +0000 (11:36 -0400)]
Add check that gnutls.h is available so pycurl can compile.  no issue #