Tom Clegg [Tue, 6 Jan 2015 22:48:42 +0000 (17:48 -0500)]
3021: Wait for shown.bs.modal before trying to click butttons in the modal.
Remove a stray Headless.new.start.
Tim Pierce [Tue, 6 Jan 2015 21:21:10 +0000 (21:21 +0000)]
4598: catch exceptions more aggressively when looking up pipeline names
Added exception handling for cases where:
* job is not recorded as belonging to any pipeline instance
* pipeline instance has no pipeline template
Peter Amstutz [Tue, 6 Jan 2015 19:06:28 +0000 (14:06 -0500)]
Merge branch '4570-multi-auth-method' refs #4570
Tom Clegg [Tue, 6 Jan 2015 18:52:16 +0000 (13:52 -0500)]
3021: Fix assertion broken in
9c10212.
Peter Amstutz [Tue, 6 Jan 2015 18:24:06 +0000 (13:24 -0500)]
4570: Fix tabs, CSS on log in button.
Peter Amstutz [Tue, 6 Jan 2015 18:19:28 +0000 (13:19 -0500)]
4823: Working on porting more StreamReader tests to ArvadosFile.
Tom Clegg [Tue, 6 Jan 2015 17:26:49 +0000 (12:26 -0500)]
3021: Wait for dialog to close before asserting page transition.
Brett Smith [Tue, 6 Jan 2015 17:12:27 +0000 (12:12 -0500)]
Merge branch '4836-first-tab-load-wip'
Closes #4836, #4870.
Brett Smith [Fri, 19 Dec 2014 22:40:13 +0000 (17:40 -0500)]
4836: Trigger Workbench infinite scroll load on tab show.
If an infinite scroller is in the first tab of a show page, but the
user is going to a different tab, we'll queue up the first event
to load data for the container, but when it fires the container won't
be visible so it will decline to load anything. Then you can only get
data to load if you resize the window.
Fire a scroll event when a new tab is shown, to spur the infinite
scroller to load data as appropriate.
Peter Amstutz [Tue, 6 Jan 2015 16:32:23 +0000 (11:32 -0500)]
Merge branch 'master' into 4823-python-sdk-writable-collection-api
Tim Pierce [Tue, 6 Jan 2015 16:03:10 +0000 (11:03 -0500)]
4598: account for queued and cancelled jobs, fix sorting
Per code review:
* Updated report to include job states "Cancelled" and "Queued" as well
as Failed, Running and Complete, and to take these into account when
calculating job counts.
* Fixed sorting for failure classes.
Radhika Chippada [Tue, 6 Jan 2015 15:48:26 +0000 (10:48 -0500)]
4523: Add test to create collection with large manifest text. This test showed that the max size
the file_names column can have for successful search indexing is 2^13. Hence, setting the column
size to 2^12, so that there is enough room to accommodate slightly larger values for other columns.
Radhika Chippada [Tue, 6 Jan 2015 14:36:49 +0000 (09:36 -0500)]
4523: extract unique file names from manifest text and set as file_names attribute during save.
Peter Amstutz [Tue, 6 Jan 2015 13:45:08 +0000 (08:45 -0500)]
Merge branch 'master' into 4570-multi-auth-method
Peter Amstutz [Tue, 6 Jan 2015 13:44:49 +0000 (08:44 -0500)]
4570: Revert to links on log in page instead of form. Fixup documentation to
to describe a production setup.
Tom Clegg [Tue, 6 Jan 2015 06:02:06 +0000 (01:02 -0500)]
3021: Use headless helper in performance and diagnostics tests, too.
Tom Clegg [Tue, 6 Jan 2015 05:59:02 +0000 (00:59 -0500)]
3021: 4399: Refactor headless stuff into a module. Clear up new/start/stop use.
* Create one Headless per test process, when encountering the first
test case that needs one.
* Call headless.start & stop exactly once for each test case that uses
it.
Radhika Chippada [Tue, 6 Jan 2015 05:11:26 +0000 (00:11 -0500)]
4523: update the collection file_names migration script to set the file_names during migration.
Radhika Chippada [Tue, 6 Jan 2015 01:25:33 +0000 (20:25 -0500)]
4523: add file_names searchable column to collections data model.
Radhika Chippada [Tue, 6 Jan 2015 01:05:10 +0000 (20:05 -0500)]
4523: set description size back to 2**19
Radhika Chippada [Mon, 5 Jan 2015 21:36:31 +0000 (16:36 -0500)]
4523: set description size to 2**17, which seems to be the upper limit for indexing.
Tom Clegg [Mon, 5 Jan 2015 20:02:54 +0000 (15:02 -0500)]
4523: Come back down to earth enough to make tests pass.
Tom Clegg [Mon, 5 Jan 2015 19:40:00 +0000 (14:40 -0500)]
4523: Restore indexes accidentally deleted from structure.sql.
Tim Pierce [Mon, 5 Jan 2015 19:22:47 +0000 (14:22 -0500)]
4598: formatting and calculation fixes (code review)
Incorporating code review feedback from #4598-13.
Bugs fixed:
* Correct counting and percentage calculation of job failures.
** Jobs were getting categorized as both "unknown" and as a specific failure type.
* Crashes fixed: should not raise any unhandled exceptions.
Formatting fixes:
* Itemized failures are now sorted in descending order by failure type
* Better horizontal alignment
* Modified formatting to account for updated description.
Peter Amstutz [Mon, 5 Jan 2015 16:37:54 +0000 (11:37 -0500)]
Merge branch '4869-keepalive' refs #4869
Tom Clegg [Mon, 5 Jan 2015 15:34:01 +0000 (10:34 -0500)]
4523: Do not search :text fields at all.
Tom Clegg [Mon, 5 Jan 2015 15:31:15 +0000 (10:31 -0500)]
4523: Simplify migration.
Tom Clegg [Mon, 5 Jan 2015 15:30:58 +0000 (10:30 -0500)]
4523: Increase string length limit. (Postgres allows nearly 1MiB here, no need for a smaller limit.)
Tom Clegg [Mon, 5 Jan 2015 15:29:44 +0000 (10:29 -0500)]
4523: Description is not unsearchable after all.
Peter Amstutz [Mon, 5 Jan 2015 15:28:40 +0000 (10:28 -0500)]
Merge branch 'master' into 4823-python-sdk-writable-collection-api
Peter Amstutz [Mon, 5 Jan 2015 15:17:42 +0000 (10:17 -0500)]
4869: Client.Timeout and Client.Transport are now correctly set in
DiscoverKeepServers(). Improved comments.
Tom Clegg [Mon, 5 Jan 2015 15:05:57 +0000 (10:05 -0500)]
4523: Merge branch '4523-search-index' of git.curoverse.com:arvados into 4523-search-index
Conflicts:
services/api/test/unit/arvados_model_test.rb
Radhika Chippada [Mon, 5 Jan 2015 14:48:35 +0000 (09:48 -0500)]
Merge branch 'master' into 4523-search-index
Conflicts:
services/api/db/migrate/20141208164553_owner_uuid_index.rb
services/api/db/structure.sql
services/api/test/unit/arvados_model_test.rb
Tom Clegg [Sun, 4 Jan 2015 08:17:45 +0000 (03:17 -0500)]
3021: Clean up headless/selenium/javascript choices.
Peter Amstutz [Sun, 4 Jan 2015 04:25:22 +0000 (23:25 -0500)]
3198: Apply StreamFileReader tests to ArvadosFileReader
Peter Amstutz [Sun, 4 Jan 2015 01:48:46 +0000 (20:48 -0500)]
3198: Fixed up the rest of the tests to mock the session.
Tom Clegg [Sat, 3 Jan 2015 06:52:34 +0000 (01:52 -0500)]
3021: Skip angular init if angular is not loaded.
Peter Amstutz [Sat, 3 Jan 2015 04:34:15 +0000 (23:34 -0500)]
3198: Figured out how to mock session object, reverted most tests updated in
previous commit.
Tom Clegg [Sat, 3 Jan 2015 03:20:59 +0000 (22:20 -0500)]
3021: Silence "invalid regexp" errors while typing regexp; put input in "has-error" state instead.
Tom Clegg [Fri, 2 Jan 2015 22:38:34 +0000 (17:38 -0500)]
3021: 4399: Convert some tests from selenium to phantomjs. Restart Headless less.
Peter Amstutz [Fri, 2 Jan 2015 04:01:27 +0000 (23:01 -0500)]
3198: Added Collection.rename (needs test). Fixing tests broken on account of
using a requests session object so mocking requests.get and requests.put
doesn't work any more.
Peter Amstutz [Fri, 2 Jan 2015 02:31:25 +0000 (21:31 -0500)]
3198: KeepClient creates a requests session to re-use connections. export_manifest correctly strips hints when portable_manifest=True.
Ward Vandewege [Fri, 2 Jan 2015 02:10:49 +0000 (21:10 -0500)]
Wording tweaks to the workbench profile page.
No issue #
Tom Clegg [Wed, 31 Dec 2014 21:33:57 +0000 (16:33 -0500)]
Remove cruft. No issue #
Ward Vandewege [Wed, 31 Dec 2014 15:01:59 +0000 (10:01 -0500)]
Merge branch '4887-invalidate-duplicate-ip-on-old-compute-nodes'
closes #4887
Ward Vandewege [Wed, 31 Dec 2014 15:01:30 +0000 (10:01 -0500)]
Merge branch 'master' into 4887-invalidate-duplicate-ip-on-old-compute-nodes
Ward Vandewege [Wed, 31 Dec 2014 15:00:21 +0000 (10:00 -0500)]
Address review comments:
* change stale_conflicting_nodes to a local variable
* minor performance optimization: add an additional check for ip_address being nil
refs #4887
Peter Amstutz [Tue, 30 Dec 2014 22:10:24 +0000 (17:10 -0500)]
3198: Bugfixes. Working on benchmarking upload/download performance.
Tim Pierce [Tue, 30 Dec 2014 21:50:04 +0000 (16:50 -0500)]
Merge branch '4877-dont-delete-stdout'
Fixes #4877
Tim Pierce [Tue, 30 Dec 2014 21:45:42 +0000 (16:45 -0500)]
4877: don't delete /dev/stdout
Fixed the filename check before trying to delete /dev/stdout.
Tim Pierce [Tue, 30 Dec 2014 21:07:02 +0000 (16:07 -0500)]
4598: added failure types and short names
Added the sys/docker failure type. Failures now reported by short
failure name rather than by regex.
Peter Amstutz [Tue, 30 Dec 2014 19:51:55 +0000 (14:51 -0500)]
3198: Many tests. Fixed lots of bugs.
Tim Pierce [Tue, 30 Dec 2014 19:48:59 +0000 (14:48 -0500)]
4598: remove more dev/debugging features.
Tim Pierce [Tue, 30 Dec 2014 19:47:31 +0000 (14:47 -0500)]
4598: take out some debug reporting and --match option
Remove debugging features.
Tim Pierce [Tue, 30 Dec 2014 19:42:28 +0000 (14:42 -0500)]
4598: fetch logs from Keep, more failure reporting
Per standup review: fetch logs with a CollectionReader on the log
collection uuid, rather than fetching log records from the API server.
Perform full failure reporting including job URL details.
Ward Vandewege [Tue, 30 Dec 2014 19:31:53 +0000 (14:31 -0500)]
Detect stale compute node records with the same IP address as the new
node on its first ping. Clear the ip_address field on the stale nodes.
Refs #4887
Ward Vandewege [Tue, 30 Dec 2014 18:28:57 +0000 (13:28 -0500)]
Cleanups:
* Remove old commented out code
* Remove superfluous test for presence of file on disk
refs #4887
Tim Pierce [Tue, 30 Dec 2014 16:00:51 +0000 (11:00 -0500)]
4598: bug fixes, added full stats collection
Added code to report full stats on failed, successful, and incomplete
jobs. Perform basic reporting on failed job causes (not yet working).
Peter Amstutz [Tue, 30 Dec 2014 15:39:50 +0000 (10:39 -0500)]
4869: Enable TCP keepalive and adjust connection timeouts to Keep client.
Tom Clegg [Mon, 29 Dec 2014 22:02:01 +0000 (17:02 -0500)]
Fix whitespace, cf. gofmt. refs #4875
Tom Clegg [Mon, 29 Dec 2014 21:59:35 +0000 (16:59 -0500)]
Merge branch '4875-keepclient-test-race' closes #4875
Peter Amstutz [Mon, 29 Dec 2014 21:43:04 +0000 (16:43 -0500)]
3198: Start using BlockManager. Needs tests.
Tom Clegg [Mon, 29 Dec 2014 21:29:17 +0000 (16:29 -0500)]
4875: Merge branch 'master' into 4875-keepclient-test-race
Conflicts:
sdk/go/keepclient/keepclient_test.go
Tom Clegg [Mon, 29 Dec 2014 20:45:30 +0000 (15:45 -0500)]
Fix version strings to comply with PEP-440. No issue #
Tom Clegg [Mon, 29 Dec 2014 20:12:46 +0000 (15:12 -0500)]
Merge branch '4523-owner_uuid-index' refs #4523
Peter Amstutz [Mon, 29 Dec 2014 20:11:05 +0000 (15:11 -0500)]
Merge branch '4869-keepproxy' refs #4869
Peter Amstutz [Mon, 29 Dec 2014 20:10:30 +0000 (15:10 -0500)]
3198: Fix syntax errors
Peter Amstutz [Mon, 29 Dec 2014 19:37:13 +0000 (14:37 -0500)]
4869: Strip all newlines from error responses, not just leading and trailing
whitespace.
Tom Clegg [Mon, 29 Dec 2014 18:58:58 +0000 (13:58 -0500)]
4523: Dry up migration and test cases.
Peter Amstutz [Mon, 29 Dec 2014 18:51:20 +0000 (13:51 -0500)]
4869: Based on Go documentation, don't set a body ReadCloser on the request
when body length is 0.
Tom Clegg [Mon, 29 Dec 2014 17:45:02 +0000 (12:45 -0500)]
4523: Fix column order to match migration order.
Tom Clegg [Mon, 29 Dec 2014 17:44:35 +0000 (12:44 -0500)]
4523: Remove dev-only checks in migration.
Tom Clegg [Mon, 29 Dec 2014 17:35:29 +0000 (12:35 -0500)]
4523: Fix down-migration.
Peter Amstutz [Mon, 29 Dec 2014 17:32:38 +0000 (12:32 -0500)]
4869: Correctly handle zero-length blocks in Keep client/Keep proxy. Remove
X-Block-Size. Choose default request timeout based on if client is talking to
a proxy or not. Use double quotes in logging. Rename "tag" to "requestId".
Tom Clegg [Mon, 29 Dec 2014 17:28:44 +0000 (12:28 -0500)]
4523: Fix whitespace.
Tom Clegg [Mon, 29 Dec 2014 17:27:26 +0000 (12:27 -0500)]
4523: Add "store long string" test.
Peter Amstutz [Mon, 29 Dec 2014 14:23:45 +0000 (09:23 -0500)]
4869: Keepstore now returns Content-Length headers, and logs the error message
sent to the client on errors.
Peter Amstutz [Mon, 29 Dec 2014 14:09:13 +0000 (09:09 -0500)]
4869: KeepClient now has a default timeout per block request (10 minutes). In
keepproxy, the timeout is set to 20 seconds per block. Also rearranged some
keepclient and keepproxy logging to provide better information.
Tom Clegg [Tue, 23 Dec 2014 20:51:49 +0000 (15:51 -0500)]
Merge branch '4754-performance-TC' closes #4754
Ward Vandewege [Tue, 23 Dec 2014 20:47:49 +0000 (15:47 -0500)]
Merge branch '4844-stricter-min-nodes-wip'
refs #4844
Ward Vandewege [Tue, 23 Dec 2014 20:47:23 +0000 (15:47 -0500)]
Merge branch 'master' into 4844-stricter-min-nodes-wip
Ward Vandewege [Tue, 23 Dec 2014 20:44:10 +0000 (15:44 -0500)]
Skip two more CLI tests that need a running API server.
refs #4156
Peter Amstutz [Tue, 23 Dec 2014 14:55:05 +0000 (09:55 -0500)]
4869: Improve logging
Peter Amstutz [Tue, 23 Dec 2014 03:39:42 +0000 (22:39 -0500)]
3198: Async put, read prefetch via BlockManager. Added arvfile tests (forgot
to add before). No testing yet.
Peter Amstutz [Mon, 22 Dec 2014 15:39:10 +0000 (10:39 -0500)]
3198: Populate collection with locator or manifest. Also added Python container API.
Tom Clegg [Sun, 21 Dec 2014 00:28:56 +0000 (19:28 -0500)]
4875: Let the OS choose port numbers for fake servers.
Fixes a race condition where test case N+1 can't listen on port 2990
because test case N hasn't shut down its listener.
Also removes the artificial acceptance requirement that nobody else on
the testing host is using the arbitrarily assigned port range
2990..299x.
Incidental changes:
* rename RunBogusKeepServer to RunFakeKeepServer (to match
RunSomeFakeKeepServers and fix the misleading implication that the
resulting server does something bogus).
* return a KeepServer object from RunFakeKeepServer (for better parity
with RunSomeFakeKeepServers).
Tom Clegg [Sat, 20 Dec 2014 23:49:53 +0000 (18:49 -0500)]
4875: Use range in for loops.
Phil Hodgson [Sat, 20 Dec 2014 18:34:39 +0000 (19:34 +0100)]
Merge branch '4858-graph-not-comparing' refs #4358
Phil Hodgson [Sat, 20 Dec 2014 17:58:03 +0000 (18:58 +0100)]
Merge branch 'master' into 4358-graph-not-comparing
Peter Amstutz [Fri, 19 Dec 2014 21:36:46 +0000 (16:36 -0500)]
3198: Writing files works
Peter Amstutz [Fri, 19 Dec 2014 21:07:30 +0000 (16:07 -0500)]
3198: More refactoring and bug/test fixing
Brett Smith [Fri, 19 Dec 2014 17:09:17 +0000 (12:09 -0500)]
4844: Node Manager doesn't treat min_nodes as min_nodes_idle.
There's a bad interaction between the past bugfixes to (a) implement
min_nodes, and (b) boot new nodes when existing nodes are busy.
Because min_nodes has been implemented at the server wishlist level in
the past, the daemon can't distinguish between "nodes requested to
fulfill min_nodes" and "nodes requested to fulfill jobs."
This commit puts all the responsibility for enforcing min_nodes in the
daemon, so that the server wishlist always represents real job
requirements. This lets the daemon correctly decide whether or not to
boot a new node when >= min_nodes are busy.
Peter Amstutz [Fri, 19 Dec 2014 14:30:26 +0000 (09:30 -0500)]
3198: Fixing tests broken by previous commit's refactoring. Creating new
ArvadosFile/ArvadosFileReader/ArvadosFileWriter classes.
Peter Amstutz [Fri, 19 Dec 2014 03:43:23 +0000 (22:43 -0500)]
3198: Refactoring. Added Range object instead of fiddling with arrays, should
make the code easier to comprehend.
Peter Amstutz [Thu, 18 Dec 2014 22:12:43 +0000 (17:12 -0500)]
3198: New Collection/AravosFile design work in progress
Brett Smith [Thu, 18 Dec 2014 21:20:15 +0000 (16:20 -0500)]
Merge branch '4670-node-manager-robust-tags-wip'
Closes #4670, #4812.
Brett Smith [Fri, 12 Dec 2014 21:16:39 +0000 (16:16 -0500)]
4670: Add a post-create hook to Node Manager for EC2 tagging.
The previous code was relying on the post-create tagging in libcloud's
EC2 driver. Unfortunately, that's not working out too well for us: if
it fails, you get no indication of that, and it doesn't get retried.
This moves the work up into Node Manager, where failures can be logged
and retried appropriately.
The retry support may be sufficient to resolve #4670. If it's not,
then the additional logging will help us track down the root cause.
Brett Smith [Fri, 12 Dec 2014 18:18:51 +0000 (13:18 -0500)]
4670: Node Manager handles more libcloud exceptions.
libcloud compute drivers (at least EC2 and GCE) raise bare Exceptions
when there's some problem talking to the cloud service. The previous
code was expecting to see a LibcloudError, so it wouldn't handle these
errors as intended.
I didn't want to just catch errors with "except Exception" everywhere,
so I added an is_cloud_exception class method to our driver classes to
more accurately identify exceptions that represent trouble talking to
the cloud service. It recognizes exact Exceptions, plus the other
classes we were catching before.
While I was at this, I gave more specific names to the wrapper methods
in compute node actor decorators, as a debugging aid.
Peter Amstutz [Thu, 18 Dec 2014 16:26:04 +0000 (11:26 -0500)]
3198: Support repacking buffer blocks when writes are superceded.
Brett Smith [Thu, 18 Dec 2014 16:04:23 +0000 (11:04 -0500)]
4800: run-command calls sys.exit() with an integer.
Closes #4800.