20219: Replace API & websocket log loading with webdav polling
authorStephen Smith <stephen@curii.com>
Fri, 28 Jul 2023 15:18:20 +0000 (11:18 -0400)
committerStephen Smith <stephen@curii.com>
Fri, 28 Jul 2023 15:18:20 +0000 (11:18 -0400)
commitd76aa6ba31074f3d41cef48628b235aab4906ce5
treef139af881bf5450fce36fe74f5f34df039d3049e
parenteaf099867227a9d5198a638205bdcddde7c17965
20219: Replace API & websocket log loading with webdav polling

* Add apiWebdavClient to log service for making webdav requests to the API and
  differentiate from keepWebdavClient
* Add listLogFiles and getLogFileContents methods to log service, contents can
  be fetched with range request
* Add loadContainerLogFileContents to load logs using webdav range requests
  * Files larger than the chunk limit (128k) are loaded as beginning and end
    in 2 64k fragments, snipline is appended to the last line of the first
    fragment
  * The chunks are all requested in parallel
* Add sorting of timestamped log types in combined log views (Main/All)
  * Lines without timestamps are merged with previous lines with timestamps
    * If no preceding lines of a sortable type exist, the leading lines are
      removed from combined log views - they can still be seen in the single
      log type filter
  * Non-timestamped log types (node-info/contaner) are pushed to the bottom
    with original ordering in All view
  * Merging & sorting is not applied to single log type view
* Remove snipline LogEventType as all sniplines now belong to specific log
  event types
* Update logs panel store and reducer to remember the last log byte requested
* Add polling useAsyncInterval utility function to poll log file sizes
  * Polling awaits for callback to finish to prevent race conditions
* Update process log code snippet style to remove gap between array of loglines
* Remove websocket handling of log events
* Replaced cypress log manipulation commands with webdav log helpers
* Add tests for polling, line sorting, correctly sized/positioned chunks, and
  snipline

Arvados-DCO-1.1-Signed-off-by: Stephen Smith <stephen@curii.com>
16 files changed:
cypress/integration/process.spec.js
cypress/support/commands.js
src/common/use-async-interval.ts [new file with mode: 0644]
src/models/log.ts
src/services/collection-service/collection-service.test.ts
src/services/collection-service/collection-service.ts
src/services/log-service/log-service.ts
src/services/services.ts
src/store/process-logs-panel/process-logs-panel-actions.ts
src/store/process-logs-panel/process-logs-panel-reducer.ts
src/store/process-logs-panel/process-logs-panel.ts
src/views/process-panel/process-log-card.tsx
src/views/process-panel/process-log-code-snippet.tsx
src/views/process-panel/process-panel-root.tsx
src/views/process-panel/process-panel.tsx
src/websocket/websocket.ts