h3. Logs table purpose & behavior
-This database table currently serves three purposes:
-* It's an audit log, permitting admins and users to look up the time and details of past changes to Arvados objects via @arvados.v1.logs.*@ endpoints.
-* It's a mechanism for passing cache-invalidation events, used by websocket servers, the Python SDK "events" library, and @arvados-cwl-runner@ to detect when an object has changed.
-* It's a staging area for stdout/stderr text coming from users' containers, permitting users to see what their containers are doing while they are still running (i.e., before those text files are written to Keep).
+This database table is accessed viad the via "the @logs@ endpoint.":../api/methods/logs.html
+
+This table currently serves several purposes:
+
+* Audit logging, permitting admins and users to look up the time and details of past changes to Arvados objects.
+* Logging other system events, specifically "file uploads and downloads from keep-web.":restricting-upload-download.html#audit_logs
+* The source for cache-invalidation events, published through websockets to Workbench to refresh the view. It can also be monitored by the Python SDK "events module.":../sdk/python/events.html
+* Prior to Arvados 2.7, it was used a staging area for stdout/stderr text coming from users' containers, permitting users to see what their containers are doing while they are still running (i.e., before those text files are written to Keep). Starting with Arvados 2.7, this is superceded by a more efficient mechanism and so these logs are disabled by default. See "2.7.0 upgrade notes":upgrading.html#v2_7_0 for details.
As a result, this table grows indefinitely, even on sites where policy does not require an audit log; making backups, migrations, and upgrades unnecessarily slow and painful.
</pre>
-h2. Accessing the audit log
+h2(#audit_log). Accessing the audit log
When @WebDAVLogEvents@ is enabled, uploads and downloads of files are logged in the Arvados audit log. These events are included in the "User Activity Report":user-activity.html. The audit log can also be accessed via the API, SDKs or command line. For example, to show the 100 most recent file downloads:
Jul 20 15:03:38 workbench.xxxx1.arvadosapi.com keepproxy[63828]: {"level":"info","locator":"abcdefghijklmnopqrstuvwxyz012345+53251584","msg":"Block download","time":"2021-07-20T15:03:38.458792300Z","user_full_name":"Albert User","user_uuid":"ce8i5-tpzed-abcdefghijklmno"}
</pre>
-It is possible to do a reverse lookup from the locator to find all matching collections: the @manifest_text@ field of a collection lists all the block locators that are part of the collection. The @manifest_text@ field also provides the relevant filename in the collection. Because this lookup is rather involved and there is no automated tool to do it, we recommend disabling @KeepproxyPermission/User/Download@ and @KeepproxyPermission/User/Upload@ for sites where the audit log is important and @arv-get@ and @arv-put@ are not essential.
+It is possible to do a reverse lookup from the locator to find all matching collections: the @manifest_text@ field of a collection lists all the block locators that are part of the collection. The @manifest_text@ field also provides the relevant filename in the collection. Because this lookup is rather involved and there is no automated tool to do it, we recommend disabling @KeepproxyPermission.User.Download@ and @KeepproxyPermission.User.Upload@ for sites where the audit log is important and @arv-get@ and @arv-put@ are not essential.
"previous: Upgrading to 2.6.3":#v2_6_3
+h3. New system for live container logs
+
+Starting with Arvados 2.7, a new system for fetching live container logs is in place. This system features significantly reduced database load compared to previous releases. When Workbench or another application need to access the logs of a process (running or completed), they should use the "log endpoint of container_requests.":https://doc.arvados.org/main/api/methods/container_requests.html which forwards requests to the running container. This supercedes the previous system where compute processes would send all of their logs to the database, which produced significant database load.
+
+The legacy logging system is now disabled by default for all installations with the setting @Containers.Logging.LimitLogBytesForJob: 0@. If you have an existing Arvados installation where you have customized this value and do not need the legacy container logging system, we recommend removing @LimitLogBytesForJob@ from your configuration.
+
+If you need to re-enable the legacy logging system, set @Containers.Logging.LimitLogBytesForJob@ to a positive value (the previous default was @Containers.Logging.LimitLogBytesForJob: 67108864@).
+
h3. Workbench 1 deprecated
The original Arvados Workbench application (referred to as "Workbench 1") is deprecated and will be removed in a future major version of Arvados. Users are advised to migrate to "Workbench 2". Starting with this release, new installations of Arvados will only set up Workbench 2 and no longer include Workbench 1 by default.
+It is also important to note that Workbench 1 only supports the legacy logging system, which is now disabled by default. If you need to re-enable the legacy logging system, see above.
+
h3. Multi-node installer's domain name configuration changes
The @domain_name@ variable at @terraform/vpc/terraform.tfvars@ and @DOMAIN@ variable at @local.params@ changed their meaning. In previous versions they were used in combination with @cluster_name@ and @CLUSTER@ to build the cluster's domain name (e.g.: @cluster_name@.@domain_name@). To allow the use of any arbitrary cluster domain, now we don't enforce using the cluster prefix as part of the domain, so @domain_name@ and @DOMAIN@ need to hold the entire domain for the given cluster.
projects / arvados.git / commitdiff