---
layout: default
navsection: architecture
title: Computing with Crunch
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.

SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}

Crunch is the name for the Arvados system for managing computation.  It provides an abstract API to various clouds and HPC resource allocation and scheduling systems, and integrates closely with Keep storage and the Arvados permission system.

h2. Container API

# To submit work, create a "container request":{{site.baseurl}}/api/methods/container_requests.html in the @Committed@ state.
# The system will fufill the container request by creating or reusing a "Container object":{{site.baseurl}}/api/methods/containers.html and assigning it to the @container_uuid@ field.  If the same request has been submitted in the past, it may reuse an existing container.  The reuse behavior can be suppressed with @use_existing: false@ in the container request.
# The dispatcher process will notice a new container in @Queued@ state and submit a container executor to the underlying work queuing system (such as SLURM).
# The container executes.  Upon termination the container goes into the  @Complete@ state.  If the container execution was interrupted or lost due to system failure, it will go into the @Cancelled@ state.
# When the container associated with the container request is completed, the container request will go into the @Final@ state.
# The @output_uuid@ field of the container request contains the uuid of output collection produced by container request.

!(full-width){{site.baseurl}}/images/Crunch_dispatch.svg!

h2. Requesting RAM for containers

The @runtime_constraints@ section of a container specifies working RAM (@ram@) and keep cache (@keep_cache_ram@).  If not specified, containers get a default keep cache (@container_default_keep_cache_ram@, default 256 MiB).  The total RAM requested for a container is the sum of working RAM, keep cache, and an additional RAM reservation configured by the admin (@ReserveExtraRAM@ in the dispatcher configuration, default zero).

The total RAM request is used to schedule containers onto compute nodes.  RAM allocations are enforced using kernel controls such as cgroups.

When running on the cloud, the memory request (along with CPU and disk) is used to select (and possibly boot) an instance type with adequate resources to run the container.  Instance type RAM is derated 5% from the published specification to accomodate virtual machine, kernel and system services overhead.  This means, for example, a node with a published size of 4 GiB is assumed to only have 3891 MiB available for running containers.

h2. Job API (deprecated)

# To submit work, create a "job":{{site.baseurl}}/api/methods/jobs.html .  If the same job has been submitted in the past, it will return an existing job in @Completed@ state.
# The dispatcher process will notice a new job in @Queued@ state and attempt to allocate nodes to run the job.
# The job executes.
# Retrieve the @output@ field with the portable data hash of the collection with the output files of the job.