]> git.arvados.org - arvados.git/commit
10932: Replaced the use of a list with a set to check if files on local collection...
authorLucas Di Pentima <lucas@curoverse.com>
Thu, 2 Feb 2017 22:10:45 +0000 (19:10 -0300)
committerLucas Di Pentima <lucas@curoverse.com>
Thu, 2 Feb 2017 22:10:45 +0000 (19:10 -0300)
commit9c9b52038aa8b9c15f02567d186539fd8794d0f2
tree7d4390ed70fdd32e3b74d5620900856590dad6d7
parent519ab4c83aeb44ce91da941ecc191d00b6c6c72b
10932: Replaced the use of a list with a set to check if files on local collection are on the local file list, so that the resume start time is greatly reduced.
Also, the save_state method was taking too much time on two operations: deepcopy() and json.dump(). Replaced both with just one call to json.dumps() that's a lot faster than json.dump().
This will improve overall performance on big file collections uploads.
sdk/python/arvados/commands/put.py