21943: Fix bug and add integration test

21943: Fix bug and add integration test

The bug was caused by the fact that the path mapper de-duplicates file
references before doing the mapping, but also did not traverse
directory listings.  If the list of files was de-duplicated by
selecting the entries from the directory listing (given a choice of 2+
places a given file reference appears in output), but then the
directory listings were skipped later on, those files wouldn't be
included in the path map at all.

Now trims the directory listings while constructing the list of files
to path map, so if the file is referenced in directory listings in the
output, it isn't lost to the dedup behavior.

Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curii.com>