Tom Clegg [Thu, 13 Jan 2022 19:47:46 +0000 (14:47 -0500)]
Fix deadlock at container finish.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 13 Jan 2022 19:47:40 +0000 (14:47 -0500)]
Add -chunked-onehot option.
refs #18581
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 7 Jan 2022 16:20:06 +0000 (11:20 -0500)]
Handle fasta with no line breaks.
fixes #18619
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 7 Jan 2022 14:20:29 +0000 (09:20 -0500)]
Filter HGVS columns by coverage & p-value threshold.
refs #18438
refs #18495
refs #18581
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 30 Dec 2021 16:24:44 +0000 (11:24 -0500)]
Fix coordinates in hgvs annotations.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 30 Dec 2021 15:20:30 +0000 (10:20 -0500)]
Container-watching fixes.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 30 Dec 2021 00:19:28 +0000 (19:19 -0500)]
Fix slice padding.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 30 Dec 2021 00:19:26 +0000 (19:19 -0500)]
Fix blocking on gob encode.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 29 Dec 2021 18:40:15 +0000 (13:40 -0500)]
More aggressive GC.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 29 Dec 2021 14:33:09 +0000 (09:33 -0500)]
Fix index out of bounds error.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 28 Dec 2021 21:45:39 +0000 (16:45 -0500)]
Refactor chunked-hgvs to use less memory.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 27 Dec 2021 18:01:24 +0000 (13:01 -0500)]
Separate options to output single/chunked hgvs matrices.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 21 Dec 2021 15:05:27 +0000 (10:05 -0500)]
Bump slice-numpy memory.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 13 Dec 2021 15:29:56 +0000 (10:29 -0500)]
Include HGVS IDs in anno2vcf.
refs #18579
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 7 Dec 2021 16:01:19 +0000 (11:01 -0500)]
Use (throttle)Go() convenience.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sat, 4 Dec 2021 06:40:19 +0000 (01:40 -0500)]
Call 2-base deletion-insertion as two adjacent SNPs.
refs #18496
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 2 Dec 2021 18:39:36 +0000 (13:39 -0500)]
Add null csv rows for ref/undiffed. Ensure 0 means ref in matrix.
refs #18496
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 30 Nov 2021 21:17:09 +0000 (16:17 -0500)]
Fix handling of TAG->CA (spell as T>C, =A, delG).
fixes #18496
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 30 Nov 2021 21:01:44 +0000 (16:01 -0500)]
Not a bug when ref is the only variant loaded.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 30 Nov 2021 21:01:19 +0000 (16:01 -0500)]
Fix deadlock in skip-on-error case.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 30 Nov 2021 19:59:45 +0000 (14:59 -0500)]
Call SNPs separately when called within 1bp of start/end of indels.
fixes #18496
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sat, 27 Nov 2021 03:15:56 +0000 (22:15 -0500)]
Mention undiffable variants in annotations, write -2 in hgvs matrix.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 26 Nov 2021 21:24:31 +0000 (16:24 -0500)]
Write hgvs-oriented matrix and annotations csv.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 26 Nov 2021 15:33:32 +0000 (10:33 -0500)]
Log exact locations of duplicate tags.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 25 Nov 2021 20:13:39 +0000 (15:13 -0500)]
Drop duplicate tags, fix ref position tracking.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 25 Nov 2021 19:41:44 +0000 (14:41 -0500)]
Fix interval tree unused node markers.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 25 Nov 2021 19:41:01 +0000 (14:41 -0500)]
More debug messages and error checks.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 24 Nov 2021 20:19:27 +0000 (15:19 -0500)]
Use non-preemptible instances.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 24 Nov 2021 20:19:19 +0000 (15:19 -0500)]
Option to merge matrices and annotations.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 22 Nov 2021 15:20:52 +0000 (10:20 -0500)]
Implement -regions and -expand-regions for slice-numpy.
refs #18438
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 19 Nov 2021 18:23:16 +0000 (13:23 -0500)]
Adjust slice memory.
refs #18414
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 4 Nov 2021 18:11:42 +0000 (14:11 -0400)]
Write deletions as "TAA T" instead of "AA .".
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 4 Nov 2021 18:10:14 +0000 (14:10 -0400)]
Fix variant numbering.
Renumbering code was incorrectly reserving ranking spots for no-calls.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 1 Nov 2021 14:41:01 +0000 (10:41 -0400)]
Update slice-numpy test.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 1 Nov 2021 14:27:04 +0000 (10:27 -0400)]
Fix duplicate entries in slice-numpy annotations.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 1 Nov 2021 14:00:38 +0000 (10:00 -0400)]
Avoid empty "ref" field in anno2vcf output.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 1 Nov 2021 13:31:04 +0000 (09:31 -0400)]
Split anno2vcf output by chromosome.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 29 Oct 2021 18:39:42 +0000 (14:39 -0400)]
Fix sort func.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 28 Oct 2021 15:28:32 +0000 (11:28 -0400)]
Bump slice-numpy memory.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 28 Oct 2021 15:28:04 +0000 (11:28 -0400)]
Sort input files.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 28 Oct 2021 14:33:43 +0000 (10:33 -0400)]
Add anno2vcf command.
refs #17763
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 8 Oct 2021 02:12:06 +0000 (22:12 -0400)]
Accept PDH on command line.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 6 Oct 2021 20:07:40 +0000 (16:07 -0400)]
Use arvados client for /any/path/$id/, not just /mnt/$id/.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 29 Sep 2021 18:47:34 +0000 (14:47 -0400)]
Implement -match-genome filter.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 29 Sep 2021 18:46:10 +0000 (14:46 -0400)]
Fix namespace types.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sun, 26 Sep 2021 02:47:00 +0000 (22:47 -0400)]
Save all ref tile data in slice 0.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sun, 26 Sep 2021 01:56:08 +0000 (21:56 -0400)]
Cleanup var name.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sat, 25 Sep 2021 13:44:22 +0000 (09:44 -0400)]
Scale keep cache to 2*(openfiles+1).
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 24 Sep 2021 18:09:16 +0000 (14:09 -0400)]
Improve logging.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 24 Sep 2021 18:08:59 +0000 (14:08 -0400)]
Tweak cpu/mem usage.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 23 Sep 2021 19:52:42 +0000 (15:52 -0400)]
Accept multiple input libraries for slice→slicenumpy.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 21 Sep 2021 13:59:03 +0000 (09:59 -0400)]
Log total variant/genome/ref counts.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 20 Sep 2021 13:20:30 +0000 (09:20 -0400)]
Fix array index out of bounds.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 17 Sep 2021 01:50:45 +0000 (21:50 -0400)]
Renumber variants by allele count.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 17 Sep 2021 00:33:38 +0000 (20:33 -0400)]
Fix reference assembly.
refs #17966
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 14 Sep 2021 20:15:35 +0000 (16:15 -0400)]
Fix mem usage, improve logging.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 13 Sep 2021 15:23:23 +0000 (11:23 -0400)]
Generate annotations for slices.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 13 Sep 2021 14:01:49 +0000 (10:01 -0400)]
Generate numpy matrices from slices.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sun, 12 Sep 2021 19:24:45 +0000 (15:24 -0400)]
Always log version at startup.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Sun, 12 Sep 2021 19:24:20 +0000 (15:24 -0400)]
Slice imported data by tag#.
refs #17996
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 9 Sep 2021 15:28:08 +0000 (11:28 -0400)]
Use callbacks in struct instead of args to Load*().
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 1 Sep 2021 17:44:57 +0000 (13:44 -0400)]
Fix missed TranslatePaths and unreported error.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 31 Aug 2021 18:56:04 +0000 (14:56 -0400)]
Add -p-value and -cases options for exporting hgvs numpy.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 31 Aug 2021 18:55:56 +0000 (14:55 -0400)]
Add chi square func.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 12 Aug 2021 13:56:50 +0000 (09:56 -0400)]
Fix unchecked error.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 12 Aug 2021 03:16:36 +0000 (23:16 -0400)]
Fix array index out of bounds.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 11 Aug 2021 22:10:26 +0000 (18:10 -0400)]
Add -match-genome=regexp filter.
refs #17939
refs #17922
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Wed, 11 Aug 2021 21:31:23 +0000 (17:31 -0400)]
Export hgvs one-hot numpy: -1 for missing / low quality tiles.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 10 Aug 2021 19:26:37 +0000 (15:26 -0400)]
Add filter options to export cmd.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 10 Aug 2021 13:18:10 +0000 (09:18 -0400)]
Add .licenseignore.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 10 Aug 2021 13:16:58 +0000 (09:16 -0400)]
Memory tweaks.
refs #17562
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 6 Aug 2021 16:49:21 +0000 (12:49 -0400)]
Add copyright headers.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 6 Aug 2021 16:11:32 +0000 (12:11 -0400)]
Reduce memory use, Limit goroutines when exporting numpy.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 3 Aug 2021 17:18:14 +0000 (13:18 -0400)]
Reduce lock contention.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 3 Aug 2021 15:27:08 +0000 (11:27 -0400)]
Write annotations through to conserve memory.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 2 Aug 2021 19:09:51 +0000 (15:09 -0400)]
Export hgvs one-hot numpy.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 30 Jul 2021 13:20:00 +0000 (09:20 -0400)]
Use tsv filename when using tab separator.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 29 Jul 2021 20:49:24 +0000 (16:49 -0400)]
Track which tile variants each hgvs.Variant appeared in.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 15 Jul 2021 15:07:08 +0000 (11:07 -0400)]
Skip diffs on long ref seqs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 13 Jul 2021 19:37:38 +0000 (15:37 -0400)]
Fix some entries skipped by WriteDir.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 13 Jul 2021 15:25:00 +0000 (11:25 -0400)]
Compress export output.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Tue, 13 Jul 2021 14:09:16 +0000 (10:09 -0400)]
Fix up VCF format.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 12 Jul 2021 16:07:31 +0000 (12:07 -0400)]
Fix recomputing diffs N times.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Mon, 12 Jul 2021 14:20:11 +0000 (10:20 -0400)]
Separate pvcf/vcf output.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 9 Jul 2021 17:04:42 +0000 (13:04 -0400)]
Progress indicator for exportSeq.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 9 Jul 2021 14:32:46 +0000 (10:32 -0400)]
Improve concurrency in export-diff.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 9 Jul 2021 13:13:03 +0000 (09:13 -0400)]
Include tagset in reference-genome files.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 23:06:07 +0000 (19:06 -0400)]
Fix missing output.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 20:48:14 +0000 (16:48 -0400)]
Write ref seqs to their own files.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 20:48:04 +0000 (16:48 -0400)]
Use more cpus.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 20:47:35 +0000 (16:47 -0400)]
Fix use of nil writer.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 15:25:43 +0000 (11:25 -0400)]
Enable network access to get port forwarding.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 8 Jul 2021 14:24:44 +0000 (10:24 -0400)]
Option to export one vcf/csv file per chromosome.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Fri, 18 Jun 2021 13:26:31 +0000 (09:26 -0400)]
Fix race.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 21:17:26 +0000 (17:17 -0400)]
Improve concurrency more.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 20:14:48 +0000 (16:14 -0400)]
Improve loading concurrency.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 19:23:37 +0000 (15:23 -0400)]
Split sequence data into 64K independently locked partitions.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 18:37:57 +0000 (14:37 -0400)]
Fix exportnumpy test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 15:25:56 +0000 (11:25 -0400)]
Add concurrent load to exportnumpy.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
Tom Clegg [Thu, 17 Jun 2021 15:25:44 +0000 (11:25 -0400)]
Skip expensive Tracef.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curii.com>
projects / lightning.git / log