Notes 2026/06/10: fragment ion indexing performance in Comet v2026.02.0

See notes/2024100_FI for the original Comet-FI notes.

All timings are run on a Dell Windows 11 computer with 64GB RAM and an Intel Core Ultra 7 265K processor.

The v2026.02.0 release of Comet implemented a number of performance optimizations to the fragment ion index code including adding threaded (concurrent) real-time search support through CometWrapper.dll. Here’s a summary of performance compared to the v2026.01.1 release:

Phospho search against human canonical target-decoy database (~40K proteins)

Parameters: trypsin, 2 missed cleavages, var mods M+16 / STY+80, 51,806 MS/MS spectra.

The .idx creation took 9 s and required 2.0 GB RAM.

20 threads (average 0.12 ms/query, max 31.1 GB RAM):

 Using 20 search threads

       query file: 20240924_Hela_02.raw
 Indexed database: human.canonical.target-decoy.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 26s
   - sort peptides by mass ... 9s
   - populate index ... 43s
   - 1.601e+08 total peptides, 4.742e+09 FI entries ... 1m:19s

 Progress: 96.2% (66500 MS2 scans of 90319 total scans)

 initialize elapsed time: 82.29 s
 MS2 search elapsed time: 6.93 s
 MS2 average search time: 0.12 ms/spectrum (66882 spectra), 8155 Hz
      total elapsed time: 94.46 s

 Done. (31.1GB)

8 threads (average 0.15 ms/query, max 29.9 GB RAM):

 Using 8 search threads

       query file: 20240924_Hela_02.raw
 Indexed database: human.canonical.target-decoy.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 26s
   - sort peptides by mass ... 9s
   - populate index ... 42s
   - 1.601e+08 total peptides, 4.742e+09 FI entries ... 1m:18s

 Progress: 96.2% (66500 MS2 scans of 90319 total scans)

 initialize elapsed time: 81.56 s
 MS2 search elapsed time: 8.92 s
 MS2 average search time: 0.15 ms/spectrum (66882 spectra), 6715 Hz
      total elapsed time: 96.29 s

 Done. (29.9GB)

4 threads (average 0.23 ms/query, max 29.3 GB RAM):

 Using 4 search threads

       query file: 20240924_Hela_02.raw
 Indexed database: human.canonical.target-decoy.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 26s
   - sort peptides by mass ... 9s
   - populate index ... 42s
   - 1.601e+08 total peptides, 4.742e+09 FI entries ... 1m:18s

 Progress: 96.2% (66500 MS2 scans of 90319 total scans)

 initialize elapsed time: 81.19 s
 MS2 search elapsed time: 15.18 s
 MS2 average search time: 0.23 ms/spectrum (66882 spectra), 4397 Hz
      total elapsed time: 103.48 s

 Done. (29.3GB)

MHC search against human canonical database (~20K proteins)

Parameters: cleave everywhere, no variable modifications, 56,292 MS/MS spectra.

The .idx creation took 155 s using 20 threads and required 44.1 GB RAM. As a comparison the corresponding .idx creation step for release v2026.01.1 took 311 s to generate.

20 threads (average 0.13 ms/query, max 40.9 GB RAM):

 Using 20 search threads

       query file: A375_Binimetinib_Exploris_1.raw
 Indexed database: human.canonical.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 18s
   - sort peptides by mass ... 4s
   - populate index ... 36s
   - 1.887e+08 total peptides, 4.549e+09 FI entries ... 59s

 Progress: 91.4% (55500 MS2 scans of 69571 total scans)

 initialize elapsed time: 77.02 s
 MS2 search elapsed time: 5.41 s
 MS2 average search time: 0.13 ms/spectrum (56292 spectra), 7714 Hz
      total elapsed time: 89.29 s

 Done. (40.9GB)

8 threads (average 0.17 ms/query, max 40.8 GB):

 Using 8 search threads

       query file: A375_Binimetinib_Exploris_1.raw
 Indexed database: human.canonical.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 17s
   - sort peptides by mass ... 4s
   - populate index ... 34s
   - 1.887e+08 total peptides, 4.549e+09 FI entries ... 56s

 Progress: 91.4% (55500 MS2 scans of 69571 total scans)

 initialize elapsed time: 74.90 s
 MS2 search elapsed time: 7.74 s
 MS2 average search time: 0.17 ms/spectrum (56292 spectra), 5924 Hz
      total elapsed time: 89.41 s

 Done. (40.8GB)

4 threads (average 0.28 ms/spectrum, max 40.6 GB RAM):

 Using 4 search threads

       query file: A375_Binimetinib_Exploris_1.raw
 Indexed database: human.canonical.fasta.idx

 - generate fragment ion index
   - store peptide list and reserve memory ... 17s
   - sort peptides by mass ... 4s
   - populate index ... 34s
   - 1.887e+08 total peptides, 4.549e+09 FI entries ... 56s

 Progress: 97.1% (56000 MS2 scans of 69571 total scans)

 initialize elapsed time: 73.26 s
 MS2 search elapsed time: 14.36 s
 MS2 average search time: 0.28 ms/spectrum (56292 spectra), 3554 Hz
      total elapsed time: 95.40 s

 Done. (40.6GB)