Why does kilosort import take so long?

[agbondy]

It seems as though JRClust does quite a lot of work when you ask it to import-kilosort. I had assumed that all it had to do was plot the kilosort sorts, maybe also add a few specific quantities that JRClust expects but Kilosort doesn't provide. But it seems like JRClust really does quite a lot of post-processing on the Kilosort output. It takes about 1/3 of the time to import a kilosort sort than it does to do its own sort from start to finish. Can you clarify what it is doing?

Thanks!
Adrian

[aliddell]

JRCLUST was originally built to depend on extracted raw and filtered traces and features living in separate files next to the sorting results. This means that if one wants to plot results from Kilosort (or any other algorithm) JRCLUST has to first go through and extract them and compute those features (in the case of an import, the features are only really used as weights in computing spike positions, so this is somewhat wasteful, but it's a small proportion of the time already spent filtering and extracting).

This is all to say that the import could definitely be sped up, but I don't quite have the bandwidth for it at the moment. If anyone is interested in helping out I'm more than happy to provide orientation and answer questions.