The onACID's parameter init_batch cannot exceed the number of frames of the first file?

[Heinrich2818]

Setup:

1. Operating System (Linux, MacOS, Windows): Windows 11
2. Hardware type (x86, ARM..) and RAM: x86
3. Python Version (e.g. 3.9): 3.10.8
4. Caiman version (e.g. 1.9.12):1.11.3
5. Which demo exhibits the problem (if applicable): demo_online_cnmfE.ipynb
6. How you installed Caiman (pure conda, conda + compile, colab, ..): following the README.md
7. Details:

I am working with a large miniscope dataset with multiple files (22 files approximately 80,000 frames). The batch processing is extremely slow, so I decided to use onACID processing instead. To make the onACID results more comparable to the batch processing results, I attempted to increase theinit_batch size.

However, I encountered following issue: If I use init_batch exceeds the number of frames in the first file, the following IndexError occurs:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[38], line 2
      1 # run online estimator
----> 2 cnm_online.fit_online();

File D:\WorkSoftware\Anaconda\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\online_cnmf.py:1237, in OnACID.fit_online(self, **kwargs)
   1235     frame_cor = frame_cor/self.img_norm
   1236 # Fit next frame
-> 1237 self.fit_next(t, frame_cor.reshape(-1, order='F'))
   1238 # Show
   1239 if self.params.get('online', 'show_movie'):

File D:\WorkSoftware\Anaconda\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\online_cnmf.py:442, in OnACID.fit_next(self, t, frame_in, num_iters_hals)
    440 C_in = self.estimates.noisyC[:self.M, t - 1].copy()
    441 if self.is1p:
--> 442     self.estimates.C_on[:self.M, t], self.estimates.noisyC[:self.M, t] = demix1p(
    443         frame, self.estimates.Ab, C_in, self.estimates.AtA, Atb=self.estimates.Atb,
    444         AtW=self.estimates.AtW, AtWA=self.estimates.AtWA, iters=num_iters_hals,
    445         groups=self.estimates.groups, ssub_B=ssub_B, 
    446         downscale_matrix=self.estimates.downscale_matrix if ssub_B > 1 else None)
    447 else:
    448     self.estimates.C_on[:self.M, t], self.estimates.noisyC[:self.M, t] = HALS4activity(
    449         frame, self.estimates.Ab, C_in, self.estimates.AtA, iters=num_iters_hals, groups=self.estimates.groups)

IndexError: index 23084 is out of bounds for axis 1 with size 23084

I wonder is it a limitation that the init_batch size cannot exceed the number of frames in the first file? And there alternative strategies or parameter adjustments that could make the onACID results more consistent with those from batch processing?

For reference, I typically run CaImAn on a 512GB server environment, but this time I am testing the demo_online_cnmfE.ipynb on a Windows machine. So I'm not sure if the same error will occur on Linux.

Any guidance or suggestions would be greatly appreciated!

[pgunn]

Hello,
We should document this, but I believe it is true.

[Heinrich2818]

Hello, We should document this, but I believe it is true.

Hello, thank you for your reply! In that case I think I will continue use patch processing + multisession register to process my data until I found a way to get better performance with online processing. For now, the init_batch is the parameter what I found make onACID results most comparable to the offline results. The others parameters like gSig, rf and stride are same with batch process. Maybe I can try add more epochs to get better online results?