Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix an error using cytotrace2_py #19

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Starlitnightly
Copy link

@Starlitnightly Starlitnightly commented Jun 26, 2024

We noticed that when using concurrent.futures.ProcessPoolExecutor, the program may raise an error during runtime, so we replaced it with concurrent.futures.ThreadPoolExecutor.

BrokenProcessPool Traceback (most recent call last)
Cell In[16], line 1
----> 1 results = cytotrace2(adata_control,
2 use_model_dir="../ov/cymodels/5_models_weights",
3 species="human")

File ~/software/rsc/lib/python3.10/site-packages/omicverse/single/_cytotrace2.py:188, in cytotrace2(adata, use_model_dir, species, batch_size, smooth_batch_size, disable_parallelization, max_cores, max_pcs, seed, output_dir)
183 results.append(executor.submit(process_subset, idx,
184 chunked_expression, smooth_batch_size,
185 smooth_cores_to_use, species,
186 use_model_dir, output_dir, max_pcs, seed))
187 for f in concurrent.futures.as_completed(results):
--> 188 smooth_by_knn_df = f.result()
189 predictions.append(smooth_by_knn_df)
191 for idx in range(chunk_number):

File ~/software/rsc/lib/python3.10/concurrent/futures/_base.py:451, in Future.result(self, timeout)
449 raise CancelledError()
450 elif self._state == FINISHED:
--> 451 return self.__get_result()
453 self._condition.wait(timeout)
455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File ~/software/rsc/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self)
401 if self._exception:
402 try:
--> 403 raise self._exception
404 finally:
405 # Break a reference cycle with the exception in self._exception
406 self = None

BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending

We added smooth_data_kNN.py to instead the smooth_data_kNN.R in cytotrace2_py.

    if chunked_expression.shape[0] < 100:
        print('cytotrace2: Fewer than 100 cells in dataset. Skipping KNN smoothing step.')
        smooth_by_knn_df = binned_score_pred_df.copy()
    else:
        #run_script = pkg_resources.resource_filename("cytotrace2_py","resources/smoothDatakNN.R")
        #knn_path = output_dir+'/smoothbykNNresult'+suffix+'.txt'
        #out = subprocess.run(['Rscript', run_script, '--output-dir', output_dir,
                               #'--suffix', suffix, '--max-pcs', str(max_pcs), '--seed', str(seed)], check=True)
        from .common.smoothDatakNN import smooth_data_kNN
        smooth_data_kNN(output_dir=output_dir, suffix=suffix, max_pcs=max_pcs, seed=seed)
        knn_path = os.path.join(output_dir, f'smoothbykNNresult{suffix}.txt')
        smooth_by_knn_df = pd.read_csv(knn_path, index_col=0, sep='\t')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant