Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to pickle when using multiprocessing #12

Open
mobone opened this issue Jan 13, 2020 · 5 comments
Open

Unable to pickle when using multiprocessing #12

mobone opened this issue Jan 13, 2020 · 5 comments

Comments

@mobone
Copy link

mobone commented Jan 13, 2020

I'm using your example from the readme and while setting the engine to multiprocessing I receive the following error. I'm using python 3.8 64 bit.


Traceback (most recent call last):
  File "mp_test.py", line 23, in <module>
    hello(i+1)
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\site-packages\multitasking\__init__.py", line 119, in async_method
    single.start()
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 326, in _Popen
    return Popen(process_obj)
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'task.<locals>._run_via_pool'

C:\Users\nbrei\Documents\GitHub\pead_ml2>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\nbrei\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 102, in spawn_main
    source_process = _winapi.OpenProcess(
OSError: [WinError 87] The parameter is incorrect
@mobone
Copy link
Author

mobone commented Jan 13, 2020

It appears to work fine on my mac with python 3.7, traceback was on windows. I'll test on my windows machine with python 3.7 when I get home from work.

@mobone
Copy link
Author

mobone commented Jan 13, 2020

Nope, didn't solve it. Still getting the same traceback on windows.

@datatalking
Copy link

So if I understand the issue, you used the code from README.rst exactly as typed and you are getting errors. As it is currently written there is no pickle function, please include a copy of your code.

@wlxklyh
Copy link

wlxklyh commented Dec 3, 2022

I have try define _run_via_pool staticmethod and it fix. Actually,I am unfamiliar with python and I dont the reason
@staticmethod
def _run_via_pool(*args, **kwargs):

@mobone
Copy link
Author

mobone commented Mar 14, 2024

I just now clean installed Python 3.11.8 and am getting the same, parameter is incorrect. And again, it's literally copy and paste from the readme, and change from threads to process.

Edit: I just now had a friend try on his windows computer and he's getting the same exception.

# example.py
import multitasking
import time
import random
import signal

# kill all tasks on ctrl-c
signal.signal(signal.SIGINT, multitasking.killall)

multitasking.set_engine("process") # "process" or "thread"

# or, wait for task to finish on ctrl-c:
# signal.signal(signal.SIGINT, multitasking.wait_for_tasks)

@multitasking.task # <== this is all it takes :-)
def hello(count):
    sleep = random.randint(1,10)/2
    print("Hello %s (sleeping for %ss)" % (count, sleep))
    time.sleep(sleep)
    print("Goodbye %s (after for %ss)" % (count, sleep))

if __name__ == "__main__":
    for i in range(0, 10):
        hello(i+1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants