wiki:Recipe/Multiprocessing
Last modified 6 months ago Last modified on 12/08/11 13:45:56

Required Changes to your code

When using the multiprocessing module, you must call

multiprocessing.freeze_support()

straight after the if __name__ == '__main__' line of the main module.

Please read the Python library manual about multiprocessing.freeze_support for more information.

Multiprocessing code fails when using a --onefile executable on Windows

NB: This problem is specific to Windows, which does not support spawn(). I does not occur on other (Posix) platforms like all flavors of Unix and Mac OS X.

For using python module multiprocess on Windows, you need to extend your multiprocessing code as shown below. See this thread about the background and ticket #182 for more information.

This recipe requires PyInstaller 1.6 or later (or trunk after 2011-10-16). If you are still using older versions of PyInstaller, you'll need an older version of the recipe.

import multiprocessing.forking
import os
import sys


class _Popen(multiprocessing.forking.Popen):
    def __init__(self, *args, **kw):
        if hasattr(sys, 'frozen'):
            # We have to set _MEIPASS2 to get
            # --onefile and --onedir mode working.
            os.putenv('_MEIPASS2', sys._MEIPASS) # last character is stripped in C-loader
        try:
            super(_Popen, self).__init__(*args, **kw)
        finally:
            if hasattr(sys, 'frozen'):
                os.unsetenv('_MEIPASS2')


class Process(multiprocessing.Process):
    _Popen = _Popen


class SendeventProcess(Process):
    def __init__(self, resultQueue):
        self.resultQueue = resultQueue

        multiprocessing.Process.__init__(self)
        self.start()

    def run(self):
        print 'SendeventProcess'
        self.resultQueue.put((1, 2))
        print 'SendeventProcess'


if __name__ == '__main__':
    multiprocessing.freeze_support()
    print 'main'
    resultQueue = multiprocessing.Queue()
    SendeventProcess(resultQueue)
    print 'main'
    if hasattr(sys, 'frozen'):
        # We need to wait for all child processes otherwise
        # --onefile mode won't work.
        while multiprocessing.active_children():
            multiprocessing.active_children()[0].join()

Console output of this code snippet should be similar to

main
main
SendeventProcess
SendeventProcess