• <small id='5c0aN'></small><noframes id='5c0aN'>

    • <bdo id='5c0aN'></bdo><ul id='5c0aN'></ul>
    <legend id='5c0aN'><style id='5c0aN'><dir id='5c0aN'><q id='5c0aN'></q></dir></style></legend>

        <tfoot id='5c0aN'></tfoot>
      1. <i id='5c0aN'><tr id='5c0aN'><dt id='5c0aN'><q id='5c0aN'><span id='5c0aN'><b id='5c0aN'><form id='5c0aN'><ins id='5c0aN'></ins><ul id='5c0aN'></ul><sub id='5c0aN'></sub></form><legend id='5c0aN'></legend><bdo id='5c0aN'><pre id='5c0aN'><center id='5c0aN'></center></pre></bdo></b><th id='5c0aN'></th></span></q></dt></tr></i><div id='5c0aN'><tfoot id='5c0aN'></tfoot><dl id='5c0aN'><fieldset id='5c0aN'></fieldset></dl></div>

      2. 带有 maxtasksperchild 的 multiprocessing.Pool 产生相等的

        时间:2023-08-06

        <small id='yriLZ'></small><noframes id='yriLZ'>

            1. <legend id='yriLZ'><style id='yriLZ'><dir id='yriLZ'><q id='yriLZ'></q></dir></style></legend>
            2. <i id='yriLZ'><tr id='yriLZ'><dt id='yriLZ'><q id='yriLZ'><span id='yriLZ'><b id='yriLZ'><form id='yriLZ'><ins id='yriLZ'></ins><ul id='yriLZ'></ul><sub id='yriLZ'></sub></form><legend id='yriLZ'></legend><bdo id='yriLZ'><pre id='yriLZ'><center id='yriLZ'></center></pre></bdo></b><th id='yriLZ'></th></span></q></dt></tr></i><div id='yriLZ'><tfoot id='yriLZ'></tfoot><dl id='yriLZ'><fieldset id='yriLZ'></fieldset></dl></div>
                <tbody id='yriLZ'></tbody>
              <tfoot id='yriLZ'></tfoot>

                  <bdo id='yriLZ'></bdo><ul id='yriLZ'></ul>
                • 本文介绍了带有 maxtasksperchild 的 multiprocessing.Pool 产生相等的 PID的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

                  问题描述

                  我需要在与所有其他内存完全隔离的进程中多次运行一个函数.我想为此使用 multiprocessing (因为我需要序列化来自函数的复杂输出).我将 start_method 设置为 'spawn' 并使用带有 maxtasksperchild=1 的池.我希望为每个任务获得不同的进程,因此会看到不同的 PID:

                  I need to run a function in a process, which is completely isolated from all other memory, several times. I would like to use multiprocessing for that (since I need to serialize a complex output coming from the functions). I set the start_method to 'spawn' and use a pool with maxtasksperchild=1. I would expect to get a different process for each task, and therefore see a different PID:

                  import multiprocessing
                  import time
                  import os
                  
                  def f(x):
                      print("PID: %d" % os.getpid())
                      time.sleep(x)
                      complex_obj = 5 #more complex axtually
                      return complex_obj
                  
                  if __name__ == '__main__':
                      multiprocessing.set_start_method('spawn')
                      pool = multiprocessing.Pool(4, maxtasksperchild=1)
                      pool.map(f, [5]*30)
                      pool.close()
                  

                  但是我得到的输出是:

                  $ python untitled1.py 
                  PID: 30010
                  PID: 30009
                  PID: 30012
                  PID: 30011
                  PID: 30010
                  PID: 30009
                  PID: 30012
                  PID: 30011
                  PID: 30018
                  PID: 30017
                  PID: 30019
                  PID: 30020
                  PID: 30018
                  PID: 30019
                  PID: 30017
                  PID: 30020
                  ...
                  

                  因此,不会在每个任务之后重新生成进程.是否有一种每次都自动获取新 PID 的方法(即无需为每组进程启动一个新池)?

                  So the processes are not being respawned after every task. Is there an automatic way of getting a new PID each time (ie without starting a new pool for each set of processes)?

                  推荐答案

                  您还需要在调用 pool.map 时指定 chunksize=1.否则,从工作进程的感知来看,您的可迭代项中的多个项目会被捆绑到一个任务"中:

                  You need to also specify chunksize=1 in the call to pool.map. Otherwise, multiple items in your iterable get bundled together into one "task" from the perception of the worker processes:

                  import multiprocessing
                  import time
                  import os
                  
                  def f(x):
                      print("PID: %d" % os.getpid())
                      time.sleep(x)
                      complex_obj = 5 #more complex axtually
                      return complex_obj
                  
                  if __name__ == '__main__':
                      multiprocessing.set_start_method('spawn')
                      pool = multiprocessing.Pool(4, maxtasksperchild=1)
                      pool.map(f, [5]*30, chunksize=1)
                      pool.close()
                  

                  现在输出没有重复的 PID:

                  Output doesn't have repeated PIDs now:

                  PID: 4912
                  PID: 4913
                  PID: 4914
                  PID: 4915
                  PID: 4938
                  PID: 4937
                  PID: 4940
                  PID: 4939
                  PID: 4966
                  PID: 4965
                  PID: 4970
                  PID: 4971
                  PID: 4991
                  PID: 4990
                  PID: 4992
                  PID: 4993
                  PID: 5013
                  PID: 5014
                  PID: 5012
                  

                  这篇关于带有 maxtasksperchild 的 multiprocessing.Pool 产生相等的 PID的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持html5模板网!

                  上一篇:Python多处理导致许多僵尸进程 下一篇:EOFError:类内的输入用完

                  相关文章

                  最新文章

                  1. <tfoot id='7EyL0'></tfoot>
                      <bdo id='7EyL0'></bdo><ul id='7EyL0'></ul>

                    <legend id='7EyL0'><style id='7EyL0'><dir id='7EyL0'><q id='7EyL0'></q></dir></style></legend>

                    <small id='7EyL0'></small><noframes id='7EyL0'>

                  2. <i id='7EyL0'><tr id='7EyL0'><dt id='7EyL0'><q id='7EyL0'><span id='7EyL0'><b id='7EyL0'><form id='7EyL0'><ins id='7EyL0'></ins><ul id='7EyL0'></ul><sub id='7EyL0'></sub></form><legend id='7EyL0'></legend><bdo id='7EyL0'><pre id='7EyL0'><center id='7EyL0'></center></pre></bdo></b><th id='7EyL0'></th></span></q></dt></tr></i><div id='7EyL0'><tfoot id='7EyL0'></tfoot><dl id='7EyL0'><fieldset id='7EyL0'></fieldset></dl></div>