study/uTools-Manuals
1
0
Fork 0
mirror of https://github.com/fofolee/uTools-Manuals.git synced 2025-06-09 07:24:04 +08:00
Code Issues Packages Projects Releases Wiki Activity
uTools-Manuals/docs/python/multiprocessing.html
fofolee 38dcd51d8a v0.0.2
2019-04-21 11:50:48 +08:00

729 lines
350 KiB
HTML
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<div class="body" role="main"><div class="section" id="module-multiprocessing"><h1><span class="yiyi-st" id="yiyi-10">17.2. <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> — 基于进程的并行</span></h1><p><span class="yiyi-st" id="yiyi-11"><strong>源代码：</strong> <a class="reference external" href="https://hg.python.org/cpython/file/3.5/Lib/multiprocessing/">Lib/multiprocessing /</a></span></p><div class="section" id="introduction"><h2><span class="yiyi-st" id="yiyi-12">17.2.1. </span><span class="yiyi-st" id="yiyi-13">引言</span></h2><p><span class="yiyi-st" id="yiyi-14"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>是一个支持使用和<a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a>模块类似的API生成进程的模块。</span><span class="yiyi-st" id="yiyi-15"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>包提供本地和远程并发，通过使用子进程而不是线程有效地转移<a class="reference internal" href="../glossary.html#term-global-interpreter-lock"><span class="xref std std-term">全局解释器锁</span></a>。</span><span class="yiyi-st" id="yiyi-16">因此，<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>模块允许程序员充分利用给定机器上的多个处理器。</span><span class="yiyi-st" id="yiyi-17">它在Unix和Windows上都可以运行。</span></p><p><span class="yiyi-st" id="yiyi-18"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>模块还引入了<a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a>模块中没有的API。</span><span class="yiyi-st" id="yiyi-19">一个主要的例子是<a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>对象，它提供了一种方便的方法，可以跨多个输入值并行化函数的执行，跨进程分配输入数据（数据并行）。</span><span class="yiyi-st" id="yiyi-20">以下示例演示了在模块中定义此类函数的常见做法，以便子进程可以成功导入该模块。</span><span class="yiyi-st" id="yiyi-21">下面是使用<a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>进行数据并行的基本示例，</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Pool</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="k">with</span> <span class="n">Pool</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="k">as</span> <span class="n">p</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]))</span>
</code></pre><p><span class="yiyi-st" id="yiyi-22">将打印到标准输出</span></p><pre><code class="language-python"><span></span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">]</span>
</code></pre><div class="section" id="the-process-class"><h3><span class="yiyi-st" id="yiyi-23">17.2.1.1. </span><span class="yiyi-st" id="yiyi-24"><a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>类</span></h3><p><span class="yiyi-st" id="yiyi-25">在<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>中，通过创建<a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>对象，然后调用其<a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a>方法来生成进程。</span><span class="yiyi-st" id="yiyi-26"><a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>遵循<a class="reference internal" href="threading.html#threading.Thread" title="threading.Thread"><code class="xref py py-class docutils literal"><span class="pre">threading.Thread</span></code></a>的API。</span><span class="yiyi-st" id="yiyi-27">多进程程序的一个简单例子是</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="s1">'bob'</span><span class="p">,))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-28">要显示涉及的单个进程ID，下面是一个扩展示例：</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="k">def</span> <span class="nf">info</span><span class="p">(</span><span class="n">title</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">title</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'module name:'</span><span class="p">,</span> <span class="n">__name__</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'parent process:'</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">getppid</span><span class="p">())</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'process id:'</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">getpid</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="n">info</span><span class="p">(</span><span class="s1">'function f'</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">info</span><span class="p">(</span><span class="s1">'main line'</span><span class="p">)</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="s1">'bob'</span><span class="p">,))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-29">关于为什么<code class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__'</span></code>是必要的，请参阅<a class="reference internal" href="#multiprocessing-programming"><span>编程指南</span></a>。</span></p></div><div class="section" id="contexts-and-start-methods"><h3><span class="yiyi-st" id="yiyi-30">17.2.1.2. </span><span class="yiyi-st" id="yiyi-31">上下文和启动方法</span></h3><p id="multiprocessing-start-methods"><span class="yiyi-st" id="yiyi-32">根据平台，<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>支持三种方式来启动进程。</span><span class="yiyi-st" id="yiyi-33">这些<em>启动方法</em>是</span></p><span class="yiyi-st" id="yiyi-48"><blockquote><div><dl class="docutils"><dt><em>spawn</em></dt> <dd><p class="first">父进程启动一个新的python解释器进程。子进程只会继承运行进程对象<a class="reference internal" href="#multiprocessing.Process.run" title="multiprocessing.Process.run"><code class="xref py py-meth docutils literal"><span class="pre">run()</span></code></a>方法所需的那些资源。特别是，不会继承父进程中不必要的文件描述符和句柄。与使用<em>fork</em>或<em>forkserver</em>相比，使用此方法启动进程相当慢。</p> <p class="last">在Unix和Windows上可用。 Windows上的默认值。</p></dd> <dt><em>fork</em></dt> <dd><p class="first">父进程使用<a class="reference internal" href="os.html#os.fork" title="os.fork"><code class="xref py py-func docutils literal"><span class="pre">os.fork()</span></code></a>来派生Python解释器。子进程在开始时实际上与父进程相同。父进程的所有资源都由子进程继承。请注意，安全分叉多线程进程是有问题的。</p> <p class="last">仅在Unix上可用。 Unix上的默认值。</p></dd> <dt><em>forkserver</em></dt> <dd><p class="first">当程序启动并选择<em>forkserver</em>启动方法时，将启动服务器进程。从那时起，每当需要一个新进程时，父进程就会连接到服务器并请求它分叉一个新进程。 fork服务器进程是单线程的，因此使用<a class="reference internal" href="os.html#os.fork" title="os.fork"><code class="xref py py-func docutils literal"><span class="pre">os.fork()</span></code></a>是安全的。没有不必要的资源被继承。</p> <p class="last">在Unix平台上可用，它支持通过Unix管道传递文件描述符。</p></dd></dl></div></blockquote></span><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-34"><span class="versionmodified">版本3.4更新：</span>所有Unix平台上添加<em>spawn</em>，以及为某些Unix平台添加<em>forkserver</em>。</span><span class="yiyi-st" id="yiyi-35">在Windows上，子进程不再继承父进程所有可继承句柄。</span></p></div><p><span class="yiyi-st" id="yiyi-36">在Unix上使用<em>spawn</em>或<em>forkserver</em>启动方法还将启动一个<em>信号量跟踪器</em>进程，该进程跟踪由程序的进程创建的未链接的命名信号量。</span><span class="yiyi-st" id="yiyi-37">当所有进程退出后，信号量跟踪器取消任何剩余信号量的链接。</span><span class="yiyi-st" id="yiyi-38">通常应该没有剩余信号量，但如果一个进程被一个信号杀死，可能有一些“泄漏”的信号量。</span><span class="yiyi-st" id="yiyi-39">（取消命名信号量的链接是一件重要的事情，因为系统只允许有限的数量，并且它们将不会自动取消链接，直到下次重新启动。）</span></p><p><span class="yiyi-st" id="yiyi-40">要选择启动的方法，请在main模块的<code class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__'</span></code>子句中使用<a class="reference internal" href="#multiprocessing.set_start_method" title="multiprocessing.set_start_method"><code class="xref py py-func docutils literal"><span class="pre">set_start_method()</span></code></a>。</span><span class="yiyi-st" id="yiyi-41">例如：</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">multiprocessing</span> <span class="k">as</span> <span class="nn">mp</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">q</span><span class="p">):</span>
<span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">mp</span><span class="o">.</span><span class="n">set_start_method</span><span class="p">(</span><span class="s1">'spawn'</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Queue</span><span class="p">()</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">foo</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">q</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-42">在程序中<a class="reference internal" href="#multiprocessing.set_start_method" title="multiprocessing.set_start_method"><code class="xref py py-func docutils literal"><span class="pre">set_start_method()</span></code></a>应最多只使用一次。</span></p><p><span class="yiyi-st" id="yiyi-43">或者，你可以使用<a class="reference internal" href="#multiprocessing.get_context" title="multiprocessing.get_context"><code class="xref py py-func docutils literal"><span class="pre">get_context()</span></code></a>来获取context对象。</span><span class="yiyi-st" id="yiyi-44">Context对象具有与multiprocessing模块相同的API，并允许在同一程序中使用多个启动方法。</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">multiprocessing</span> <span class="k">as</span> <span class="nn">mp</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">q</span><span class="p">):</span>
<span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">ctx</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">get_context</span><span class="p">(</span><span class="s1">'spawn'</span><span class="p">)</span>
<span class="n">q</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">Queue</span><span class="p">()</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">foo</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">q</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-45">注意，与context相关的对象可能与用于不同context的进程不兼容。</span><span class="yiyi-st" id="yiyi-46">特别地，使用<em>fork</em> context创建的锁不能传递到使用<em>spawn</em>或<em>forkserver</em>启动方法启动的进程。</span></p><p><span class="yiyi-st" id="yiyi-47">想要使用特定启动方法的库应该使用<a class="reference internal" href="#multiprocessing.get_context" title="multiprocessing.get_context"><code class="xref py py-func docutils literal"><span class="pre">get_context()</span></code></a>，以避免干扰库用户的选择。</span></p></div><div class="section" id="exchanging-objects-between-processes"><h3><span class="yiyi-st" id="yiyi-49">17.2.1.3. </span><span class="yiyi-st" id="yiyi-50">在进程之间交换对象</span></h3><p><span class="yiyi-st" id="yiyi-51"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing </span></code></a>支持进程之间的两种通信通道：</span></p><p><span class="yiyi-st" id="yiyi-52"><strong>Queues</strong></span></p><span class="yiyi-st" id="yiyi-54"><blockquote><div><p>The <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> class is a near clone of <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a>. For example:</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">q</span><span class="p">):</span> <span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">([</span><span class="mi">42</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'hello'</span><span class="p">])</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">q</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">q</span><span class="p">,))</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="nb">print</span><span class="p">(</span><span class="n">q</span><span class="o">.</span><span class="n">get</span><span class="p">())</span> <span class="c1"># prints "[42, None, 'hello']"</span> <span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span></code></pre> <p>Queues are thread and process safe.</p></div></blockquote></span><p><span class="yiyi-st" id="yiyi-53"><strong>Pipes</strong></span></p><span class="yiyi-st" id="yiyi-55"> <blockquote> <div><p>The <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a> function returns a pair of connection objects connected by a pipe which by default is duplex (two-way). For example:</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Pipe</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">conn</span><span class="p">):</span> <span class="n">conn</span><span class="o">.</span><span class="n">send</span><span class="p">([</span><span class="mi">42</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'hello'</span><span class="p">])</span> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">parent_conn</span><span class="p">,</span> <span class="n">child_conn</span> <span class="o">=</span> <span class="n">Pipe</span><span class="p">()</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">child_conn</span><span class="p">,))</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="nb">print</span><span class="p">(</span><span class="n">parent_conn</span><span class="o">.</span><span class="n">recv</span><span class="p">())</span> <span class="c1"># prints "[42, None, 'hello']"</span> <span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span> </code></pre> <p>The two connection objects returned by <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a> represent the two ends of the pipe. Each connection object has <a class="reference internal" href="#multiprocessing.Connection.send" title="multiprocessing.Connection.send"><code class="xref py py-meth docutils literal"><span class="pre">send()</span></code></a> and <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">recv()</span></code></a> methods (among others). Note that data in a pipe may become corrupted if two processes (or threads) try to read from or write to the <em>same</em> end of the pipe at the same time. Of course there is no risk of corruption from processes using different ends of the pipe at the same time.</p> </div></blockquote></span></div><div class="section" id="synchronization-between-processes"><h3><span class="yiyi-st" id="yiyi-56">17.2.1.4. </span><span class="yiyi-st" id="yiyi-57">进程之间的同步</span></h3><p><span class="yiyi-st" id="yiyi-58"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>包含了全部和<a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a>相同的同步原语 。</span><span class="yiyi-st" id="yiyi-59">例如，可以使用锁来保证同一时间只有一个进程在使用标准输出：</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Lock</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="n">i</span><span class="p">):</span>
<span class="n">l</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'hello world'</span><span class="p">,</span> <span class="n">i</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">l</span><span class="o">.</span><span class="n">release</span><span class="p">()</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">lock</span> <span class="o">=</span> <span class="n">Lock</span><span class="p">()</span>
<span class="k">for</span> <span class="n">num</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">lock</span><span class="p">,</span> <span class="n">num</span><span class="p">))</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-60">如果不使用锁，来自不同进程的输出很容易都混淆在一起。</span></p></div><div class="section" id="sharing-state-between-processes"><h3><span class="yiyi-st" id="yiyi-61">17.2.1.5. </span><span class="yiyi-st" id="yiyi-62">进程之间的状态共享</span></h3><p><span class="yiyi-st" id="yiyi-63">如上所述，当进行并发编程时，最好尽量避免使用共享状态。</span><span class="yiyi-st" id="yiyi-64">使用多个进程时尤其如此。</span></p><p><span class="yiyi-st" id="yiyi-65">但是，如果你确实需要使用某些共享数据，那么<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>提供了两种方法。</span></p><p><span class="yiyi-st" id="yiyi-66"><strong>共享内存</strong></span></p><span class="yiyi-st" id="yiyi-68"> <blockquote> <div><p>Data can be stored in a shared memory map using <a class="reference internal" href="#multiprocessing.Value" title="multiprocessing.Value"><code class="xref py py-class docutils literal"><span class="pre">Value</span></code></a> or <a class="reference internal" href="#multiprocessing.Array" title="multiprocessing.Array"><code class="xref py py-class docutils literal"><span class="pre">Array</span></code></a>. For example, the following code</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Value</span><span class="p">,</span> <span class="n">Array</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">a</span><span class="p">):</span> <span class="n">n</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="mf">3.1415927</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">a</span><span class="p">)):</span> <span class="n">a</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="o">-</span><span class="n">a</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">num</span> <span class="o">=</span> <span class="n">Value</span><span class="p">(</span><span class="s1">'d'</span><span class="p">,</span> <span class="mf">0.0</span><span class="p">)</span> <span class="n">arr</span> <span class="o">=</span> <span class="n">Array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">num</span><span class="p">,</span> <span class="n">arr</span><span class="p">))</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span> <span class="nb">print</span><span class="p">(</span><span class="n">num</span><span class="o">.</span><span class="n">value</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="n">arr</span><span class="p">[:])</span> </code></pre> <p>will print</p> <pre><code class="language-python"><span></span><span class="mf">3.1415927</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">,</span> <span class="o">-</span><span class="mi">4</span><span class="p">,</span> <span class="o">-</span><span class="mi">5</span><span class="p">,</span> <span class="o">-</span><span class="mi">6</span><span class="p">,</span> <span class="o">-</span><span class="mi">7</span><span class="p">,</span> <span class="o">-</span><span class="mi">8</span><span class="p">,</span> <span class="o">-</span><span class="mi">9</span><span class="p">]</span> </code></pre> <p>The <code class="docutils literal"><span class="pre">'d'</span></code> and <code class="docutils literal"><span class="pre">'i'</span></code> arguments used when creating <code class="docutils literal"><span class="pre">num</span></code> and <code class="docutils literal"><span class="pre">arr</span></code> are typecodes of the kind used by the <a class="reference internal" href="array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal"><span class="pre">array</span></code></a> module: <code class="docutils literal"><span class="pre">'d'</span></code> indicates a double precision float and <code class="docutils literal"><span class="pre">'i'</span></code> indicates a signed integer. These shared objects will be process and thread-safe.</p> <p>For more flexibility in using shared memory one can use the <a class="reference internal" href="#module-multiprocessing.sharedctypes" title="multiprocessing.sharedctypes: Allocate ctypes objects from shared memory."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.sharedctypes</span></code></a> module which supports the creation of arbitrary ctypes objects allocated from shared memory.</p> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-67"><strong>Server process</strong></span></p><span class="yiyi-st" id="yiyi-69"> <blockquote> <div><p>A manager object returned by <code class="xref py py-func docutils literal"><span class="pre">Manager()</span></code> controls a server process which holds Python objects and allows other processes to manipulate them using proxies.</p> <p>A manager returned by <code class="xref py py-func docutils literal"><span class="pre">Manager()</span></code> will support types <a class="reference internal" href="stdtypes.html#list" title="list"><code class="xref py py-class docutils literal"><span class="pre">list</span></code></a>, <a class="reference internal" href="stdtypes.html#dict" title="dict"><code class="xref py py-class docutils literal"><span class="pre">dict</span></code></a>, <a class="reference internal" href="#multiprocessing.managers.Namespace" title="multiprocessing.managers.Namespace"><code class="xref py py-class docutils literal"><span class="pre">Namespace</span></code></a>, <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a>, <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a>, <a class="reference internal" href="#multiprocessing.Semaphore" title="multiprocessing.Semaphore"><code class="xref py py-class docutils literal"><span class="pre">Semaphore</span></code></a>, <a class="reference internal" href="#multiprocessing.BoundedSemaphore" title="multiprocessing.BoundedSemaphore"><code class="xref py py-class docutils literal"><span class="pre">BoundedSemaphore</span></code></a>, <a class="reference internal" href="#multiprocessing.Condition" title="multiprocessing.Condition"><code class="xref py py-class docutils literal"><span class="pre">Condition</span></code></a>, <a class="reference internal" href="#multiprocessing.Event" title="multiprocessing.Event"><code class="xref py py-class docutils literal"><span class="pre">Event</span></code></a>, <a class="reference internal" href="#multiprocessing.Barrier" title="multiprocessing.Barrier"><code class="xref py py-class docutils literal"><span class="pre">Barrier</span></code></a>, <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a>, <a class="reference internal" href="#multiprocessing.Value" title="multiprocessing.Value"><code class="xref py py-class docutils literal"><span class="pre">Value</span></code></a> and <a class="reference internal" href="#multiprocessing.Array" title="multiprocessing.Array"><code class="xref py py-class docutils literal"><span class="pre">Array</span></code></a>. For example,</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Manager</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">l</span><span class="p">):</span> <span class="n">d</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'1'</span> <span class="n">d</span><span class="p">[</span><span class="s1">'2'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span> <span class="n">d</span><span class="p">[</span><span class="mf">0.25</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span> <span class="n">l</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="k">with</span> <span class="n">Manager</span><span class="p">()</span> <span class="k">as</span> <span class="n">manager</span><span class="p">:</span> <span class="n">d</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">dict</span><span class="p">()</span> <span class="n">l</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">l</span><span class="p">))</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span> <span class="nb">print</span><span class="p">(</span><span class="n">d</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="n">l</span><span class="p">)</span> </code></pre> <p>will print</p> <pre><code class="language-python"><span></span><span class="p">{</span><span class="mf">0.25</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">1</span><span class="p">:</span> <span class="s1">'1'</span><span class="p">,</span> <span class="s1">'2'</span><span class="p">:</span> <span class="mi">2</span><span class="p">}</span> <span class="p">[</span><span class="mi">9</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span> </code></pre> <p>Server process managers are more flexible than using shared memory objects because they can be made to support arbitrary object types. Also, a single manager can be shared by processes on different computers over a network. They are, however, slower than using shared memory.</p> </div></blockquote></span></div><div class="section" id="using-a-pool-of-workers"><h3><span class="yiyi-st" id="yiyi-70">17.2.1.6. </span><span class="yiyi-st" id="yiyi-71">使用工作进程的进程池</span></h3><p><span class="yiyi-st" id="yiyi-72"><a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>类表示工作进程的进程池。</span><span class="yiyi-st" id="yiyi-73">它具有允许将任务以几种不同的方式分配到工作进程的方法。</span></p><p><span class="yiyi-st" id="yiyi-74">例如：</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Pool</span><span class="p">,</span> <span class="ne">TimeoutError</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="c1"># start 4 worker processes</span>
<span class="k">with</span> <span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span>
<span class="c1"># print "[0, 1, 4,..., 81]"</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pool</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)))</span>
<span class="c1"># print same numbers in arbitrary order</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap_unordered</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="c1"># evaluate "f(20)" asynchronously</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="p">(</span><span class="mi">20</span><span class="p">,))</span> <span class="c1"># runs in *only* one process</span>
<span class="nb">print</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="c1"># prints "400"</span>
<span class="c1"># evaluate "os.getpid()" asynchronously</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getpid</span><span class="p">,</span> <span class="p">())</span> <span class="c1"># runs in *only* one process</span>
<span class="nb">print</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="c1"># prints the PID of that process</span>
<span class="c1"># launching multiple evaluations asynchronously *may* use more processes</span>
<span class="n">multiple_results</span> <span class="o">=</span> <span class="p">[</span><span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getpid</span><span class="p">,</span> <span class="p">())</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">)]</span>
<span class="nb">print</span><span class="p">([</span><span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="k">for</span> <span class="n">res</span> <span class="ow">in</span> <span class="n">multiple_results</span><span class="p">])</span>
<span class="c1"># make a single worker sleep for 10 secs</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">,</span> <span class="p">(</span><span class="mi">10</span><span class="p">,))</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span>
<span class="k">except</span> <span class="ne">TimeoutError</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"We lacked patience and got a multiprocessing.TimeoutError"</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"For the moment, the pool remains available for more work"</span><span class="p">)</span>
<span class="c1"># exiting the 'with'-block has stopped the pool</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"Now the pool is closed and no longer available"</span><span class="p">)</span>
</code></pre><p><span class="yiyi-st" id="yiyi-75">注意，进程池的方法只应该被创建它的进程使用。</span></p><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-76">注意</span></p><p><span class="yiyi-st" id="yiyi-77">此包中的功能要求<code class="docutils literal"><span class="pre">__main__</span></code>模块可由子进程导入。</span><span class="yiyi-st" id="yiyi-78">这在<a class="reference internal" href="#multiprocessing-programming"><span>编程指南</span></a>中有所说明，但值得在这里指出。</span><span class="yiyi-st" id="yiyi-79">这意味着一些示例，如<a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">multiprocessing.pool.Pool</span></code></a>示例在交互式解释器中不能工作。</span><span class="yiyi-st" id="yiyi-80">例如：</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Pool</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span> <span class="o">=</span> <span class="n">Pool</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="gp">...</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">])</span>
<span class="go">Process PoolWorker-1:</span>
<span class="go">Process PoolWorker-2:</span>
<span class="go">Process PoolWorker-3:</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="gr">AttributeError</span>: <span class="n">'module' object has no attribute 'f'</span>
<span class="go">AttributeError: 'module' object has no attribute 'f'</span>
<span class="go">AttributeError: 'module' object has no attribute 'f'</span>
</code></pre><p class="last"><span class="yiyi-st" id="yiyi-81">（如果你尝试这样，它实际上会输出以半随机方式交错的三个完整的traceback，然后你可能不得不停止主进程。）</span></p></div></div></div><div class="section" id="reference"><h2><span class="yiyi-st" id="yiyi-82">17.2.2. </span><span class="yiyi-st" id="yiyi-83">Reference</span></h2><p><span class="yiyi-st" id="yiyi-84">The <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> package mostly replicates the API of the <a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a> module.</span></p><div class="section" id="process-and-exceptions"><h3><span class="yiyi-st" id="yiyi-85">17.2.2.1. </span><span class="yiyi-st" id="yiyi-86"><a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">进程</span></code></a>和异常</span></h3><dl class="class"><dt id="multiprocessing.Process"><span class="yiyi-st" id="yiyi-87"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Process</code><span class="sig-paren">(</span><em>group=None</em>, <em>target=None</em>, <em>name=None</em>, <em>args=()</em>, <em>kwargs={}</em>, <em>*</em>, <em>daemon=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-88">进程对象表示在单独的进程中运行的活动。</span><span class="yiyi-st" id="yiyi-89">进程(<a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>)类有类似<a class="reference internal" href="threading.html#threading.Thread" title="threading.Thread"><code class="xref py py-class docutils literal"><span class="pre">threading.Thread</span></code></a>的所有方法。</span></p><p><span class="yiyi-st" id="yiyi-90">应始终使用关键字参数调用构造函数。</span><span class="yiyi-st" id="yiyi-91"><em>group</em>应始终为<code class="docutils literal"><span class="pre">None</span></code>；它仅用于与<a class="reference internal" href="threading.html#threading.Thread" title="threading.Thread"><code class="xref py py-class docutils literal"><span class="pre">threading.Thread</span></code></a>兼容。</span><span class="yiyi-st" id="yiyi-92"><em>target</em>是<a class="reference internal" href="#multiprocessing.Process.run" title="multiprocessing.Process.run"><code class="xref py py-meth docutils literal"><span class="pre">run()</span></code></a>方法调用的可调用对象。</span><span class="yiyi-st" id="yiyi-93">它默认为<code class="docutils literal"><span class="pre">None</span></code>，这意味着什么都不会被调用。</span><span class="yiyi-st" id="yiyi-94"><em>name</em>是进程名称（有关更多详细信息，请参阅<a class="reference internal" href="#multiprocessing.Process.name" title="multiprocessing.Process.name"><code class="xref py py-attr docutils literal"><span class="pre">name</span></code></a>）。</span><span class="yiyi-st" id="yiyi-95"><em>args</em>是目标调用的参数元组。</span><span class="yiyi-st" id="yiyi-96"><em>kwargs</em>是目标调用的关键字参数字典。</span><span class="yiyi-st" id="yiyi-97">如果给形参赋值，则<em>daemon</em>参数将进程<a class="reference internal" href="#multiprocessing.Process.daemon" title="multiprocessing.Process.daemon"><code class="xref py py-attr docutils literal"><span class="pre">daemon</span></code></a>设置为<code class="docutils literal"><span class="pre">True</span></code>或<code class="docutils literal"><span class="pre">False</span></code>。</span><span class="yiyi-st" id="yiyi-98">如果为<code class="docutils literal"><span class="pre">None</span></code>（默认），该标志将从创建过程中继承。</span></p><p><span class="yiyi-st" id="yiyi-99">默认情况下，没有参数传递给<em>target</em>。</span></p><p><span class="yiyi-st" id="yiyi-100">如果一个子类重写构造函数，它必须确保它在执行任何其他操作之前调用基类构造函数（<code class="xref py py-meth docutils literal"><span class="pre">Process .__ init __()</span></code>）。</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-101"><span class="versionmodified">在版本3.3中更改：</span>添加了<em>daemon</em>参数。</span></p></div><dl class="method"><dt id="multiprocessing.Process.run"><span class="yiyi-st" id="yiyi-102"> <code class="descname">run</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-103">表示进程行为的方法。</span></p><p><span class="yiyi-st" id="yiyi-104">您可以在子类中重写此方法。</span><span class="yiyi-st" id="yiyi-105">标准<a class="reference internal" href="#multiprocessing.Process.run" title="multiprocessing.Process.run"><code class="xref py py-meth docutils literal"><span class="pre">run()</span></code></a>方法调用传递给对象构造函数的可调用对象作为目标参数，如果有的话，从<em>args</em>和<em>kwargs</em>(元组和字典)传参。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Process.start"><span class="yiyi-st" id="yiyi-106"> <code class="descname">start</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-107">开启进程。</span></p><p><span class="yiyi-st" id="yiyi-108">每个进程对象最多只能调用一次。</span><span class="yiyi-st" id="yiyi-109">它安排对象的<a class="reference internal" href="#multiprocessing.Process.run" title="multiprocessing.Process.run"><code class="xref py py-meth docutils literal"><span class="pre">run()</span></code></a>方法在单独的进程中调用。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Process.join"><span class="yiyi-st" id="yiyi-110"> <code class="descname">join</code><span class="sig-paren">(</span><span class="optional">[</span><em>timeout</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-111">如果可选参数<em>timeout</em>为<code class="docutils literal"><span class="pre">None</span></code>（缺省值），则该方法将阻塞，直到调用<a class="reference internal" href="#multiprocessing.Process.join" title="multiprocessing.Process.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a>方法的进程终止。</span><span class="yiyi-st" id="yiyi-112">如果<em>timeout</em>是一个正数，它会阻止至多<em>timeout</em>秒。</span></p><p><span class="yiyi-st" id="yiyi-113">一个过程可以连接多次。</span></p><p><span class="yiyi-st" id="yiyi-114">进程无法自行加入，因为这会导致死锁。</span><span class="yiyi-st" id="yiyi-115">尝试在启动之前加入进程是错误的。</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.Process.name"><span class="yiyi-st" id="yiyi-116"> <code class="descname">name</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-117">该进程的名称。 </span><span class="yiyi-st" id="yiyi-118">该名称是仅用于识别目的的字符串。</span><span class="yiyi-st" id="yiyi-119">它没有语义。</span><span class="yiyi-st" id="yiyi-120">多个进程可以被赋予相同的名称。</span></p><p><span class="yiyi-st" id="yiyi-121">初始名称由构造函数设置。</span><span class="yiyi-st" id="yiyi-122">如果没有明确的名字被提供给构造函数，则构造一个名为“Process-N1：N2：...：Nk”的表单，其中每个Nk是其父进程的第N个子进程。 <sub></sub><sub></sub><sub></sub><sub></sub></span></p></dd></dl><dl class="method"><dt id="multiprocessing.Process.is_alive"><span class="yiyi-st" id="yiyi-123"> <code class="descname">is_alive</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-124">返回进程是否存在。 </span></p><p><span class="yiyi-st" id="yiyi-125">粗略地说，一个进程对象在<a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a>方法返回之前一直存在，直到子进程终止。</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.Process.daemon"><span class="yiyi-st" id="yiyi-126"> <code class="descname">daemon</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-127">进程的守护进程标志，一个布尔值。</span><span class="yiyi-st" id="yiyi-128">这必须在<a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a>被调用之前设置。</span></p><p><span class="yiyi-st" id="yiyi-129">初始值是从创建过程继承的。</span></p><p><span class="yiyi-st" id="yiyi-130">当进程退出时，它将尝试终止其所有守护进程的子进程。</span></p><p><span class="yiyi-st" id="yiyi-131">注意，守护进程不允许创建子进程。</span><span class="yiyi-st" id="yiyi-132">因为如果容许这样做，当守护进程的父进程退出时，该父进程终止守护进程，守护进程的子进程就会被孤立</span><span class="yiyi-st" id="yiyi-133">另外，这些是<strong>不是</strong> Unix守护进程或服务，它们是正常的进程，如果非守护进程退出，它们将被终止（而不是加入）。</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-134">除了<a class="reference internal" href="threading.html#threading.Thread" title="threading.Thread"><code class="xref py py-class docutils literal"><span class="pre">threading.Thread</span></code></a> API之外，<a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>对象还支持以下属性和方法：</span></p><dl class="attribute"><dt id="multiprocessing.Process.pid"><span class="yiyi-st" id="yiyi-135"> <code class="descname">pid</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-136">返回进程ID。</span><span class="yiyi-st" id="yiyi-137">在生成过程之前，这将是<code class="docutils literal"><span class="pre">None</span></code>。</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.Process.exitcode"><span class="yiyi-st" id="yiyi-138"> <code class="descname">exitcode</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-139">子进程的退出代码。</span><span class="yiyi-st" id="yiyi-140">如果进程尚未终止，则这将是<code class="docutils literal"><span class="pre">None</span></code>。</span><span class="yiyi-st" id="yiyi-141">负值<em>-N</em>表示子进程被信号<em>N</em>终止。</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.Process.authkey"><span class="yiyi-st" id="yiyi-142"> <code class="descname">authkey</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-143">进程的身份验证密钥（一个字节字符串）。 </span></p><p><span class="yiyi-st" id="yiyi-144">当多进程(<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>)被初始化时，主进程会使用<a class="reference internal" href="os.html#os.urandom" title="os.urandom"><code class="xref py py-func docutils literal"><span class="pre">os.urandom()</span></code></a>分配一个随机字符串。</span></p><p><span class="yiyi-st" id="yiyi-145">创建一个进程(<a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>)对象时，它将继承其父进程的身份验证密钥，但可以通过将<a class="reference internal" href="#multiprocessing.Process.authkey" title="multiprocessing.Process.authkey"><code class="xref py py-attr docutils literal"><span class="pre">authkey</span></code></a>设置为另一个字节字符串来更改。</span></p><p><span class="yiyi-st" id="yiyi-146">请参阅<a class="reference internal" href="#multiprocessing-auth-keys"><span>身份验证密钥</span></a>。</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.Process.sentinel"><span class="yiyi-st" id="yiyi-147"> <code class="descname">sentinel</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-148">系统对象的数字句柄，当过程结束时它将变为“就绪”。</span></p><p><span class="yiyi-st" id="yiyi-149">如果您想使用<a class="reference internal" href="#multiprocessing.connection.wait" title="multiprocessing.connection.wait"><code class="xref py py-func docutils literal"><span class="pre">multiprocessing.connection.wait()</span></code></a>一次等待几个事件，则可以使用此值。</span><span class="yiyi-st" id="yiyi-150">否则调用<a class="reference internal" href="#multiprocessing.Process.join" title="multiprocessing.Process.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a>更简单。</span></p><p><span class="yiyi-st" id="yiyi-151">在Windows上，这是一个可与<code class="docutils literal"><span class="pre">WaitForSingleObject</span></code>和<code class="docutils literal"><span class="pre">WaitForMultipleObjects</span></code>系列API调用一起使用的操作系统句柄。</span><span class="yiyi-st" id="yiyi-152">在Unix上，这是一个可用于<a class="reference internal" href="select.html#module-select" title="select: Wait for I/O completion on multiple streams."><code class="xref py py-mod docutils literal"><span class="pre">select</span></code></a>模块的基元的文件描述符。</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-153"><span class="versionmodified">3.3版本中的新功能</span></span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.Process.terminate"><span class="yiyi-st" id="yiyi-154"> <code class="descname">terminate</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-155">终止该过程。</span><span class="yiyi-st" id="yiyi-156">在Unix上使用<code class="docutils literal"><span class="pre">SIGTERM</span></code>信号完成；在Windows上使用<code class="xref c c-func docutils literal"><span class="pre">TerminateProcess()</span></code>。</span><span class="yiyi-st" id="yiyi-157">请注意，退出处理程序执行后,最后的子句等将不会执行。</span></p><p><span class="yiyi-st" id="yiyi-158">请注意，进程的后代进程将<em>不</em>终止 - 它们将简单地变为孤立。</span></p><div class="admonition warning"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-159">Warning</span></p><p class="last"><span class="yiyi-st" id="yiyi-160">If this method is used when the associated process is using a pipe or queue then the pipe or queue is liable to become corrupted and may become unusable by other process. </span><span class="yiyi-st" id="yiyi-161">Similarly, if the process has acquired a lock or semaphore etc. </span><span class="yiyi-st" id="yiyi-162">then terminating it is liable to cause other processes to deadlock.</span></p></div></dd></dl><p><span class="yiyi-st" id="yiyi-163">注意<a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a>，<a class="reference internal" href="#multiprocessing.Process.join" title="multiprocessing.Process.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a>，<a class="reference internal" href="#multiprocessing.Process.is_alive" title="multiprocessing.Process.is_alive"><code class="xref py py-meth docutils literal"><span class="pre">is_alive()</span></code></a>，<a class="reference internal" href="#multiprocessing.Process.terminate" title="multiprocessing.Process.terminate"><code class="xref py py-meth docutils literal"><span class="pre">terminate()</span></code></a>和<a class="reference internal" href="#multiprocessing.Process.exitcode" title="multiprocessing.Process.exitcode"><code class="xref py py-attr docutils literal"><span class="pre">exitcode</span></code></a>方法只应由创建过程对象的进程调用。</span></p><p><span class="yiyi-st" id="yiyi-164">进程(<a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>)的一些方法的示例用法：</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">multiprocessing</span><span class="o">,</span> <span class="nn">time</span><span class="o">,</span> <span class="nn">signal</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="mi">1000</span><span class="p">,))</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">p</span><span class="o">.</span><span class="n">is_alive</span><span class="p">())</span>
<span class="go">&lt;Process(Process-1, initial)&gt; False</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">p</span><span class="o">.</span><span class="n">is_alive</span><span class="p">())</span>
<span class="go">&lt;Process(Process-1, started)&gt; True</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span><span class="o">.</span><span class="n">terminate</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">p</span><span class="o">.</span><span class="n">is_alive</span><span class="p">())</span>
<span class="go">&lt;Process(Process-1, stopped[SIGTERM])&gt; False</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">p</span><span class="o">.</span><span class="n">exitcode</span> <span class="o">==</span> <span class="o">-</span><span class="n">signal</span><span class="o">.</span><span class="n">SIGTERM</span>
<span class="go">True</span>
</code></pre></dd></dl><dl class="exception"><dt id="multiprocessing.ProcessError"><span class="yiyi-st" id="yiyi-165"> <em class="property">exception </em><code class="descclassname">multiprocessing.</code><code class="descname">ProcessError</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-166">所有<a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">多线程</span></code></a>异常的基类。</span></p></dd></dl><dl class="exception"><dt id="multiprocessing.BufferTooShort"><span class="yiyi-st" id="yiyi-167"> <em class="property">exception </em><code class="descclassname">multiprocessing.</code><code class="descname">BufferTooShort</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-168">当提供的缓冲区对象对于消息读取而言太小时，由<a class="reference internal" href="#multiprocessing.Connection.recv_bytes_into" title="multiprocessing.Connection.recv_bytes_into"><code class="xref py py-meth docutils literal"><span class="pre">Connection.recv_bytes_into()</span></code></a>引发异常。</span></p><p><span class="yiyi-st" id="yiyi-169">如果<code class="docutils literal"><span class="pre">e</span></code>是<a class="reference internal" href="#multiprocessing.BufferTooShort" title="multiprocessing.BufferTooShort"><code class="xref py py-exc docutils literal"><span class="pre">BufferTooShort</span></code></a>的实例，则<code class="docutils literal"><span class="pre">e.a​​rgs [0]</span></code>将以字节字符串形式给出消息。</span></p></dd></dl><dl class="exception"><dt id="multiprocessing.AuthenticationError"><span class="yiyi-st" id="yiyi-170"> <em class="property">exception </em><code class="descclassname">multiprocessing.</code><code class="descname">AuthenticationError</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-171">出现身份验证错误时引发。</span></p></dd></dl><dl class="exception"><dt id="multiprocessing.TimeoutError"><span class="yiyi-st" id="yiyi-172"> <em class="property">exception </em><code class="descclassname">multiprocessing.</code><code class="descname">TimeoutError</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-173">当超时过期时由超时方法提出。</span></p></dd></dl></div><div class="section" id="pipes-and-queues"><h3><span class="yiyi-st" id="yiyi-174">17.2.2.2. </span><span class="yiyi-st" id="yiyi-175">Pipes 和 Queues</span></h3><p><span class="yiyi-st" id="yiyi-176">When using multiple processes, one generally uses message passing for communication between processes and avoids having to use any synchronization primitives like locks.</span></p><p><span class="yiyi-st" id="yiyi-177">For passing messages one can use <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a> (for a connection between two processes) or a queue (which allows multiple producers and consumers).</span></p><p><span class="yiyi-st" id="yiyi-178">The <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a>, <a class="reference internal" href="#multiprocessing.SimpleQueue" title="multiprocessing.SimpleQueue"><code class="xref py py-class docutils literal"><span class="pre">SimpleQueue</span></code></a> and <a class="reference internal" href="#multiprocessing.JoinableQueue" title="multiprocessing.JoinableQueue"><code class="xref py py-class docutils literal"><span class="pre">JoinableQueue</span></code></a> types are multi-producer, multi-consumer FIFO queues modelled on the <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a> class in the standard library. </span><span class="yiyi-st" id="yiyi-179">They differ in that <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> lacks the <a class="reference internal" href="queue.html#queue.Queue.task_done" title="queue.Queue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> and <a class="reference internal" href="queue.html#queue.Queue.join" title="queue.Queue.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a> methods introduced into Python 2.5’s <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a> class.</span></p><p><span class="yiyi-st" id="yiyi-180">如果您使用<a class="reference internal" href="#multiprocessing.JoinableQueue" title="multiprocessing.JoinableQueue"><code class="xref py py-class docutils literal"><span class="pre">JoinableQueue</span></code></a>，那么您<strong>必须</strong>为每一个从队列中移除的任务去调用<a class="reference internal" href="#multiprocessing.JoinableQueue.task_done" title="multiprocessing.JoinableQueue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">JoinableQueue.task_done()</span></code></a>。否则的话，那些用于记录未完成的任务数量的信号量或许最终会溢出，从而引发异常。</span></p><p><span class="yiyi-st" id="yiyi-181">Note that one can also create a shared queue by using a manager object – see <a class="reference internal" href="#multiprocessing-managers"><span>Managers</span></a>.</span></p><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-182">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-183"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> uses the usual <a class="reference internal" href="queue.html#queue.Empty" title="queue.Empty"><code class="xref py py-exc docutils literal"><span class="pre">queue.Empty</span></code></a> and <a class="reference internal" href="queue.html#queue.Full" title="queue.Full"><code class="xref py py-exc docutils literal"><span class="pre">queue.Full</span></code></a> exceptions to signal a timeout. </span><span class="yiyi-st" id="yiyi-184">They are not available in the <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> namespace so you need to import them from <a class="reference internal" href="queue.html#module-queue" title="queue: A synchronized queue class."><code class="xref py py-mod docutils literal"><span class="pre">queue</span></code></a>.</span></p></div><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-185">Note</span></p><p><span class="yiyi-st" id="yiyi-186">When an object is put on a queue, the object is pickled and a background thread later flushes the pickled data to an underlying pipe. </span><span class="yiyi-st" id="yiyi-187">This has some consequences which are a little surprising, but should not cause any practical difficulties – if they really bother you then you can instead use a queue created with a <a class="reference internal" href="#multiprocessing-managers"><span>manager</span></a>.</span></p><ol class="last arabic simple"><li><span class="yiyi-st" id="yiyi-188">After putting an object on an empty queue there may be an infinitesimal delay before the queue’s <a class="reference internal" href="#multiprocessing.Queue.empty" title="multiprocessing.Queue.empty"><code class="xref py py-meth docutils literal"><span class="pre">empty()</span></code></a> method returns <a class="reference internal" href="constants.html#False" title="False"><code class="xref py py-const docutils literal"><span class="pre">False</span></code></a> and <a class="reference internal" href="#multiprocessing.Queue.get_nowait" title="multiprocessing.Queue.get_nowait"><code class="xref py py-meth docutils literal"><span class="pre">get_nowait()</span></code></a> can return without raising <a class="reference internal" href="queue.html#queue.Empty" title="queue.Empty"><code class="xref py py-exc docutils literal"><span class="pre">queue.Empty</span></code></a>.</span></li><li><span class="yiyi-st" id="yiyi-189">If multiple processes are enqueuing objects, it is possible for the objects to be received at the other end out-of-order. </span><span class="yiyi-st" id="yiyi-190">However, objects enqueued by the same process will always be in the expected order with respect to each other.</span></li></ol></div><div class="admonition warning"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-191">Warning</span></p><p class="last"><span class="yiyi-st" id="yiyi-192">If a process is killed using <a class="reference internal" href="#multiprocessing.Process.terminate" title="multiprocessing.Process.terminate"><code class="xref py py-meth docutils literal"><span class="pre">Process.terminate()</span></code></a> or <a class="reference internal" href="os.html#os.kill" title="os.kill"><code class="xref py py-func docutils literal"><span class="pre">os.kill()</span></code></a> while it is trying to use a <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a>, then the data in the queue is likely to become corrupted. </span><span class="yiyi-st" id="yiyi-193">This may cause any other process to get an exception when it tries to use the queue later on.</span></p></div><div class="admonition warning"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-194">Warning</span></p><p><span class="yiyi-st" id="yiyi-195">As mentioned above, if a child process has put items on a queue (and it has not used <a class="reference internal" href="#multiprocessing.Queue.cancel_join_thread" title="multiprocessing.Queue.cancel_join_thread"><code class="xref py py-meth docutils literal"><span class="pre">JoinableQueue.cancel_join_thread</span></code></a>), then that process will not terminate until all buffered items have been flushed to the pipe.</span></p><p><span class="yiyi-st" id="yiyi-196">This means that if you try joining that process you may get a deadlock unless you are sure that all items which have been put on the queue have been consumed. </span><span class="yiyi-st" id="yiyi-197">Similarly, if the child process is non-daemonic then the parent process may hang on exit when it tries to join all its non-daemonic children.</span></p><p class="last"><span class="yiyi-st" id="yiyi-198">Note that a queue created using a manager does not have this issue. </span><span class="yiyi-st" id="yiyi-199">See <a class="reference internal" href="#multiprocessing-programming"><span>Programming guidelines</span></a>.</span></p></div><p><span class="yiyi-st" id="yiyi-200">For an example of the usage of queues for interprocess communication see <a class="reference internal" href="#multiprocessing-examples"><span>Examples</span></a>.</span></p><dl class="function"><dt id="multiprocessing.Pipe"><span class="yiyi-st" id="yiyi-201"> <code class="descclassname">multiprocessing.</code><code class="descname">Pipe</code><span class="sig-paren">(</span><span class="optional">[</span><em>duplex</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-202">Returns a pair <code class="docutils literal"><span class="pre">(conn1,</span> <span class="pre">conn2)</span></code> of <a class="reference internal" href="#multiprocessing.Connection" title="multiprocessing.Connection"><code class="xref py py-class docutils literal"><span class="pre">Connection</span></code></a> objects representing the ends of a pipe.</span></p><p><span class="yiyi-st" id="yiyi-203">If <em>duplex</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) then the pipe is bidirectional. </span><span class="yiyi-st" id="yiyi-204">If <em>duplex</em> is <code class="docutils literal"><span class="pre">False</span></code> then the pipe is unidirectional: <code class="docutils literal"><span class="pre">conn1</span></code> can only be used for receiving messages and <code class="docutils literal"><span class="pre">conn2</span></code> can only be used for sending messages.</span></p></dd></dl><dl class="class"><dt id="multiprocessing.Queue"><span class="yiyi-st" id="yiyi-205"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Queue</code><span class="sig-paren">(</span><span class="optional">[</span><em>maxsize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-206">返回进程共享的队列，底层使用管道和锁来实现。</span><span class="yiyi-st" id="yiyi-207">When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe.</span></p><p><span class="yiyi-st" id="yiyi-208">The usual <a class="reference internal" href="queue.html#queue.Empty" title="queue.Empty"><code class="xref py py-exc docutils literal"><span class="pre">queue.Empty</span></code></a> and <a class="reference internal" href="queue.html#queue.Full" title="queue.Full"><code class="xref py py-exc docutils literal"><span class="pre">queue.Full</span></code></a> exceptions from the standard library’s <a class="reference internal" href="queue.html#module-queue" title="queue: A synchronized queue class."><code class="xref py py-mod docutils literal"><span class="pre">queue</span></code></a> module are raised to signal timeouts.</span></p><p><span class="yiyi-st" id="yiyi-209"><a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> implements all the methods of <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a> except for <a class="reference internal" href="queue.html#queue.Queue.task_done" title="queue.Queue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> and <a class="reference internal" href="queue.html#queue.Queue.join" title="queue.Queue.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a>.</span></p><dl class="method"><dt id="multiprocessing.Queue.qsize"><span class="yiyi-st" id="yiyi-210"> <code class="descname">qsize</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-211">Return the approximate size of the queue. </span><span class="yiyi-st" id="yiyi-212">Because of multithreading/multiprocessing semantics, this number is not reliable.</span></p><p><span class="yiyi-st" id="yiyi-213">Note that this may raise <a class="reference internal" href="exceptions.html#NotImplementedError" title="NotImplementedError"><code class="xref py py-exc docutils literal"><span class="pre">NotImplementedError</span></code></a> on Unix platforms like Mac OS X where <code class="docutils literal"><span class="pre">sem_getvalue()</span></code> is not implemented.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.empty"><span class="yiyi-st" id="yiyi-214"> <code class="descname">empty</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-215">Return <code class="docutils literal"><span class="pre">True</span></code> if the queue is empty, <code class="docutils literal"><span class="pre">False</span></code> otherwise. </span><span class="yiyi-st" id="yiyi-216">Because of multithreading/multiprocessing semantics, this is not reliable.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.full"><span class="yiyi-st" id="yiyi-217"> <code class="descname">full</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-218">Return <code class="docutils literal"><span class="pre">True</span></code> if the queue is full, <code class="docutils literal"><span class="pre">False</span></code> otherwise. </span><span class="yiyi-st" id="yiyi-219">Because of multithreading/multiprocessing semantics, this is not reliable.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.put"><span class="yiyi-st" id="yiyi-220"> <code class="descname">put</code><span class="sig-paren">(</span><em>obj</em><span class="optional">[</span>, <em>block</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-221">Put obj into the queue. </span><span class="yiyi-st" id="yiyi-222">If the optional argument <em>block</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) and <em>timeout</em> is <code class="docutils literal"><span class="pre">None</span></code> (the default), block if necessary until a free slot is available. </span><span class="yiyi-st" id="yiyi-223">If <em>timeout</em> is a positive number, it blocks at most <em>timeout</em> seconds and raises the <a class="reference internal" href="queue.html#queue.Full" title="queue.Full"><code class="xref py py-exc docutils literal"><span class="pre">queue.Full</span></code></a> exception if no free slot was available within that time. </span><span class="yiyi-st" id="yiyi-224">Otherwise (<em>block</em> is <code class="docutils literal"><span class="pre">False</span></code>), put an item on the queue if a free slot is immediately available, else raise the <a class="reference internal" href="queue.html#queue.Full" title="queue.Full"><code class="xref py py-exc docutils literal"><span class="pre">queue.Full</span></code></a> exception (<em>timeout</em> is ignored in that case).</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.put_nowait"><span class="yiyi-st" id="yiyi-225"> <code class="descname">put_nowait</code><span class="sig-paren">(</span><em>obj</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-226">Equivalent to <code class="docutils literal"><span class="pre">put(obj,</span> <span class="pre">False)</span></code>.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.get"><span class="yiyi-st" id="yiyi-227"> <code class="descname">get</code><span class="sig-paren">(</span><span class="optional">[</span><em>block</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-228">Remove and return an item from the queue. </span><span class="yiyi-st" id="yiyi-229">如果可选的参数 <em>block</em>为<code class="docutils literal"><span class="pre">True</span></code>（默认值）和<em>超时</em>为<code class="docutils literal"><span class="pre">None</span></code>（默认值）将阻塞，直到队列里有项目可用。</span><span class="yiyi-st" id="yiyi-230">If <em>timeout</em> is a positive number, it blocks at most <em>timeout</em> seconds and raises the <a class="reference internal" href="queue.html#queue.Empty" title="queue.Empty"><code class="xref py py-exc docutils literal"><span class="pre">queue.Empty</span></code></a> exception if no item was available within that time. </span><span class="yiyi-st" id="yiyi-231">Otherwise (block is <code class="docutils literal"><span class="pre">False</span></code>), return an item if one is immediately available, else raise the <a class="reference internal" href="queue.html#queue.Empty" title="queue.Empty"><code class="xref py py-exc docutils literal"><span class="pre">queue.Empty</span></code></a> exception (<em>timeout</em> is ignored in that case).</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.get_nowait"><span class="yiyi-st" id="yiyi-232"> <code class="descname">get_nowait</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-233">Equivalent to <code class="docutils literal"><span class="pre">get(False)</span></code>.</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-234"><a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">multiprocessing.Queue</span></code></a> has a few additional methods not found in <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a>. </span><span class="yiyi-st" id="yiyi-235">These methods are usually unnecessary for most code:</span></p><dl class="method"><dt id="multiprocessing.Queue.close"><span class="yiyi-st" id="yiyi-236"> <code class="descname">close</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-237">Indicate that no more data will be put on this queue by the current process. </span><span class="yiyi-st" id="yiyi-238">The background thread will quit once it has flushed all buffered data to the pipe.</span><span class="yiyi-st" id="yiyi-239">This is called automatically when the queue is garbage collected.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.join_thread"><span class="yiyi-st" id="yiyi-240"> <code class="descname">join_thread</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-241">Join the background thread. </span><span class="yiyi-st" id="yiyi-242">This can only be used after <a class="reference internal" href="#multiprocessing.Queue.close" title="multiprocessing.Queue.close"><code class="xref py py-meth docutils literal"><span class="pre">close()</span></code></a> has been called. </span><span class="yiyi-st" id="yiyi-243">It blocks until the background thread exits, ensuring that all data in the buffer has been flushed to the pipe.</span></p><p><span class="yiyi-st" id="yiyi-244">By default if a process is not the creator of the queue then on exit it will attempt to join the queue’s background thread. </span><span class="yiyi-st" id="yiyi-245">The process can call <a class="reference internal" href="#multiprocessing.Queue.cancel_join_thread" title="multiprocessing.Queue.cancel_join_thread"><code class="xref py py-meth docutils literal"><span class="pre">cancel_join_thread()</span></code></a> to make <a class="reference internal" href="#multiprocessing.Queue.join_thread" title="multiprocessing.Queue.join_thread"><code class="xref py py-meth docutils literal"><span class="pre">join_thread()</span></code></a> do nothing.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Queue.cancel_join_thread"><span class="yiyi-st" id="yiyi-246"> <code class="descname">cancel_join_thread</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-247">Prevent <a class="reference internal" href="#multiprocessing.Queue.join_thread" title="multiprocessing.Queue.join_thread"><code class="xref py py-meth docutils literal"><span class="pre">join_thread()</span></code></a> from blocking. </span><span class="yiyi-st" id="yiyi-248">In particular, this prevents the background thread from being joined automatically when the process exits – see <a class="reference internal" href="#multiprocessing.Queue.join_thread" title="multiprocessing.Queue.join_thread"><code class="xref py py-meth docutils literal"><span class="pre">join_thread()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-249">A better name for this method might be <code class="docutils literal"><span class="pre">allow_exit_without_flush()</span></code>. </span><span class="yiyi-st" id="yiyi-250">It is likely to cause enqueued data to lost, and you almost certainly will not need to use it. </span><span class="yiyi-st" id="yiyi-251">It is really only there if you need the current process to exit immediately without waiting to flush enqueued data to the underlying pipe, and you don’t care about lost data.</span></p></dd></dl><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-252">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-253">This class’s functionality requires a functioning shared semaphore implementation on the host operating system. </span><span class="yiyi-st" id="yiyi-254">Without one, the functionality in this class will be disabled, and attempts to instantiate a <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> will result in an <a class="reference internal" href="exceptions.html#ImportError" title="ImportError"><code class="xref py py-exc docutils literal"><span class="pre">ImportError</span></code></a>. </span><span class="yiyi-st" id="yiyi-255">See <a class="reference external" href="https://bugs.python.org/issue3770">issue 3770</a> for additional information. </span><span class="yiyi-st" id="yiyi-256">The same holds true for any of the specialized queue types listed below.</span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.SimpleQueue"><span class="yiyi-st" id="yiyi-257"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">SimpleQueue</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-258">It is a simplified <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> type, very close to a locked <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-class docutils literal"><span class="pre">Pipe</span></code></a>.</span></p><dl class="method"><dt id="multiprocessing.SimpleQueue.empty"><span class="yiyi-st" id="yiyi-259"> <code class="descname">empty</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-260">Return <code class="docutils literal"><span class="pre">True</span></code> if the queue is empty, <code class="docutils literal"><span class="pre">False</span></code> otherwise.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.SimpleQueue.get"><span class="yiyi-st" id="yiyi-261"> <code class="descname">get</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-262">Remove and return an item from the queue.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.SimpleQueue.put"><span class="yiyi-st" id="yiyi-263"> <code class="descname">put</code><span class="sig-paren">(</span><em>item</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-264">Put <em>item</em> into the queue.</span></p></dd></dl></dd></dl><dl class="class"><dt id="multiprocessing.JoinableQueue"><span class="yiyi-st" id="yiyi-265"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">JoinableQueue</code><span class="sig-paren">(</span><span class="optional">[</span><em>maxsize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-266"><a class="reference internal" href="#multiprocessing.JoinableQueue" title="multiprocessing.JoinableQueue"><code class="xref py py-class docutils literal"><span class="pre">JoinableQueue</span></code></a>, a <a class="reference internal" href="#multiprocessing.Queue" title="multiprocessing.Queue"><code class="xref py py-class docutils literal"><span class="pre">Queue</span></code></a> subclass, is a queue which additionally has <a class="reference internal" href="#multiprocessing.JoinableQueue.task_done" title="multiprocessing.JoinableQueue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> and <a class="reference internal" href="#multiprocessing.JoinableQueue.join" title="multiprocessing.JoinableQueue.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a> methods.</span></p><dl class="method"><dt id="multiprocessing.JoinableQueue.task_done"><span class="yiyi-st" id="yiyi-267"> <code class="descname">task_done</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-268">Indicate that a formerly enqueued task is complete. </span><span class="yiyi-st" id="yiyi-269">Used by queue consumers. </span><span class="yiyi-st" id="yiyi-270">For each <a class="reference internal" href="#multiprocessing.Queue.get" title="multiprocessing.Queue.get"><code class="xref py py-meth docutils literal"><span class="pre">get()</span></code></a> used to fetch a task, a subsequent call to <a class="reference internal" href="#multiprocessing.JoinableQueue.task_done" title="multiprocessing.JoinableQueue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> tells the queue that the processing on the task is complete.</span></p><p><span class="yiyi-st" id="yiyi-271">If a <a class="reference internal" href="queue.html#queue.Queue.join" title="queue.Queue.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a> is currently blocking, it will resume when all items have been processed (meaning that a <a class="reference internal" href="#multiprocessing.JoinableQueue.task_done" title="multiprocessing.JoinableQueue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> call was received for every item that had been <a class="reference internal" href="#multiprocessing.Queue.put" title="multiprocessing.Queue.put"><code class="xref py py-meth docutils literal"><span class="pre">put()</span></code></a> into the queue).</span></p><p><span class="yiyi-st" id="yiyi-272">Raises a <a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a> if called more times than there were items placed in the queue.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.JoinableQueue.join"><span class="yiyi-st" id="yiyi-273"> <code class="descname">join</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-274">Block until all items in the queue have been gotten and processed.</span></p><p><span class="yiyi-st" id="yiyi-275">The count of unfinished tasks goes up whenever an item is added to the queue. </span><span class="yiyi-st" id="yiyi-276">The count goes down whenever a consumer calls <a class="reference internal" href="#multiprocessing.JoinableQueue.task_done" title="multiprocessing.JoinableQueue.task_done"><code class="xref py py-meth docutils literal"><span class="pre">task_done()</span></code></a> to indicate that the item was retrieved and all work on it is complete. </span><span class="yiyi-st" id="yiyi-277">When the count of unfinished tasks drops to zero, <a class="reference internal" href="queue.html#queue.Queue.join" title="queue.Queue.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a> unblocks.</span></p></dd></dl></dd></dl></div><div class="section" id="miscellaneous"><h3><span class="yiyi-st" id="yiyi-278">17.2.2.3. </span><span class="yiyi-st" id="yiyi-279">Miscellaneous</span></h3><dl class="function"><dt id="multiprocessing.active_children"><span class="yiyi-st" id="yiyi-280"> <code class="descclassname">multiprocessing.</code><code class="descname">active_children</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-281">Return list of all live children of the current process.</span></p><p><span class="yiyi-st" id="yiyi-282">Calling this has the side effect of “joining” any processes which have already finished.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.cpu_count"><span class="yiyi-st" id="yiyi-283"> <code class="descclassname">multiprocessing.</code><code class="descname">cpu_count</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-284">返回系统中的CPU数量。</span><span class="yiyi-st" id="yiyi-285">可能会引发<a class="reference internal" href="exceptions.html#NotImplementedError" title="NotImplementedError"><code class="xref py py-exc docutils literal"><span class="pre">NotImplementedError</span></code></a>。</span></p><div class="admonition seealso"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-286">See also</span></p><p class="last"><span class="yiyi-st" id="yiyi-287"><a class="reference internal" href="os.html#os.cpu_count" title="os.cpu_count"><code class="xref py py-func docutils literal"><span class="pre">os.cpu_count()</span></code></a></span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.current_process"><span class="yiyi-st" id="yiyi-288"> <code class="descclassname">multiprocessing.</code><code class="descname">current_process</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-289">Return the <a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a> object corresponding to the current process.</span></p><p><span class="yiyi-st" id="yiyi-290">An analogue of <a class="reference internal" href="threading.html#threading.current_thread" title="threading.current_thread"><code class="xref py py-func docutils literal"><span class="pre">threading.current_thread()</span></code></a>.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.freeze_support"><span class="yiyi-st" id="yiyi-291"> <code class="descclassname">multiprocessing.</code><code class="descname">freeze_support</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-292">Add support for when a program which uses <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> has been frozen to produce a Windows executable. </span><span class="yiyi-st" id="yiyi-293">(Has been tested with <strong>py2exe</strong>, <strong>PyInstaller</strong> and <strong>cx_Freeze</strong>.)</span></p><p><span class="yiyi-st" id="yiyi-294">One needs to call this function straight after the <code class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__'</span></code> line of the main module. </span><span class="yiyi-st" id="yiyi-295">For example:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">freeze_support</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">():</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'hello world!'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">freeze_support</span><span class="p">()</span>
<span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">)</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-296">If the <code class="docutils literal"><span class="pre">freeze_support()</span></code> line is omitted then trying to run the frozen executable will raise <a class="reference internal" href="exceptions.html#RuntimeError" title="RuntimeError"><code class="xref py py-exc docutils literal"><span class="pre">RuntimeError</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-297">Calling <code class="docutils literal"><span class="pre">freeze_support()</span></code> has no effect when invoked on any operating system other than Windows. </span><span class="yiyi-st" id="yiyi-298">In addition, if the module is being run normally by the Python interpreter on Windows (the program has not been frozen), then <code class="docutils literal"><span class="pre">freeze_support()</span></code> has no effect.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.get_all_start_methods"><span class="yiyi-st" id="yiyi-299"> <code class="descclassname">multiprocessing.</code><code class="descname">get_all_start_methods</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-300">Returns a list of the supported start methods, the first of which is the default. </span><span class="yiyi-st" id="yiyi-301">The possible start methods are <code class="docutils literal"><span class="pre">'fork'</span></code>, <code class="docutils literal"><span class="pre">'spawn'</span></code> and <code class="docutils literal"><span class="pre">'forkserver'</span></code>. </span><span class="yiyi-st" id="yiyi-302">On Windows only <code class="docutils literal"><span class="pre">'spawn'</span></code> is available. </span><span class="yiyi-st" id="yiyi-303">On Unix <code class="docutils literal"><span class="pre">'fork'</span></code> and <code class="docutils literal"><span class="pre">'spawn'</span></code> are always supported, with <code class="docutils literal"><span class="pre">'fork'</span></code> being the default.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-304"><span class="versionmodified">New in version 3.4.</span></span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.get_context"><span class="yiyi-st" id="yiyi-305"> <code class="descclassname">multiprocessing.</code><code class="descname">get_context</code><span class="sig-paren">(</span><em>method=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-306">Return a context object which has the same attributes as the <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> module.</span></p><p><span class="yiyi-st" id="yiyi-307">If <em>method</em> is <em>None</em> then the default context is returned. </span><span class="yiyi-st" id="yiyi-308">Otherwise <em>method</em> should be <code class="docutils literal"><span class="pre">'fork'</span></code>, <code class="docutils literal"><span class="pre">'spawn'</span></code>, <code class="docutils literal"><span class="pre">'forkserver'</span></code>. </span><span class="yiyi-st" id="yiyi-309"><a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a> is raised if the specified start method is not available.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-310"><span class="versionmodified">New in version 3.4.</span></span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.get_start_method"><span class="yiyi-st" id="yiyi-311"> <code class="descclassname">multiprocessing.</code><code class="descname">get_start_method</code><span class="sig-paren">(</span><em>allow_none=False</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-312">Return the name of start method used for starting processes.</span></p><p><span class="yiyi-st" id="yiyi-313">If the start method has not been fixed and <em>allow_none</em> is false, then the start method is fixed to the default and the name is returned. </span><span class="yiyi-st" id="yiyi-314">If the start method has not been fixed and <em>allow_none</em> is true then <em>None</em> is returned.</span></p><p><span class="yiyi-st" id="yiyi-315">The return value can be <code class="docutils literal"><span class="pre">'fork'</span></code>, <code class="docutils literal"><span class="pre">'spawn'</span></code>, <code class="docutils literal"><span class="pre">'forkserver'</span></code> or <em>None</em>. </span><span class="yiyi-st" id="yiyi-316"><code class="docutils literal"><span class="pre">'fork'</span></code> is the default on Unix, while <code class="docutils literal"><span class="pre">'spawn'</span></code> is the default on Windows.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-317"><span class="versionmodified">New in version 3.4.</span></span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.set_executable"><span class="yiyi-st" id="yiyi-318"> <code class="descclassname">multiprocessing.</code><code class="descname">set_executable</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-319">Sets the path of the Python interpreter to use when starting a child process. </span><span class="yiyi-st" id="yiyi-320">(By default <a class="reference internal" href="sys.html#sys.executable" title="sys.executable"><code class="xref py py-data docutils literal"><span class="pre">sys.executable</span></code></a> is used). </span><span class="yiyi-st" id="yiyi-321">Embedders will probably need to do some thing like</span></p><pre><code class="language-python"><span></span><span class="n">set_executable</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">exec_prefix</span><span class="p">,</span> <span class="s1">'pythonw.exe'</span><span class="p">))</span>
</code></pre><p><span class="yiyi-st" id="yiyi-322">before they can create child processes.</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-323"><span class="versionmodified">Changed in version 3.4: </span>Now supported on Unix when the <code class="docutils literal"><span class="pre">'spawn'</span></code> start method is used.</span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.set_start_method"><span class="yiyi-st" id="yiyi-324"> <code class="descclassname">multiprocessing.</code><code class="descname">set_start_method</code><span class="sig-paren">(</span><em>method</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-325">Set the method which should be used to start child processes. </span><span class="yiyi-st" id="yiyi-326"><em>method</em> can be <code class="docutils literal"><span class="pre">'fork'</span></code>, <code class="docutils literal"><span class="pre">'spawn'</span></code> or <code class="docutils literal"><span class="pre">'forkserver'</span></code>.</span></p><p><span class="yiyi-st" id="yiyi-327">Note that this should be called at most once, and it should be protected inside the <code class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__'</span></code> clause of the main module.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-328"><span class="versionmodified">New in version 3.4.</span></span></p></div></dd></dl><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-329">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-330"><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> contains no analogues of <a class="reference internal" href="threading.html#threading.active_count" title="threading.active_count"><code class="xref py py-func docutils literal"><span class="pre">threading.active_count()</span></code></a>, <a class="reference internal" href="threading.html#threading.enumerate" title="threading.enumerate"><code class="xref py py-func docutils literal"><span class="pre">threading.enumerate()</span></code></a>, <a class="reference internal" href="threading.html#threading.settrace" title="threading.settrace"><code class="xref py py-func docutils literal"><span class="pre">threading.settrace()</span></code></a>, <a class="reference internal" href="threading.html#threading.setprofile" title="threading.setprofile"><code class="xref py py-func docutils literal"><span class="pre">threading.setprofile()</span></code></a>, <a class="reference internal" href="threading.html#threading.Timer" title="threading.Timer"><code class="xref py py-class docutils literal"><span class="pre">threading.Timer</span></code></a>, or <a class="reference internal" href="threading.html#threading.local" title="threading.local"><code class="xref py py-class docutils literal"><span class="pre">threading.local</span></code></a>.</span></p></div></div><div class="section" id="connection-objects"><h3><span class="yiyi-st" id="yiyi-331">17.2.2.4. </span><span class="yiyi-st" id="yiyi-332">Connection Objects</span></h3><p><span class="yiyi-st" id="yiyi-333">Connection objects allow the sending and receiving of picklable objects or strings. </span><span class="yiyi-st" id="yiyi-334">They can be thought of as message oriented connected sockets.</span></p><p><span class="yiyi-st" id="yiyi-335">Connection objects are usually created using <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a> – see also <a class="reference internal" href="#multiprocessing-listeners-clients"><span>Listeners and Clients</span></a>.</span></p><dl class="class"><dt id="multiprocessing.Connection"><span class="yiyi-st" id="yiyi-336"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Connection</code></span></dt><dd><dl class="method"><dt id="multiprocessing.Connection.send"><span class="yiyi-st" id="yiyi-337"> <code class="descname">send</code><span class="sig-paren">(</span><em>obj</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-338">Send an object to the other end of the connection which should be read using <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">recv()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-339">The object must be picklable. </span><span class="yiyi-st" id="yiyi-340">Very large pickles (approximately 32 MB+, though it depends on the OS) may raise a <a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a> exception.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.recv"><span class="yiyi-st" id="yiyi-341"> <code class="descname">recv</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-342">Return an object sent from the other end of the connection using <a class="reference internal" href="#multiprocessing.Connection.send" title="multiprocessing.Connection.send"><code class="xref py py-meth docutils literal"><span class="pre">send()</span></code></a>. </span><span class="yiyi-st" id="yiyi-343">Blocks until there its something to receive. </span><span class="yiyi-st" id="yiyi-344">Raises <a class="reference internal" href="exceptions.html#EOFError" title="EOFError"><code class="xref py py-exc docutils literal"><span class="pre">EOFError</span></code></a> if there is nothing left to receive and the other end was closed.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.fileno"><span class="yiyi-st" id="yiyi-345"> <code class="descname">fileno</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-346">Return the file descriptor or handle used by the connection.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.close"><span class="yiyi-st" id="yiyi-347"> <code class="descname">close</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-348">Close the connection.</span></p><p><span class="yiyi-st" id="yiyi-349">This is called automatically when the connection is garbage collected.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.poll"><span class="yiyi-st" id="yiyi-350"> <code class="descname">poll</code><span class="sig-paren">(</span><span class="optional">[</span><em>timeout</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-351">Return whether there is any data available to be read.</span></p><p><span class="yiyi-st" id="yiyi-352">If <em>timeout</em> is not specified then it will return immediately. </span><span class="yiyi-st" id="yiyi-353">If <em>timeout</em> is a number then this specifies the maximum time in seconds to block. </span><span class="yiyi-st" id="yiyi-354">If <em>timeout</em> is <code class="docutils literal"><span class="pre">None</span></code> then an infinite timeout is used.</span></p><p><span class="yiyi-st" id="yiyi-355">Note that multiple connection objects may be polled at once by using <a class="reference internal" href="#multiprocessing.connection.wait" title="multiprocessing.connection.wait"><code class="xref py py-func docutils literal"><span class="pre">multiprocessing.connection.wait()</span></code></a>.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.send_bytes"><span class="yiyi-st" id="yiyi-356"> <code class="descname">send_bytes</code><span class="sig-paren">(</span><em>buffer</em><span class="optional">[</span>, <em>offset</em><span class="optional">[</span>, <em>size</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-357">Send byte data from a <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a> as a complete message.</span></p><p><span class="yiyi-st" id="yiyi-358">If <em>offset</em> is given then data is read from that position in <em>buffer</em>. </span><span class="yiyi-st" id="yiyi-359">If <em>size</em> is given then that many bytes will be read from buffer. </span><span class="yiyi-st" id="yiyi-360">Very large buffers (approximately 32 MB+, though it depends on the OS) may raise a <a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a> exception</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Connection.recv_bytes"><span class="yiyi-st" id="yiyi-361"> <code class="descname">recv_bytes</code><span class="sig-paren">(</span><span class="optional">[</span><em>maxlength</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-362">Return a complete message of byte data sent from the other end of the connection as a string. </span><span class="yiyi-st" id="yiyi-363">Blocks until there is something to receive. </span><span class="yiyi-st" id="yiyi-364">Raises <a class="reference internal" href="exceptions.html#EOFError" title="EOFError"><code class="xref py py-exc docutils literal"><span class="pre">EOFError</span></code></a> if there is nothing left to receive and the other end has closed.</span></p><p><span class="yiyi-st" id="yiyi-365">If <em>maxlength</em> is specified and the message is longer than <em>maxlength</em> then <a class="reference internal" href="exceptions.html#OSError" title="OSError"><code class="xref py py-exc docutils literal"><span class="pre">OSError</span></code></a> is raised and the connection will no longer be readable.</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-366"><span class="versionmodified">Changed in version 3.3: </span>This function used to raise <a class="reference internal" href="exceptions.html#IOError" title="IOError"><code class="xref py py-exc docutils literal"><span class="pre">IOError</span></code></a>, which is now an alias of <a class="reference internal" href="exceptions.html#OSError" title="OSError"><code class="xref py py-exc docutils literal"><span class="pre">OSError</span></code></a>.</span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.Connection.recv_bytes_into"><span class="yiyi-st" id="yiyi-367"> <code class="descname">recv_bytes_into</code><span class="sig-paren">(</span><em>buffer</em><span class="optional">[</span>, <em>offset</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-368">Read into <em>buffer</em> a complete message of byte data sent from the other end of the connection and return the number of bytes in the message. </span><span class="yiyi-st" id="yiyi-369">Blocks until there is something to receive. </span><span class="yiyi-st" id="yiyi-370">Raises <a class="reference internal" href="exceptions.html#EOFError" title="EOFError"><code class="xref py py-exc docutils literal"><span class="pre">EOFError</span></code></a> if there is nothing left to receive and the other end was closed.</span></p><p><span class="yiyi-st" id="yiyi-371"><em>buffer</em> must be a writable <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a>. </span><span class="yiyi-st" id="yiyi-372">If <em>offset</em> is given then the message will be written into the buffer from that position. </span><span class="yiyi-st" id="yiyi-373">Offset must be a non-negative integer less than the length of <em>buffer</em> (in bytes).</span></p><p><span class="yiyi-st" id="yiyi-374">If the buffer is too short then a <a class="reference internal" href="#multiprocessing.BufferTooShort" title="multiprocessing.BufferTooShort"><code class="xref py py-exc docutils literal"><span class="pre">BufferTooShort</span></code></a> exception is raised and the complete message is available as <code class="docutils literal"><span class="pre">e.args[0]</span></code> where <code class="docutils literal"><span class="pre">e</span></code> is the exception instance.</span></p></dd></dl><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-375"><span class="versionmodified">Changed in version 3.3: </span>Connection objects themselves can now be transferred between processes using <a class="reference internal" href="#multiprocessing.Connection.send" title="multiprocessing.Connection.send"><code class="xref py py-meth docutils literal"><span class="pre">Connection.send()</span></code></a> and <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">Connection.recv()</span></code></a>.</span></p></div><div class="versionadded"><p><span class="yiyi-st" id="yiyi-376"><span class="versionmodified">New in version 3.3: </span>Connection objects now support the context management protocol – see <a class="reference internal" href="stdtypes.html#typecontextmanager"><span>Context Manager Types</span></a>. </span><span class="yiyi-st" id="yiyi-377"><a class="reference internal" href="stdtypes.html#contextmanager.__enter__" title="contextmanager.__enter__"><code class="xref py py-meth docutils literal"><span class="pre">__enter__()</span></code></a> returns the connection object, and <a class="reference internal" href="stdtypes.html#contextmanager.__exit__" title="contextmanager.__exit__"><code class="xref py py-meth docutils literal"><span class="pre">__exit__()</span></code></a> calls <a class="reference internal" href="#multiprocessing.Connection.close" title="multiprocessing.Connection.close"><code class="xref py py-meth docutils literal"><span class="pre">close()</span></code></a>.</span></p></div></dd></dl><p><span class="yiyi-st" id="yiyi-378">For example:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Pipe</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">Pipe</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="o">.</span><span class="n">send</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'hello'</span><span class="p">,</span> <span class="kc">None</span><span class="p">])</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span><span class="o">.</span><span class="n">recv</span><span class="p">()</span>
<span class="go">[1, 'hello', None]</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span><span class="o">.</span><span class="n">send_bytes</span><span class="p">(</span><span class="n">b</span><span class="s1">'thank you'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="o">.</span><span class="n">recv_bytes</span><span class="p">()</span>
<span class="go">b'thank you'</span>
<span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">array</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">arr1</span> <span class="o">=</span> <span class="n">array</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">arr2</span> <span class="o">=</span> <span class="n">array</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="mi">10</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="o">.</span><span class="n">send_bytes</span><span class="p">(</span><span class="n">arr1</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">count</span> <span class="o">=</span> <span class="n">b</span><span class="o">.</span><span class="n">recv_bytes_into</span><span class="p">(</span><span class="n">arr2</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">assert</span> <span class="n">count</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">arr1</span><span class="p">)</span> <span class="o">*</span> <span class="n">arr1</span><span class="o">.</span><span class="n">itemsize</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">arr2</span>
<span class="go">array('i', [0, 1, 2, 3, 4, 0, 0, 0, 0, 0])</span>
</code></pre><div class="admonition warning"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-379">Warning</span></p><p><span class="yiyi-st" id="yiyi-380">The <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">Connection.recv()</span></code></a> method automatically unpickles the data it receives, which can be a security risk unless you can trust the process which sent the message.</span></p><p class="last"><span class="yiyi-st" id="yiyi-381">Therefore, unless the connection object was produced using <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a> you should only use the <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">recv()</span></code></a> and <a class="reference internal" href="#multiprocessing.Connection.send" title="multiprocessing.Connection.send"><code class="xref py py-meth docutils literal"><span class="pre">send()</span></code></a> methods after performing some sort of authentication. </span><span class="yiyi-st" id="yiyi-382">See <a class="reference internal" href="#multiprocessing-auth-keys"><span>Authentication keys</span></a>.</span></p></div><div class="admonition warning"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-383">Warning</span></p><p class="last"><span class="yiyi-st" id="yiyi-384">If a process is killed while it is trying to read or write to a pipe then the data in the pipe is likely to become corrupted, because it may become impossible to be sure where the message boundaries lie.</span></p></div></div><div class="section" id="synchronization-primitives"><h3><span class="yiyi-st" id="yiyi-385">17.2.2.5. </span><span class="yiyi-st" id="yiyi-386">同步原语</span></h3><p><span class="yiyi-st" id="yiyi-387">通常，同步原语在多进程程序中并不像在多线程程序中那样必要。</span><span class="yiyi-st" id="yiyi-388">参阅<a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a>模块的文档。</span></p><p><span class="yiyi-st" id="yiyi-389">注意，也可以使用Manager对象创建同步原语 - 请参阅<a class="reference internal" href="#multiprocessing-managers"><span>Managers</span></a>。</span></p><dl class="class"><dt id="multiprocessing.Barrier"><span class="yiyi-st" id="yiyi-390"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Barrier</code><span class="sig-paren">(</span><em>parties</em><span class="optional">[</span>, <em>action</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-391">A barrier object: a clone of <a class="reference internal" href="threading.html#threading.Barrier" title="threading.Barrier"><code class="xref py py-class docutils literal"><span class="pre">threading.Barrier</span></code></a>.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-392"><span class="versionmodified">New in version 3.3.</span></span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.BoundedSemaphore"><span class="yiyi-st" id="yiyi-393"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">BoundedSemaphore</code><span class="sig-paren">(</span><span class="optional">[</span><em>value</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-394">A bounded semaphore object: a close analog of <a class="reference internal" href="threading.html#threading.BoundedSemaphore" title="threading.BoundedSemaphore"><code class="xref py py-class docutils literal"><span class="pre">threading.BoundedSemaphore</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-395">A solitary difference from its close analog exists: its <code class="docutils literal"><span class="pre">acquire</span></code> method’s first argument is named <em>block</em>, as is consistent with <a class="reference internal" href="#multiprocessing.Lock.acquire" title="multiprocessing.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">Lock.acquire()</span></code></a>.</span></p><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-396">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-397">On Mac OS X, this is indistinguishable from <a class="reference internal" href="#multiprocessing.Semaphore" title="multiprocessing.Semaphore"><code class="xref py py-class docutils literal"><span class="pre">Semaphore</span></code></a> because <code class="docutils literal"><span class="pre">sem_getvalue()</span></code> is not implemented on that platform.</span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.Condition"><span class="yiyi-st" id="yiyi-398"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Condition</code><span class="sig-paren">(</span><span class="optional">[</span><em>lock</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-399">A condition variable: an alias for <a class="reference internal" href="threading.html#threading.Condition" title="threading.Condition"><code class="xref py py-class docutils literal"><span class="pre">threading.Condition</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-400">If <em>lock</em> is specified then it should be a <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> or <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> object from <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>.</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-401"><span class="versionmodified">Changed in version 3.3: </span>The <a class="reference internal" href="threading.html#threading.Condition.wait_for" title="threading.Condition.wait_for"><code class="xref py py-meth docutils literal"><span class="pre">wait_for()</span></code></a> method was added.</span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.Event"><span class="yiyi-st" id="yiyi-402"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Event</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-403">A clone of <a class="reference internal" href="threading.html#threading.Event" title="threading.Event"><code class="xref py py-class docutils literal"><span class="pre">threading.Event</span></code></a>.</span></p></dd></dl><dl class="class"><dt id="multiprocessing.Lock"><span class="yiyi-st" id="yiyi-404"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Lock</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-405">A non-recursive lock object: a close analog of <a class="reference internal" href="threading.html#threading.Lock" title="threading.Lock"><code class="xref py py-class docutils literal"><span class="pre">threading.Lock</span></code></a>. </span><span class="yiyi-st" id="yiyi-406">Once a process or thread has acquired a lock, subsequent attempts to acquire it from any process or thread will block until it is released; any process or thread may release it. </span><span class="yiyi-st" id="yiyi-407">The concepts and behaviors of <a class="reference internal" href="threading.html#threading.Lock" title="threading.Lock"><code class="xref py py-class docutils literal"><span class="pre">threading.Lock</span></code></a> as it applies to threads are replicated here in <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">multiprocessing.Lock</span></code></a> as it applies to either processes or threads, except as noted.</span></p><p><span class="yiyi-st" id="yiyi-408">Note that <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> is actually a factory function which returns an instance of <code class="docutils literal"><span class="pre">multiprocessing.synchronize.Lock</span></code> initialized with a default context.</span></p><p><span class="yiyi-st" id="yiyi-409"><a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> supports the <a class="reference internal" href="../glossary.html#term-context-manager"><span class="xref std std-term">context manager</span></a> protocol and thus may be used in <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal"><span class="pre">with</span></code></a> statements.</span></p><dl class="method"><dt id="multiprocessing.Lock.acquire"><span class="yiyi-st" id="yiyi-410"> <code class="descname">acquire</code><span class="sig-paren">(</span><em>block=True</em>, <em>timeout=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-411">Acquire a lock, blocking or non-blocking.</span></p><p><span class="yiyi-st" id="yiyi-412">With the <em>block</em> argument set to <code class="docutils literal"><span class="pre">True</span></code> (the default), the method call will block until the lock is in an unlocked state, then set it to locked and return <code class="docutils literal"><span class="pre">True</span></code>. </span><span class="yiyi-st" id="yiyi-413">Note that the name of this first argument differs from that in <a class="reference internal" href="threading.html#threading.Lock.acquire" title="threading.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">threading.Lock.acquire()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-414">With the <em>block</em> argument set to <code class="docutils literal"><span class="pre">False</span></code>, the method call does not block. </span><span class="yiyi-st" id="yiyi-415">If the lock is currently in a locked state, return <code class="docutils literal"><span class="pre">False</span></code>; otherwise set the lock to a locked state and return <code class="docutils literal"><span class="pre">True</span></code>.</span></p><p><span class="yiyi-st" id="yiyi-416">When invoked with a positive, floating-point value for <em>timeout</em>, block for at most the number of seconds specified by <em>timeout</em> as long as the lock can not be acquired. </span><span class="yiyi-st" id="yiyi-417">Invocations with a negative value for <em>timeout</em> are equivalent to a <em>timeout</em> of zero. </span><span class="yiyi-st" id="yiyi-418">Invocations with a <em>timeout</em> value of <code class="docutils literal"><span class="pre">None</span></code> (the default) set the timeout period to infinite. </span><span class="yiyi-st" id="yiyi-419">Note that the treatment of negative or <code class="docutils literal"><span class="pre">None</span></code> values for <em>timeout</em> differs from the implemented behavior in <a class="reference internal" href="threading.html#threading.Lock.acquire" title="threading.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">threading.Lock.acquire()</span></code></a>. </span><span class="yiyi-st" id="yiyi-420">The <em>timeout</em> argument has no practical implications if the <em>block</em> argument is set to <code class="docutils literal"><span class="pre">False</span></code> and is thus ignored. </span><span class="yiyi-st" id="yiyi-421">Returns <code class="docutils literal"><span class="pre">True</span></code> if the lock has been acquired or <code class="docutils literal"><span class="pre">False</span></code> if the timeout period has elapsed.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.Lock.release"><span class="yiyi-st" id="yiyi-422"> <code class="descname">release</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-423">Release a lock. </span><span class="yiyi-st" id="yiyi-424">This can be called from any process or thread, not only the process or thread which originally acquired the lock.</span></p><p><span class="yiyi-st" id="yiyi-425">Behavior is the same as in <a class="reference internal" href="threading.html#threading.Lock.release" title="threading.Lock.release"><code class="xref py py-meth docutils literal"><span class="pre">threading.Lock.release()</span></code></a> except that when invoked on an unlocked lock, a <a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a> is raised.</span></p></dd></dl></dd></dl><dl class="class"><dt id="multiprocessing.RLock"><span class="yiyi-st" id="yiyi-426"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">RLock</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-427">A recursive lock object: a close analog of <a class="reference internal" href="threading.html#threading.RLock" title="threading.RLock"><code class="xref py py-class docutils literal"><span class="pre">threading.RLock</span></code></a>. </span><span class="yiyi-st" id="yiyi-428">A recursive lock must be released by the process or thread that acquired it. </span><span class="yiyi-st" id="yiyi-429">Once a process or thread has acquired a recursive lock, the same process or thread may acquire it again without blocking; that process or thread must release it once for each time it has been acquired.</span></p><p><span class="yiyi-st" id="yiyi-430">Note that <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> is actually a factory function which returns an instance of <code class="docutils literal"><span class="pre">multiprocessing.synchronize.RLock</span></code> initialized with a default context.</span></p><p><span class="yiyi-st" id="yiyi-431"><a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> supports the <a class="reference internal" href="../glossary.html#term-context-manager"><span class="xref std std-term">context manager</span></a> protocol and thus may be used in <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal"><span class="pre">with</span></code></a> statements.</span></p><dl class="method"><dt id="multiprocessing.RLock.acquire"><span class="yiyi-st" id="yiyi-432"> <code class="descname">acquire</code><span class="sig-paren">(</span><em>block=True</em>, <em>timeout=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-433">Acquire a lock, blocking or non-blocking.</span></p><p><span class="yiyi-st" id="yiyi-434">When invoked with the <em>block</em> argument set to <code class="docutils literal"><span class="pre">True</span></code>, block until the lock is in an unlocked state (not owned by any process or thread) unless the lock is already owned by the current process or thread. </span><span class="yiyi-st" id="yiyi-435">The current process or thread then takes ownership of the lock (if it does not already have ownership) and the recursion level inside the lock increments by one, resulting in a return value of <code class="docutils literal"><span class="pre">True</span></code>. </span><span class="yiyi-st" id="yiyi-436">Note that there are several differences in this first argument’s behavior compared to the implementation of <a class="reference internal" href="threading.html#threading.RLock.acquire" title="threading.RLock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">threading.RLock.acquire()</span></code></a>, starting with the name of the argument itself.</span></p><p><span class="yiyi-st" id="yiyi-437">When invoked with the <em>block</em> argument set to <code class="docutils literal"><span class="pre">False</span></code>, do not block. </span><span class="yiyi-st" id="yiyi-438">If the lock has already been acquired (and thus is owned) by another process or thread, the current process or thread does not take ownership and the recursion level within the lock is not changed, resulting in a return value of <code class="docutils literal"><span class="pre">False</span></code>. </span><span class="yiyi-st" id="yiyi-439">If the lock is in an unlocked state, the current process or thread takes ownership and the recursion level is incremented, resulting in a return value of <code class="docutils literal"><span class="pre">True</span></code>.</span></p><p><span class="yiyi-st" id="yiyi-440">Use and behaviors of the <em>timeout</em> argument are the same as in <a class="reference internal" href="#multiprocessing.Lock.acquire" title="multiprocessing.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">Lock.acquire()</span></code></a>. </span><span class="yiyi-st" id="yiyi-441">Note that some of these behaviors of <em>timeout</em> differ from the implemented behaviors in <a class="reference internal" href="threading.html#threading.RLock.acquire" title="threading.RLock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">threading.RLock.acquire()</span></code></a>.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.RLock.release"><span class="yiyi-st" id="yiyi-442"> <code class="descname">release</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-443">释放一个锁，同时递归级别减一。</span><span class="yiyi-st" id="yiyi-444">如果递归级别递减到零，则将锁重置为unlocked状态（即不属于任何进程或线程），如果有任何其他进程或线程被阻塞，等待解锁，则准许其中一个进程继续。</span><span class="yiyi-st" id="yiyi-445">如果递归级别仍然不为零，则锁保持锁定并上交给调用进程或线程。</span></p><p><span class="yiyi-st" id="yiyi-446">只有在调用进程或线程拥有锁时才能调用此方法。</span><span class="yiyi-st" id="yiyi-447">如果此方法被所有者以外的进程或线程调用，或者锁处于解锁状态（无主），则会引发<a class="reference internal" href="exceptions.html#AssertionError" title="AssertionError"><code class="xref py py-exc docutils literal"><span class="pre">AssertionError</span></code></a>。</span><span class="yiyi-st" id="yiyi-448">请注意，在这种情况下引发的异常类型与<a class="reference internal" href="threading.html#threading.RLock.release" title="threading.RLock.release"><code class="xref py py-meth docutils literal"><span class="pre">threading.RLock.release()</span></code></a>中实现的方式有所区别。</span></p></dd></dl></dd></dl><dl class="class"><dt id="multiprocessing.Semaphore"><span class="yiyi-st" id="yiyi-449"> <em class="property">class </em><code class="descclassname">multiprocessing.</code><code class="descname">Semaphore</code><span class="sig-paren">(</span><span class="optional">[</span><em>value</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-450">A semaphore object: a close analog of <a class="reference internal" href="threading.html#threading.Semaphore" title="threading.Semaphore"><code class="xref py py-class docutils literal"><span class="pre">threading.Semaphore</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-451">A solitary difference from its close analog exists: its <code class="docutils literal"><span class="pre">acquire</span></code> method’s first argument is named <em>block</em>, as is consistent with <a class="reference internal" href="#multiprocessing.Lock.acquire" title="multiprocessing.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">Lock.acquire()</span></code></a>.</span></p></dd></dl><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-452">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-453">On Mac OS X, <code class="docutils literal"><span class="pre">sem_timedwait</span></code> is unsupported, so calling <code class="docutils literal"><span class="pre">acquire()</span></code> with a timeout will emulate that function’s behavior using a sleeping loop.</span></p></div><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-454">Note</span></p><p><span class="yiyi-st" id="yiyi-455">If the SIGINT signal generated by <code class="kbd docutils literal"><span class="pre">Ctrl-C</span></code> arrives while the main thread is blocked by a call to <code class="xref py py-meth docutils literal"><span class="pre">BoundedSemaphore.acquire()</span></code>, <a class="reference internal" href="#multiprocessing.Lock.acquire" title="multiprocessing.Lock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">Lock.acquire()</span></code></a>, <a class="reference internal" href="#multiprocessing.RLock.acquire" title="multiprocessing.RLock.acquire"><code class="xref py py-meth docutils literal"><span class="pre">RLock.acquire()</span></code></a>, <code class="xref py py-meth docutils literal"><span class="pre">Semaphore.acquire()</span></code>, <code class="xref py py-meth docutils literal"><span class="pre">Condition.acquire()</span></code> or <code class="xref py py-meth docutils literal"><span class="pre">Condition.wait()</span></code> then the call will be immediately interrupted and <a class="reference internal" href="exceptions.html#KeyboardInterrupt" title="KeyboardInterrupt"><code class="xref py py-exc docutils literal"><span class="pre">KeyboardInterrupt</span></code></a> will be raised.</span></p><p class="last"><span class="yiyi-st" id="yiyi-456">This differs from the behaviour of <a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a> where SIGINT will be ignored while the equivalent blocking calls are in progress.</span></p></div><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-457">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-458">Some of this package’s functionality requires a functioning shared semaphore implementation on the host operating system. </span><span class="yiyi-st" id="yiyi-459">Without one, the <code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.synchronize</span></code> module will be disabled, and attempts to import it will result in an <a class="reference internal" href="exceptions.html#ImportError" title="ImportError"><code class="xref py py-exc docutils literal"><span class="pre">ImportError</span></code></a>. </span><span class="yiyi-st" id="yiyi-460">See <a class="reference external" href="https://bugs.python.org/issue3770">issue 3770</a> for additional information.</span></p></div></div><div class="section" id="shared-ctypes-objects"><h3><span class="yiyi-st" id="yiyi-461">17.2.2.6. </span><span class="yiyi-st" id="yiyi-462">Shared <a class="reference internal" href="ctypes.html#module-ctypes" title="ctypes: A foreign function library for Python."><code class="xref py py-mod docutils literal"><span class="pre">ctypes</span></code></a> Objects</span></h3><p><span class="yiyi-st" id="yiyi-463">It is possible to create shared objects using shared memory which can be inherited by child processes.</span></p><dl class="function"><dt id="multiprocessing.Value"><span class="yiyi-st" id="yiyi-464"> <code class="descclassname">multiprocessing.</code><code class="descname">Value</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>*args</em>, <em>lock=True</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-465">返回从共享内存分配的<a class="reference internal" href="ctypes.html#module-ctypes" title="ctypes: A foreign function library for Python."><code class="xref py py-mod docutils literal"><span class="pre">ctypes</span></code></a>对象。</span><span class="yiyi-st" id="yiyi-466">By default the return value is actually a synchronized wrapper for the object. </span><span class="yiyi-st" id="yiyi-467">The object itself can be accessed via the <em>value</em> attribute of a <a class="reference internal" href="#multiprocessing.Value" title="multiprocessing.Value"><code class="xref py py-class docutils literal"><span class="pre">Value</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-468"><em>typecode_or_type</em> determines the type of the returned object: it is either a ctypes type or a one character typecode of the kind used by the <a class="reference internal" href="array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal"><span class="pre">array</span></code></a> module. </span><span class="yiyi-st" id="yiyi-469"><em>*args</em> is passed on to the constructor for the type.</span></p><p><span class="yiyi-st" id="yiyi-470">If <em>lock</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) then a new recursive lock object is created to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-471">If <em>lock</em> is a <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> or <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> object then that will be used to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-472">If <em>lock</em> is <code class="docutils literal"><span class="pre">False</span></code> then access to the returned object will not be automatically protected by a lock, so it will not necessarily be “process-safe”.</span></p><p><span class="yiyi-st" id="yiyi-473">Operations like <code class="docutils literal"><span class="pre">+=</span></code> which involve a read and write are not atomic. </span><span class="yiyi-st" id="yiyi-474">So if, for instance, you want to atomically increment a shared value it is insufficient to just do</span></p><pre><code class="language-python"><span></span><span class="n">counter</span><span class="o">.</span><span class="n">value</span> <span class="o">+=</span> <span class="mi">1</span>
</code></pre><p><span class="yiyi-st" id="yiyi-475">Assuming the associated lock is recursive (which it is by default) you can instead do</span></p><pre><code class="language-python"><span></span><span class="k">with</span> <span class="n">counter</span><span class="o">.</span><span class="n">get_lock</span><span class="p">():</span>
<span class="n">counter</span><span class="o">.</span><span class="n">value</span> <span class="o">+=</span> <span class="mi">1</span>
</code></pre><p><span class="yiyi-st" id="yiyi-476">Note that <em>lock</em> is a keyword-only argument.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.Array"><span class="yiyi-st" id="yiyi-477"> <code class="descclassname">multiprocessing.</code><code class="descname">Array</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>size_or_initializer</em>, <em>*</em>, <em>lock=True</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-478">Return a ctypes array allocated from shared memory. </span><span class="yiyi-st" id="yiyi-479">By default the return value is actually a synchronized wrapper for the array.</span></p><p><span class="yiyi-st" id="yiyi-480"><em>typecode_or_type</em> determines the type of the elements of the returned array: it is either a ctypes type or a one character typecode of the kind used by the <a class="reference internal" href="array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal"><span class="pre">array</span></code></a> module. </span><span class="yiyi-st" id="yiyi-481">If <em>size_or_initializer</em> is an integer, then it determines the length of the array, and the array will be initially zeroed. </span><span class="yiyi-st" id="yiyi-482">Otherwise, <em>size_or_initializer</em> is a sequence which is used to initialize the array and whose length determines the length of the array.</span></p><p><span class="yiyi-st" id="yiyi-483">If <em>lock</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) then a new lock object is created to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-484">If <em>lock</em> is a <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> or <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> object then that will be used to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-485">If <em>lock</em> is <code class="docutils literal"><span class="pre">False</span></code> then access to the returned object will not be automatically protected by a lock, so it will not necessarily be “process-safe”.</span></p><p><span class="yiyi-st" id="yiyi-486">Note that <em>lock</em> is a keyword only argument.</span></p><p><span class="yiyi-st" id="yiyi-487">Note that an array of <a class="reference internal" href="ctypes.html#ctypes.c_char" title="ctypes.c_char"><code class="xref py py-data docutils literal"><span class="pre">ctypes.c_char</span></code></a> has <em>value</em> and <em>raw</em> attributes which allow one to use it to store and retrieve strings.</span></p></dd></dl><div class="section" id="module-multiprocessing.sharedctypes"><h4><span class="yiyi-st" id="yiyi-488">17.2.2.6.1. </span><span class="yiyi-st" id="yiyi-489">The <a class="reference internal" href="#module-multiprocessing.sharedctypes" title="multiprocessing.sharedctypes: Allocate ctypes objects from shared memory."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.sharedctypes</span></code></a> module</span></h4><p><span class="yiyi-st" id="yiyi-490">The <a class="reference internal" href="#module-multiprocessing.sharedctypes" title="multiprocessing.sharedctypes: Allocate ctypes objects from shared memory."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.sharedctypes</span></code></a> module provides functions for allocating <a class="reference internal" href="ctypes.html#module-ctypes" title="ctypes: A foreign function library for Python."><code class="xref py py-mod docutils literal"><span class="pre">ctypes</span></code></a> objects from shared memory which can be inherited by child processes.</span></p><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-491">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-492">Although it is possible to store a pointer in shared memory remember that this will refer to a location in the address space of a specific process. </span><span class="yiyi-st" id="yiyi-493">However, the pointer is quite likely to be invalid in the context of a second process and trying to dereference the pointer from the second process may cause a crash.</span></p></div><dl class="function"><dt id="multiprocessing.sharedctypes.RawArray"><span class="yiyi-st" id="yiyi-494"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">RawArray</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>size_or_initializer</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-495">Return a ctypes array allocated from shared memory.</span></p><p><span class="yiyi-st" id="yiyi-496"><em>typecode_or_type</em> determines the type of the elements of the returned array: it is either a ctypes type or a one character typecode of the kind used by the <a class="reference internal" href="array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal"><span class="pre">array</span></code></a> module. </span><span class="yiyi-st" id="yiyi-497">If <em>size_or_initializer</em> is an integer then it determines the length of the array, and the array will be initially zeroed. </span><span class="yiyi-st" id="yiyi-498">Otherwise <em>size_or_initializer</em> is a sequence which is used to initialize the array and whose length determines the length of the array.</span></p><p><span class="yiyi-st" id="yiyi-499">Note that setting and getting an element is potentially non-atomic – use <a class="reference internal" href="#multiprocessing.sharedctypes.Array" title="multiprocessing.sharedctypes.Array"><code class="xref py py-func docutils literal"><span class="pre">Array()</span></code></a> instead to make sure that access is automatically synchronized using a lock.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.sharedctypes.RawValue"><span class="yiyi-st" id="yiyi-500"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">RawValue</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>*args</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-501">Return a ctypes object allocated from shared memory.</span></p><p><span class="yiyi-st" id="yiyi-502"><em>typecode_or_type</em> determines the type of the returned object: it is either a ctypes type or a one character typecode of the kind used by the <a class="reference internal" href="array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal"><span class="pre">array</span></code></a> module. </span><span class="yiyi-st" id="yiyi-503"><em>*args</em> is passed on to the constructor for the type.</span></p><p><span class="yiyi-st" id="yiyi-504">Note that setting and getting the value is potentially non-atomic – use <a class="reference internal" href="#multiprocessing.sharedctypes.Value" title="multiprocessing.sharedctypes.Value"><code class="xref py py-func docutils literal"><span class="pre">Value()</span></code></a> instead to make sure that access is automatically synchronized using a lock.</span></p><p><span class="yiyi-st" id="yiyi-505">Note that an array of <a class="reference internal" href="ctypes.html#ctypes.c_char" title="ctypes.c_char"><code class="xref py py-data docutils literal"><span class="pre">ctypes.c_char</span></code></a> has <code class="docutils literal"><span class="pre">value</span></code> and <code class="docutils literal"><span class="pre">raw</span></code> attributes which allow one to use it to store and retrieve strings – see documentation for <a class="reference internal" href="ctypes.html#module-ctypes" title="ctypes: A foreign function library for Python."><code class="xref py py-mod docutils literal"><span class="pre">ctypes</span></code></a>.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.sharedctypes.Array"><span class="yiyi-st" id="yiyi-506"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">Array</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>size_or_initializer</em>, <em>*</em>, <em>lock=True</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-507">The same as <a class="reference internal" href="#multiprocessing.sharedctypes.RawArray" title="multiprocessing.sharedctypes.RawArray"><code class="xref py py-func docutils literal"><span class="pre">RawArray()</span></code></a> except that depending on the value of <em>lock</em> a process-safe synchronization wrapper may be returned instead of a raw ctypes array.</span></p><p><span class="yiyi-st" id="yiyi-508">If <em>lock</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) then a new lock object is created to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-509">If <em>lock</em> is a <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> or <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> object then that will be used to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-510">If <em>lock</em> is <code class="docutils literal"><span class="pre">False</span></code> then access to the returned object will not be automatically protected by a lock, so it will not necessarily be “process-safe”.</span></p><p><span class="yiyi-st" id="yiyi-511">请注意，<em>lock</em>是一个强制关键字参数。</span></p></dd></dl><dl class="function"><dt id="multiprocessing.sharedctypes.Value"><span class="yiyi-st" id="yiyi-512"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">Value</code><span class="sig-paren">(</span><em>typecode_or_type</em>, <em>*args</em>, <em>lock=True</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-513">The same as <a class="reference internal" href="#multiprocessing.sharedctypes.RawValue" title="multiprocessing.sharedctypes.RawValue"><code class="xref py py-func docutils literal"><span class="pre">RawValue()</span></code></a> except that depending on the value of <em>lock</em> a process-safe synchronization wrapper may be returned instead of a raw ctypes object.</span></p><p><span class="yiyi-st" id="yiyi-514">If <em>lock</em> is <code class="docutils literal"><span class="pre">True</span></code> (the default) then a new lock object is created to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-515">If <em>lock</em> is a <a class="reference internal" href="#multiprocessing.Lock" title="multiprocessing.Lock"><code class="xref py py-class docutils literal"><span class="pre">Lock</span></code></a> or <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">RLock</span></code></a> object then that will be used to synchronize access to the value. </span><span class="yiyi-st" id="yiyi-516">If <em>lock</em> is <code class="docutils literal"><span class="pre">False</span></code> then access to the returned object will not be automatically protected by a lock, so it will not necessarily be “process-safe”.</span></p><p><span class="yiyi-st" id="yiyi-517">请注意，<em>lock</em>是一个强制关键字参数。</span></p></dd></dl><dl class="function"><dt id="multiprocessing.sharedctypes.copy"><span class="yiyi-st" id="yiyi-518"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">copy</code><span class="sig-paren">(</span><em>obj</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-519">Return a ctypes object allocated from shared memory which is a copy of the ctypes object <em>obj</em>.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.sharedctypes.synchronized"><span class="yiyi-st" id="yiyi-520"> <code class="descclassname">multiprocessing.sharedctypes.</code><code class="descname">synchronized</code><span class="sig-paren">(</span><em>obj</em><span class="optional">[</span>, <em>lock</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-521">Return a process-safe wrapper object for a ctypes object which uses <em>lock</em> to synchronize access. </span><span class="yiyi-st" id="yiyi-522">If <em>lock</em> is <code class="docutils literal"><span class="pre">None</span></code> (the default) then a <a class="reference internal" href="#multiprocessing.RLock" title="multiprocessing.RLock"><code class="xref py py-class docutils literal"><span class="pre">multiprocessing.RLock</span></code></a> object is created automatically.</span></p><p><span class="yiyi-st" id="yiyi-523">A synchronized wrapper will have two methods in addition to those of the object it wraps: <code class="xref py py-meth docutils literal"><span class="pre">get_obj()</span></code> returns the wrapped object and <code class="xref py py-meth docutils literal"><span class="pre">get_lock()</span></code> returns the lock object used for synchronization.</span></p><p><span class="yiyi-st" id="yiyi-524">Note that accessing the ctypes object through the wrapper can be a lot slower than accessing the raw ctypes object.</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-525"><span class="versionmodified">Changed in version 3.5: </span>Synchronized objects support the <a class="reference internal" href="../glossary.html#term-context-manager"><span class="xref std std-term">context manager</span></a> protocol.</span></p></div></dd></dl><p><span class="yiyi-st" id="yiyi-526">The table below compares the syntax for creating shared ctypes objects from shared memory with the normal ctypes syntax. </span><span class="yiyi-st" id="yiyi-527">(In the table <code class="docutils literal"><span class="pre">MyStruct</span></code> is some subclass of <a class="reference internal" href="ctypes.html#ctypes.Structure" title="ctypes.Structure"><code class="xref py py-class docutils literal"><span class="pre">ctypes.Structure</span></code></a>.)</span></p><table border="1" class="docutils"><thead valign="bottom"><tr class="row-odd"><th class="head"><span class="yiyi-st" id="yiyi-528">ctypes</span></th><th class="head"><span class="yiyi-st" id="yiyi-529">sharedctypes using type</span></th><th class="head"><span class="yiyi-st" id="yiyi-530">sharedctypes using typecode</span></th></tr></thead><tbody valign="top"><tr class="row-even"><td><span class="yiyi-st" id="yiyi-531">c_double(2.4)</span></td><td><span class="yiyi-st" id="yiyi-532">RawValue(c_double, 2.4)</span></td><td><span class="yiyi-st" id="yiyi-533">RawValue(‘d’, 2.4)</span></td></tr><tr class="row-odd"><td><span class="yiyi-st" id="yiyi-534">MyStruct(4, 6)</span></td><td><span class="yiyi-st" id="yiyi-535">RawValue(MyStruct, 4, 6)</span></td><td></td></tr><tr class="row-even"><td><span class="yiyi-st" id="yiyi-536">(c_short * 7)()</span></td><td><span class="yiyi-st" id="yiyi-537">RawArray(c_short, 7)</span></td><td><span class="yiyi-st" id="yiyi-538">RawArray(‘h’, 7)</span></td></tr><tr class="row-odd"><td><span class="yiyi-st" id="yiyi-539">(c_int * 3)(9, 2, 8)</span></td><td><span class="yiyi-st" id="yiyi-540">RawArray(c_int, (9, 2, 8))</span></td><td><span class="yiyi-st" id="yiyi-541">RawArray(‘i’, (9, 2, 8))</span></td></tr></tbody></table><p><span class="yiyi-st" id="yiyi-542">Below is an example where a number of ctypes objects are modified by a child process:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Lock</span>
<span class="kn">from</span> <span class="nn">multiprocessing.sharedctypes</span> <span class="k">import</span> <span class="n">Value</span><span class="p">,</span> <span class="n">Array</span>
<span class="kn">from</span> <span class="nn">ctypes</span> <span class="k">import</span> <span class="n">Structure</span><span class="p">,</span> <span class="n">c_double</span>
<span class="k">class</span> <span class="nc">Point</span><span class="p">(</span><span class="n">Structure</span><span class="p">):</span>
<span class="n">_fields_</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">'x'</span><span class="p">,</span> <span class="n">c_double</span><span class="p">),</span> <span class="p">(</span><span class="s1">'y'</span><span class="p">,</span> <span class="n">c_double</span><span class="p">)]</span>
<span class="k">def</span> <span class="nf">modify</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">s</span><span class="p">,</span> <span class="n">A</span><span class="p">):</span>
<span class="n">n</span><span class="o">.</span><span class="n">value</span> <span class="o">**=</span> <span class="mi">2</span>
<span class="n">x</span><span class="o">.</span><span class="n">value</span> <span class="o">**=</span> <span class="mi">2</span>
<span class="n">s</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="n">s</span><span class="o">.</span><span class="n">value</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
<span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">A</span><span class="p">:</span>
<span class="n">a</span><span class="o">.</span><span class="n">x</span> <span class="o">**=</span> <span class="mi">2</span>
<span class="n">a</span><span class="o">.</span><span class="n">y</span> <span class="o">**=</span> <span class="mi">2</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">lock</span> <span class="o">=</span> <span class="n">Lock</span><span class="p">()</span>
<span class="n">n</span> <span class="o">=</span> <span class="n">Value</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">Value</span><span class="p">(</span><span class="n">c_double</span><span class="p">,</span> <span class="mf">1.0</span><span class="o">/</span><span class="mf">3.0</span><span class="p">,</span> <span class="n">lock</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">Array</span><span class="p">(</span><span class="s1">'c'</span><span class="p">,</span> <span class="n">b</span><span class="s1">'hello world'</span><span class="p">,</span> <span class="n">lock</span><span class="o">=</span><span class="n">lock</span><span class="p">)</span>
<span class="n">A</span> <span class="o">=</span> <span class="n">Array</span><span class="p">(</span><span class="n">Point</span><span class="p">,</span> <span class="p">[(</span><span class="mf">1.875</span><span class="p">,</span><span class="o">-</span><span class="mf">6.25</span><span class="p">),</span> <span class="p">(</span><span class="o">-</span><span class="mf">5.75</span><span class="p">,</span><span class="mf">2.0</span><span class="p">),</span> <span class="p">(</span><span class="mf">2.375</span><span class="p">,</span><span class="mf">9.5</span><span class="p">)],</span> <span class="n">lock</span><span class="o">=</span><span class="n">lock</span><span class="p">)</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">modify</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">s</span><span class="p">,</span> <span class="n">A</span><span class="p">))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">n</span><span class="o">.</span><span class="n">value</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">value</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">value</span><span class="p">)</span>
<span class="nb">print</span><span class="p">([(</span><span class="n">a</span><span class="o">.</span><span class="n">x</span><span class="p">,</span> <span class="n">a</span><span class="o">.</span><span class="n">y</span><span class="p">)</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">A</span><span class="p">])</span>
</code></pre><p><span class="yiyi-st" id="yiyi-543">The results printed are</span></p><div class="highlight-none"><div class="highlight"><pre><span></span>49
0.1111111111111111
HELLO WORLD
[(3.515625, 39.0625), (33.0625, 4.0), (5.640625, 90.25)]
</pre></div></div></div></div><div class="section" id="managers"><h3><span class="yiyi-st" id="yiyi-544">17.2.2.7. </span><span class="yiyi-st" id="yiyi-545">Managers</span></h3><p><span class="yiyi-st" id="yiyi-546">Managers提供了一种方法来创建可以在不同进程之间共享的数据，包括通过网络在不同机器上运行的进程之间共享。</span><span class="yiyi-st" id="yiyi-547">manager对象控制着管理<em>共享对象</em>的服务器进程。</span><span class="yiyi-st" id="yiyi-548">其他进程可以通过使用代理来访问共享对象。</span></p><dl class="function"><dt id="multiprocessing.sharedctypes.multiprocessing.Manager"><span class="yiyi-st" id="yiyi-549"> <code class="descclassname">multiprocessing.</code><code class="descname">Manager</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-550">Returns a started <a class="reference internal" href="#multiprocessing.managers.SyncManager" title="multiprocessing.managers.SyncManager"><code class="xref py py-class docutils literal"><span class="pre">SyncManager</span></code></a> object which can be used for sharing objects between processes. </span><span class="yiyi-st" id="yiyi-551">The returned manager object corresponds to a spawned child process and has methods which will create shared objects and return corresponding proxies.</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-552">Manager processes will be shutdown as soon as they are garbage collected or their parent process exits. </span><span class="yiyi-st" id="yiyi-553">The manager classes are defined in the <a class="reference internal" href="#module-multiprocessing.managers" title="multiprocessing.managers: Share data between process with shared objects."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.managers</span></code></a> module:</span></p><dl class="class"><dt id="multiprocessing.managers.BaseManager"><span class="yiyi-st" id="yiyi-554"> <em class="property">class </em><code class="descclassname">multiprocessing.managers.</code><code class="descname">BaseManager</code><span class="sig-paren">(</span><span class="optional">[</span><em>address</em><span class="optional">[</span>, <em>authkey</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-555">Create a BaseManager object.</span></p><p><span class="yiyi-st" id="yiyi-556">Once created one should call <a class="reference internal" href="#multiprocessing.managers.BaseManager.start" title="multiprocessing.managers.BaseManager.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a> or <code class="docutils literal"><span class="pre">get_server().serve_forever()</span></code> to ensure that the manager object refers to a started manager process.</span></p><p><span class="yiyi-st" id="yiyi-557"><em>address</em>是管理器进程侦听新连接的地址。</span><span class="yiyi-st" id="yiyi-558">如果<em>address</em>为<code class="docutils literal"><span class="pre">None</span></code>，则选择任意一个。</span></p><p><span class="yiyi-st" id="yiyi-559"><em>authkey</em> is the authentication key which will be used to check the validity of incoming connections to the server process. </span><span class="yiyi-st" id="yiyi-560">If <em>authkey</em> is <code class="docutils literal"><span class="pre">None</span></code> then <code class="docutils literal"><span class="pre">current_process().authkey</span></code> is used. </span><span class="yiyi-st" id="yiyi-561">Otherwise <em>authkey</em> is used and it must be a byte string.</span></p><dl class="method"><dt id="multiprocessing.managers.BaseManager.start"><span class="yiyi-st" id="yiyi-562"> <code class="descname">start</code><span class="sig-paren">(</span><span class="optional">[</span><em>initializer</em><span class="optional">[</span>, <em>initargs</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-563">Start a subprocess to start the manager. </span><span class="yiyi-st" id="yiyi-564">If <em>initializer</em> is not <code class="docutils literal"><span class="pre">None</span></code> then the subprocess will call <code class="docutils literal"><span class="pre">initializer(*initargs)</span></code> when it starts.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseManager.get_server"><span class="yiyi-st" id="yiyi-565"> <code class="descname">get_server</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-566">Returns a <code class="xref py py-class docutils literal"><span class="pre">Server</span></code> object which represents the actual server under the control of the Manager. </span><span class="yiyi-st" id="yiyi-567">The <code class="xref py py-class docutils literal"><span class="pre">Server</span></code> object supports the <code class="xref py py-meth docutils literal"><span class="pre">serve_forever()</span></code> method:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">manager</span> <span class="o">=</span> <span class="n">BaseManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">''</span><span class="p">,</span> <span class="mi">50000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abc'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">server</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">get_server</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">server</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-568"><code class="xref py py-class docutils literal"><span class="pre">Server</span></code> additionally has an <a class="reference internal" href="#multiprocessing.managers.BaseManager.address" title="multiprocessing.managers.BaseManager.address"><code class="xref py py-attr docutils literal"><span class="pre">address</span></code></a> attribute.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseManager.connect"><span class="yiyi-st" id="yiyi-569"> <code class="descname">connect</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-570">Connect a local manager object to a remote manager process:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">BaseManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">'127.0.0.1'</span><span class="p">,</span> <span class="mi">5000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abc'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span>
</code></pre></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseManager.shutdown"><span class="yiyi-st" id="yiyi-571"> <code class="descname">shutdown</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-572">Stop the process used by the manager. </span><span class="yiyi-st" id="yiyi-573">This is only available if <a class="reference internal" href="#multiprocessing.managers.BaseManager.start" title="multiprocessing.managers.BaseManager.start"><code class="xref py py-meth docutils literal"><span class="pre">start()</span></code></a> has been used to start the server process.</span></p><p><span class="yiyi-st" id="yiyi-574">This can be called multiple times.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseManager.register"><span class="yiyi-st" id="yiyi-575"> <code class="descname">register</code><span class="sig-paren">(</span><em>typeid</em><span class="optional">[</span>, <em>callable</em><span class="optional">[</span>, <em>proxytype</em><span class="optional">[</span>, <em>exposed</em><span class="optional">[</span>, <em>method_to_typeid</em><span class="optional">[</span>, <em>create_method</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-576">类方法，用于注册类型或者可调用的manager类</span></p><p><span class="yiyi-st" id="yiyi-577"><em>typeid</em> is a “type identifier” which is used to identify a particular type of shared object. </span><span class="yiyi-st" id="yiyi-578">This must be a string.</span></p><p><span class="yiyi-st" id="yiyi-579"><em>callable</em> is a callable used for creating objects for this type identifier. </span><span class="yiyi-st" id="yiyi-580">If a manager instance will be connected to the server using the <a class="reference internal" href="#multiprocessing.managers.BaseManager.connect" title="multiprocessing.managers.BaseManager.connect"><code class="xref py py-meth docutils literal"><span class="pre">connect()</span></code></a> method, or if the <em>create_method</em> argument is <code class="docutils literal"><span class="pre">False</span></code> then this can be left as <code class="docutils literal"><span class="pre">None</span></code>.</span></p><p><span class="yiyi-st" id="yiyi-581"><em>proxytype</em> is a subclass of <a class="reference internal" href="#multiprocessing.managers.BaseProxy" title="multiprocessing.managers.BaseProxy"><code class="xref py py-class docutils literal"><span class="pre">BaseProxy</span></code></a> which is used to create proxies for shared objects with this <em>typeid</em>. </span><span class="yiyi-st" id="yiyi-582">If <code class="docutils literal"><span class="pre">None</span></code> then a proxy class is created automatically.</span></p><p><span class="yiyi-st" id="yiyi-583"><em>exposed</em> is used to specify a sequence of method names which proxies for this typeid should be allowed to access using <a class="reference internal" href="#multiprocessing.managers.BaseProxy._callmethod" title="multiprocessing.managers.BaseProxy._callmethod"><code class="xref py py-meth docutils literal"><span class="pre">BaseProxy._callmethod()</span></code></a>. </span><span class="yiyi-st" id="yiyi-584">(If <em>exposed</em> is <code class="docutils literal"><span class="pre">None</span></code> then <code class="xref py py-attr docutils literal"><span class="pre">proxytype._exposed_</span></code> is used instead if it exists.) </span><span class="yiyi-st" id="yiyi-585">In the case where no exposed list is specified, all “public methods” of the shared object will be accessible. </span><span class="yiyi-st" id="yiyi-586">(Here a “public method” means any attribute which has a <a class="reference internal" href="../reference/datamodel.html#object.__call__" title="object.__call__"><code class="xref py py-meth docutils literal"><span class="pre">__call__()</span></code></a> method and whose name does not begin with <code class="docutils literal"><span class="pre">'_'</span></code>.)</span></p><p><span class="yiyi-st" id="yiyi-587"><em>method_to_typeid</em> is a mapping used to specify the return type of those exposed methods which should return a proxy. </span><span class="yiyi-st" id="yiyi-588">It maps method names to typeid strings. </span><span class="yiyi-st" id="yiyi-589">(If <em>method_to_typeid</em> is <code class="docutils literal"><span class="pre">None</span></code> then <code class="xref py py-attr docutils literal"><span class="pre">proxytype._method_to_typeid_</span></code> is used instead if it exists.) </span><span class="yiyi-st" id="yiyi-590">If a method’s name is not a key of this mapping or if the mapping is <code class="docutils literal"><span class="pre">None</span></code> then the object returned by the method will be copied by value.</span></p><p><span class="yiyi-st" id="yiyi-591"><em>create_method</em> determines whether a method should be created with name <em>typeid</em> which can be used to tell the server process to create a new shared object and return a proxy for it. </span><span class="yiyi-st" id="yiyi-592">By default it is <code class="docutils literal"><span class="pre">True</span></code>.</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-593"><a class="reference internal" href="#multiprocessing.managers.BaseManager" title="multiprocessing.managers.BaseManager"><code class="xref py py-class docutils literal"><span class="pre">BaseManager</span></code></a> instances also have one read-only property:</span></p><dl class="attribute"><dt id="multiprocessing.managers.BaseManager.address"><span class="yiyi-st" id="yiyi-594"> <code class="descname">address</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-595">管理器使用的地址。</span></p></dd></dl><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-596"><span class="versionmodified">Changed in version 3.3: </span>Manager objects support the context management protocol – see <a class="reference internal" href="stdtypes.html#typecontextmanager"><span>Context Manager Types</span></a>. </span><span class="yiyi-st" id="yiyi-597"><a class="reference internal" href="stdtypes.html#contextmanager.__enter__" title="contextmanager.__enter__"><code class="xref py py-meth docutils literal"><span class="pre">__enter__()</span></code></a> starts the server process (if it has not already started) and then returns the manager object. </span><span class="yiyi-st" id="yiyi-598"><a class="reference internal" href="stdtypes.html#contextmanager.__exit__" title="contextmanager.__exit__"><code class="xref py py-meth docutils literal"><span class="pre">__exit__()</span></code></a> calls <a class="reference internal" href="#multiprocessing.managers.BaseManager.shutdown" title="multiprocessing.managers.BaseManager.shutdown"><code class="xref py py-meth docutils literal"><span class="pre">shutdown()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-599">In previous versions <a class="reference internal" href="stdtypes.html#contextmanager.__enter__" title="contextmanager.__enter__"><code class="xref py py-meth docutils literal"><span class="pre">__enter__()</span></code></a> did not start the manager’s server process if it was not already started.</span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.managers.SyncManager"><span class="yiyi-st" id="yiyi-600"> <em class="property">class </em><code class="descclassname">multiprocessing.managers.</code><code class="descname">SyncManager</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-601">A subclass of <a class="reference internal" href="#multiprocessing.managers.BaseManager" title="multiprocessing.managers.BaseManager"><code class="xref py py-class docutils literal"><span class="pre">BaseManager</span></code></a> which can be used for the synchronization of processes. </span><span class="yiyi-st" id="yiyi-602">Objects of this type are returned by <code class="xref py py-func docutils literal"><span class="pre">multiprocessing.Manager()</span></code>.</span></p><p><span class="yiyi-st" id="yiyi-603">It also supports creation of shared lists and dictionaries.</span></p><dl class="method"><dt id="multiprocessing.managers.SyncManager.Barrier"><span class="yiyi-st" id="yiyi-604"> <code class="descname">Barrier</code><span class="sig-paren">(</span><em>parties</em><span class="optional">[</span>, <em>action</em><span class="optional">[</span>, <em>timeout</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-605">Create a shared <a class="reference internal" href="threading.html#threading.Barrier" title="threading.Barrier"><code class="xref py py-class docutils literal"><span class="pre">threading.Barrier</span></code></a> object and return a proxy for it.</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-606"><span class="versionmodified">New in version 3.3.</span></span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.BoundedSemaphore"><span class="yiyi-st" id="yiyi-607"> <code class="descname">BoundedSemaphore</code><span class="sig-paren">(</span><span class="optional">[</span><em>value</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-608">Create a shared <a class="reference internal" href="threading.html#threading.BoundedSemaphore" title="threading.BoundedSemaphore"><code class="xref py py-class docutils literal"><span class="pre">threading.BoundedSemaphore</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Condition"><span class="yiyi-st" id="yiyi-609"> <code class="descname">Condition</code><span class="sig-paren">(</span><span class="optional">[</span><em>lock</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-610">Create a shared <a class="reference internal" href="threading.html#threading.Condition" title="threading.Condition"><code class="xref py py-class docutils literal"><span class="pre">threading.Condition</span></code></a> object and return a proxy for it.</span></p><p><span class="yiyi-st" id="yiyi-611">If <em>lock</em> is supplied then it should be a proxy for a <a class="reference internal" href="threading.html#threading.Lock" title="threading.Lock"><code class="xref py py-class docutils literal"><span class="pre">threading.Lock</span></code></a> or <a class="reference internal" href="threading.html#threading.RLock" title="threading.RLock"><code class="xref py py-class docutils literal"><span class="pre">threading.RLock</span></code></a> object.</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-612"><span class="versionmodified">Changed in version 3.3: </span>The <a class="reference internal" href="threading.html#threading.Condition.wait_for" title="threading.Condition.wait_for"><code class="xref py py-meth docutils literal"><span class="pre">wait_for()</span></code></a> method was added.</span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Event"><span class="yiyi-st" id="yiyi-613"> <code class="descname">Event</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-614">Create a shared <a class="reference internal" href="threading.html#threading.Event" title="threading.Event"><code class="xref py py-class docutils literal"><span class="pre">threading.Event</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Lock"><span class="yiyi-st" id="yiyi-615"> <code class="descname">Lock</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-616">Create a shared <a class="reference internal" href="threading.html#threading.Lock" title="threading.Lock"><code class="xref py py-class docutils literal"><span class="pre">threading.Lock</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Namespace"><span class="yiyi-st" id="yiyi-617"> <code class="descname">Namespace</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-618">Create a shared <a class="reference internal" href="#multiprocessing.managers.Namespace" title="multiprocessing.managers.Namespace"><code class="xref py py-class docutils literal"><span class="pre">Namespace</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Queue"><span class="yiyi-st" id="yiyi-619"> <code class="descname">Queue</code><span class="sig-paren">(</span><span class="optional">[</span><em>maxsize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-620">Create a shared <a class="reference internal" href="queue.html#queue.Queue" title="queue.Queue"><code class="xref py py-class docutils literal"><span class="pre">queue.Queue</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.RLock"><span class="yiyi-st" id="yiyi-621"> <code class="descname">RLock</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-622">Create a shared <a class="reference internal" href="threading.html#threading.RLock" title="threading.RLock"><code class="xref py py-class docutils literal"><span class="pre">threading.RLock</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Semaphore"><span class="yiyi-st" id="yiyi-623"> <code class="descname">Semaphore</code><span class="sig-paren">(</span><span class="optional">[</span><em>value</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-624">Create a shared <a class="reference internal" href="threading.html#threading.Semaphore" title="threading.Semaphore"><code class="xref py py-class docutils literal"><span class="pre">threading.Semaphore</span></code></a> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Array"><span class="yiyi-st" id="yiyi-625"> <code class="descname">Array</code><span class="sig-paren">(</span><em>typecode</em>, <em>sequence</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-626">Create an array and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.Value"><span class="yiyi-st" id="yiyi-627"> <code class="descname">Value</code><span class="sig-paren">(</span><em>typecode</em>, <em>value</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-628">Create an object with a writable <code class="docutils literal"><span class="pre">value</span></code> attribute and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.dict"><span class="yiyi-st" id="yiyi-629"> <code class="descname">dict</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dt><span class="yiyi-st" id="yiyi-630"> <code class="descname">dict</code><span class="sig-paren">(</span><em>mapping</em><span class="sig-paren">)</span></span></dt><dt><span class="yiyi-st" id="yiyi-631"> <code class="descname">dict</code><span class="sig-paren">(</span><em>sequence</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-632">Create a shared <code class="docutils literal"><span class="pre">dict</span></code> object and return a proxy for it.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.SyncManager.list"><span class="yiyi-st" id="yiyi-633"> <code class="descname">list</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dt><span class="yiyi-st" id="yiyi-634"> <code class="descname">list</code><span class="sig-paren">(</span><em>sequence</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-635">Create a shared <code class="docutils literal"><span class="pre">list</span></code> object and return a proxy for it.</span></p></dd></dl><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-636">Note</span></p><p><span class="yiyi-st" id="yiyi-637">Modifications to mutable values or items in dict and list proxies will not be propagated through the manager, because the proxy has no way of knowing when its values or items are modified. </span><span class="yiyi-st" id="yiyi-638">To modify such an item, you can re-assign the modified object to the container proxy:</span></p><div class="last highlight-python3"><div class="highlight"><pre><span></span><span class="c1"># create a list proxy and append a mutable object (a dictionary)</span>
<span class="n">lproxy</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">()</span>
<span class="n">lproxy</span><span class="o">.</span><span class="n">append</span><span class="p">({})</span>
<span class="c1"># now mutate the dictionary</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">lproxy</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">d</span><span class="p">[</span><span class="s1">'a'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">d</span><span class="p">[</span><span class="s1">'b'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">2</span>
<span class="c1"># at this point, the changes to d are not yet synced, but by</span>
<span class="c1"># reassigning the dictionary, the proxy is notified of the change</span>
<span class="n">lproxy</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">d</span>
</pre></div></div></div></dd></dl><dl class="class"><dt id="multiprocessing.managers.Namespace"><span class="yiyi-st" id="yiyi-639"> <em class="property">class </em><code class="descclassname">multiprocessing.managers.</code><code class="descname">Namespace</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-640">A type that can register with <a class="reference internal" href="#multiprocessing.managers.SyncManager" title="multiprocessing.managers.SyncManager"><code class="xref py py-class docutils literal"><span class="pre">SyncManager</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-641">A namespace object has no public methods, but does have writable attributes. </span><span class="yiyi-st" id="yiyi-642">Its representation shows the values of its attributes.</span></p><p><span class="yiyi-st" id="yiyi-643">However, when using a proxy for a namespace object, an attribute beginning with <code class="docutils literal"><span class="pre">'_'</span></code> will be an attribute of the proxy and not an attribute of the referent:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">manager</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Manager</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">Global</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">Namespace</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">Global</span><span class="o">.</span><span class="n">x</span> <span class="o">=</span> <span class="mi">10</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">Global</span><span class="o">.</span><span class="n">y</span> <span class="o">=</span> <span class="s1">'hello'</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">Global</span><span class="o">.</span><span class="n">_z</span> <span class="o">=</span> <span class="mf">12.3</span> <span class="c1"># this is an attribute of the proxy</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">Global</span><span class="p">)</span>
<span class="go">Namespace(x=10, y='hello')</span>
</code></pre></dd></dl><div class="section" id="customized-managers"><h4><span class="yiyi-st" id="yiyi-644">17.2.2.7.1. </span><span class="yiyi-st" id="yiyi-645">Customized managers</span></h4><p><span class="yiyi-st" id="yiyi-646">To create one’s own manager, one creates a subclass of <a class="reference internal" href="#multiprocessing.managers.BaseManager" title="multiprocessing.managers.BaseManager"><code class="xref py py-class docutils literal"><span class="pre">BaseManager</span></code></a> and uses the <a class="reference internal" href="#multiprocessing.managers.BaseManager.register" title="multiprocessing.managers.BaseManager.register"><code class="xref py py-meth docutils literal"><span class="pre">register()</span></code></a> classmethod to register new types or callables with the manager class. </span><span class="yiyi-st" id="yiyi-647">For example:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="k">class</span> <span class="nc">MathsClass</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="n">y</span>
<span class="k">def</span> <span class="nf">mul</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">*</span> <span class="n">y</span>
<span class="k">class</span> <span class="nc">MyManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span>
<span class="k">pass</span>
<span class="n">MyManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'Maths'</span><span class="p">,</span> <span class="n">MathsClass</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="k">with</span> <span class="n">MyManager</span><span class="p">()</span> <span class="k">as</span> <span class="n">manager</span><span class="p">:</span>
<span class="n">maths</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">Maths</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="n">maths</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">))</span> <span class="c1"># prints 7</span>
<span class="nb">print</span><span class="p">(</span><span class="n">maths</span><span class="o">.</span><span class="n">mul</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">8</span><span class="p">))</span> <span class="c1"># prints 56</span>
</code></pre></div><div class="section" id="using-a-remote-manager"><h4><span class="yiyi-st" id="yiyi-648">17.2.2.7.2. </span><span class="yiyi-st" id="yiyi-649">Using a remote manager</span></h4><p><span class="yiyi-st" id="yiyi-650">It is possible to run a manager server on one machine and have clients use it from other machines (assuming that the firewalls involved allow it).</span></p><p><span class="yiyi-st" id="yiyi-651">Running the following commands creates a server for a single shared queue which remote clients can access:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">queue</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span> <span class="o">=</span> <span class="n">queue</span><span class="o">.</span><span class="n">Queue</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">QueueManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span> <span class="k">pass</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">QueueManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'get_queue'</span><span class="p">,</span> <span class="n">callable</span><span class="o">=</span><span class="k">lambda</span><span class="p">:</span><span class="n">queue</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">QueueManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">''</span><span class="p">,</span> <span class="mi">50000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abracadabra'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">s</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">get_server</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">s</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-652">One client can access the server as follows:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">QueueManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span> <span class="k">pass</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">QueueManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'get_queue'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">QueueManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">'foo.bar.org'</span><span class="p">,</span> <span class="mi">50000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abracadabra'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">get_queue</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span>
</code></pre><p><span class="yiyi-st" id="yiyi-653">Another client can also use it:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">QueueManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span> <span class="k">pass</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">QueueManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'get_queue'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">QueueManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">'foo.bar.org'</span><span class="p">,</span> <span class="mi">50000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abracadabra'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">get_queue</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span><span class="o">.</span><span class="n">get</span><span class="p">()</span>
<span class="go">'hello'</span>
</code></pre><p><span class="yiyi-st" id="yiyi-654">Local processes can also access that queue, using the code from above on the client to access it remotely:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span>
<span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">Worker</span><span class="p">(</span><span class="n">Process</span><span class="p">):</span>
<span class="gp">... </span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">q</span><span class="p">):</span>
<span class="gp">... </span> <span class="bp">self</span><span class="o">.</span><span class="n">q</span> <span class="o">=</span> <span class="n">q</span>
<span class="gp">... </span> <span class="nb">super</span><span class="p">(</span><span class="n">Worker</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">__init__</span><span class="p">()</span>
<span class="gp">... </span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="gp">... </span> <span class="bp">self</span><span class="o">.</span><span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'local hello'</span><span class="p">)</span>
<span class="gp">...</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">queue</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">w</span> <span class="o">=</span> <span class="n">Worker</span><span class="p">(</span><span class="n">queue</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">w</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">QueueManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span> <span class="k">pass</span>
<span class="gp">...</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">QueueManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'get_queue'</span><span class="p">,</span> <span class="n">callable</span><span class="o">=</span><span class="k">lambda</span><span class="p">:</span> <span class="n">queue</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">QueueManager</span><span class="p">(</span><span class="n">address</span><span class="o">=</span><span class="p">(</span><span class="s1">''</span><span class="p">,</span> <span class="mi">50000</span><span class="p">),</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'abracadabra'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">s</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">get_server</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">s</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
</code></pre></div></div><div class="section" id="proxy-objects"><h3><span class="yiyi-st" id="yiyi-655">17.2.2.8. </span><span class="yiyi-st" id="yiyi-656">Proxy Objects</span></h3><p><span class="yiyi-st" id="yiyi-657">A proxy is an object which <em>refers</em> to a shared object which lives (presumably) in a different process. </span><span class="yiyi-st" id="yiyi-658">The shared object is said to be the <em>referent</em> of the proxy. </span><span class="yiyi-st" id="yiyi-659">Multiple proxy objects may have the same referent.</span></p><p><span class="yiyi-st" id="yiyi-660">A proxy object has methods which invoke corresponding methods of its referent (although not every method of the referent will necessarily be available through the proxy). </span><span class="yiyi-st" id="yiyi-661">A proxy can usually be used in most of the same ways that its referent can:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Manager</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">manager</span> <span class="o">=</span> <span class="n">Manager</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">([</span><span class="n">i</span><span class="o">*</span><span class="n">i</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)])</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">l</span><span class="p">)</span>
<span class="go">[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="nb">repr</span><span class="p">(</span><span class="n">l</span><span class="p">))</span>
<span class="go">&lt;ListProxy object, typeid 'list' at 0x...&gt;</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span><span class="p">[</span><span class="mi">4</span><span class="p">]</span>
<span class="go">16</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="mi">5</span><span class="p">]</span>
<span class="go">[4, 9, 16]</span>
</code></pre><p><span class="yiyi-st" id="yiyi-662">Notice that applying <a class="reference internal" href="stdtypes.html#str" title="str"><code class="xref py py-func docutils literal"><span class="pre">str()</span></code></a> to a proxy will return the representation of the referent, whereas applying <a class="reference internal" href="functions.html#repr" title="repr"><code class="xref py py-func docutils literal"><span class="pre">repr()</span></code></a> will return the representation of the proxy.</span></p><p><span class="yiyi-st" id="yiyi-663">An important feature of proxy objects is that they are picklable so they can be passed between processes. </span><span class="yiyi-st" id="yiyi-664">Note, however, that if a proxy is sent to the corresponding manager’s process then unpickling it will produce the referent itself. </span><span class="yiyi-st" id="yiyi-665">This means, for example, that one shared object can contain a second:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">a</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">a</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">b</span><span class="p">)</span> <span class="c1"># referent of a now contains referent of b</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="go">[[]] []</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">b</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="go">[['hello']] ['hello']</span>
</code></pre><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-666">Note</span></p><p><span class="yiyi-st" id="yiyi-667">The proxy types in <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> do nothing to support comparisons by value. </span><span class="yiyi-st" id="yiyi-668">So, for instance, we have:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">])</span> <span class="o">==</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">]</span>
<span class="go">False</span>
</code></pre><p class="last"><span class="yiyi-st" id="yiyi-669">One should just use a copy of the referent instead when making comparisons.</span></p></div><dl class="class"><dt id="multiprocessing.managers.BaseProxy"><span class="yiyi-st" id="yiyi-670"> <em class="property">class </em><code class="descclassname">multiprocessing.managers.</code><code class="descname">BaseProxy</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-671">Proxy objects are instances of subclasses of <a class="reference internal" href="#multiprocessing.managers.BaseProxy" title="multiprocessing.managers.BaseProxy"><code class="xref py py-class docutils literal"><span class="pre">BaseProxy</span></code></a>.</span></p><dl class="method"><dt id="multiprocessing.managers.BaseProxy._callmethod"><span class="yiyi-st" id="yiyi-672"> <code class="descname">_callmethod</code><span class="sig-paren">(</span><em>methodname</em><span class="optional">[</span>, <em>args</em><span class="optional">[</span>, <em>kwds</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-673">Call and return the result of a method of the proxy’s referent.</span></p><p><span class="yiyi-st" id="yiyi-674">If <code class="docutils literal"><span class="pre">proxy</span></code> is a proxy whose referent is <code class="docutils literal"><span class="pre">obj</span></code> then the expression</span></p><pre><code class="language-python"><span></span><span class="n">proxy</span><span class="o">.</span><span class="n">_callmethod</span><span class="p">(</span><span class="n">methodname</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">kwds</span><span class="p">)</span>
</code></pre><p><span class="yiyi-st" id="yiyi-675">will evaluate the expression</span></p><pre><code class="language-python"><span></span><span class="nb">getattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">methodname</span><span class="p">)(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwds</span><span class="p">)</span>
</code></pre><p><span class="yiyi-st" id="yiyi-676">in the manager’s process.</span></p><p><span class="yiyi-st" id="yiyi-677">The returned value will be a copy of the result of the call or a proxy to a new shared object – see documentation for the <em>method_to_typeid</em> argument of <a class="reference internal" href="#multiprocessing.managers.BaseManager.register" title="multiprocessing.managers.BaseManager.register"><code class="xref py py-meth docutils literal"><span class="pre">BaseManager.register()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-678">If an exception is raised by the call, then is re-raised by <a class="reference internal" href="#multiprocessing.managers.BaseProxy._callmethod" title="multiprocessing.managers.BaseProxy._callmethod"><code class="xref py py-meth docutils literal"><span class="pre">_callmethod()</span></code></a>. </span><span class="yiyi-st" id="yiyi-679">If some other exception is raised in the manager’s process then this is converted into a <code class="xref py py-exc docutils literal"><span class="pre">RemoteError</span></code> exception and is raised by <a class="reference internal" href="#multiprocessing.managers.BaseProxy._callmethod" title="multiprocessing.managers.BaseProxy._callmethod"><code class="xref py py-meth docutils literal"><span class="pre">_callmethod()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-680">Note in particular that an exception will be raised if <em>methodname</em> has not been <em>exposed</em>.</span></p><p><span class="yiyi-st" id="yiyi-681">An example of the usage of <a class="reference internal" href="#multiprocessing.managers.BaseProxy._callmethod" title="multiprocessing.managers.BaseProxy._callmethod"><code class="xref py py-meth docutils literal"><span class="pre">_callmethod()</span></code></a>:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">l</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span><span class="o">.</span><span class="n">_callmethod</span><span class="p">(</span><span class="s1">'__len__'</span><span class="p">)</span>
<span class="go">10</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span><span class="o">.</span><span class="n">_callmethod</span><span class="p">(</span><span class="s1">'__getitem__'</span><span class="p">,</span> <span class="p">(</span><span class="nb">slice</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">7</span><span class="p">),))</span> <span class="c1"># equivalent to l[2:7]</span>
<span class="go">[2, 3, 4, 5, 6]</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">l</span><span class="o">.</span><span class="n">_callmethod</span><span class="p">(</span><span class="s1">'__getitem__'</span><span class="p">,</span> <span class="p">(</span><span class="mi">20</span><span class="p">,))</span> <span class="c1"># equivalent to l[20]</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="c">...</span>
<span class="gr">IndexError</span>: <span class="n">list index out of range</span>
</code></pre></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseProxy._getvalue"><span class="yiyi-st" id="yiyi-682"> <code class="descname">_getvalue</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-683">Return a copy of the referent.</span></p><p><span class="yiyi-st" id="yiyi-684">If the referent is unpicklable then this will raise an exception.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseProxy.__repr__"><span class="yiyi-st" id="yiyi-685"> <code class="descname">__repr__</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-686">Return a representation of the proxy object.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.managers.BaseProxy.__str__"><span class="yiyi-st" id="yiyi-687"> <code class="descname">__str__</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-688">Return the representation of the referent.</span></p></dd></dl></dd></dl><div class="section" id="cleanup"><h4><span class="yiyi-st" id="yiyi-689">17.2.2.8.1. </span><span class="yiyi-st" id="yiyi-690">Cleanup</span></h4><p><span class="yiyi-st" id="yiyi-691">A proxy object uses a weakref callback so that when it gets garbage collected it deregisters itself from the manager which owns its referent.</span></p><p><span class="yiyi-st" id="yiyi-692">A shared object gets deleted from the manager process when there are no longer any proxies referring to it.</span></p></div></div><div class="section" id="module-multiprocessing.pool"><h3><span class="yiyi-st" id="yiyi-693">17.2.2.9. </span><span class="yiyi-st" id="yiyi-694">进程池</span></h3><p><span class="yiyi-st" id="yiyi-695">可以用<a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>类创建一个进程池来执行提交给它的任务。</span></p><dl class="class"><dt id="multiprocessing.pool.Pool"><span class="yiyi-st" id="yiyi-696"> <em class="property">class </em><code class="descclassname">multiprocessing.pool.</code><code class="descname">Pool</code><span class="sig-paren">(</span><span class="optional">[</span><em>processes</em><span class="optional">[</span>, <em>initializer</em><span class="optional">[</span>, <em>initargs</em><span class="optional">[</span>, <em>maxtasksperchild</em><span class="optional">[</span>, <em>context</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-697">一个进程池对象，控制一个包含工作进程的进程池，作业可以提交给它。</span><span class="yiyi-st" id="yiyi-698">它支持具有超时和回调的异步结果，并具有并行映射实现。</span></p><p><span class="yiyi-st" id="yiyi-699"><em>processes</em>是要使用的工作进程数。</span><span class="yiyi-st" id="yiyi-700">如果<em>processes</em>为<code class="docutils literal"><span class="pre">None</span></code>，则使用<a class="reference internal" href="os.html#os.cpu_count" title="os.cpu_count"><code class="xref py py-func docutils literal"><span class="pre">os.cpu_count()</span></code></a>返回的数字。</span></p><p><span class="yiyi-st" id="yiyi-701">如果<em>initializer</em>不是<code class="docutils literal"><span class="pre">None</span></code>，则每个工作进程在启动时将调用<code class="docutils literal"><span class="pre">initializer(*initargs)</span></code>。</span></p><p><span class="yiyi-st" id="yiyi-702"><em>maxtasksperchild</em>是工作进程在退出并由新工作进程替换之前可以完成的任务数，以使未使用的资源可以释放。</span><span class="yiyi-st" id="yiyi-703">默认的<em>maxtasksperchild</em>是None，这意味着工作进程的生存时间将与进程池一样长。</span></p><p><span class="yiyi-st" id="yiyi-704"><em>context</em>可用于指定用于启动工作进程的上下文。</span><span class="yiyi-st" id="yiyi-705">通常使用<code class="xref py py-func docutils literal"><span class="pre">multiprocessing.Pool()</span></code>或上下文对象的<a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-meth docutils literal"><span class="pre">Pool()</span></code></a>方法创建进程池。</span><span class="yiyi-st" id="yiyi-706">在这两种情况下，都会正确地设置<em>context</em>。</span></p><p><span class="yiyi-st" id="yiyi-707">注意，进程池对象的方法只能由创建进程池的进程调用。</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-708"><span class="versionmodified">版本3.2中的新功能：</span> <em>maxtasksperchild</em></span></p></div><div class="versionadded"><p><span class="yiyi-st" id="yiyi-709"><span class="versionmodified">版本3.4中的新功能：</span> <em>context</em></span></p></div><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-710">注</span></p><p class="last"><span class="yiyi-st" id="yiyi-711"><a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>内的工作进程通常在进程池的工作队列的整个持续时间内生效。</span><span class="yiyi-st" id="yiyi-712">在其他系统（例如Apache、mod_wsgi等）中，释放工作进程持有的资源的常见模式是允许进程池内的工作进程只完成一定数量的作业，然后再退出、清理并生成一个新的进程来替换旧的进程。</span><span class="yiyi-st" id="yiyi-713"><a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>的<em>maxtasksperchild</em>参数就是向用户提供这个功能。</span></p></div><dl class="method"><dt id="multiprocessing.pool.Pool.apply"><span class="yiyi-st" id="yiyi-714"> <code class="descname">apply</code><span class="sig-paren">(</span><em>func</em><span class="optional">[</span>, <em>args</em><span class="optional">[</span>, <em>kwds</em><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-715">使用参数<em>args</em>和关键字参数<em>kwds</em>调用<em>func</em>。</span><span class="yiyi-st" id="yiyi-716">它阻塞直到结果完成。</span><span class="yiyi-st" id="yiyi-717">相比这个代码块，<a class="reference internal" href="#multiprocessing.pool.Pool.apply_async" title="multiprocessing.pool.Pool.apply_async"><code class="xref py py-meth docutils literal"><span class="pre">apply_async()</span></code></a>更适合并行执行工作。</span><span class="yiyi-st" id="yiyi-718">此外，<em>func</em>仅在进程池的其中一个工作进程中执行。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.apply_async"><span class="yiyi-st" id="yiyi-719"> <code class="descname">apply_async</code><span class="sig-paren">(</span><em>func</em><span class="optional">[</span>, <em>args</em><span class="optional">[</span>, <em>kwds</em><span class="optional">[</span>, <em>callback</em><span class="optional">[</span>, <em>error_callback</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-720"><a class="reference internal" href="#multiprocessing.pool.Pool.apply" title="multiprocessing.pool.Pool.apply"><code class="xref py py-meth docutils literal"><span class="pre">apply()</span></code></a>方法的一个变体，它返回一个结果对象。</span></p><p><span class="yiyi-st" id="yiyi-721">如果指定<em>callback</em>，那么它应该是一个可接受单个参数的可调用对象。</span><span class="yiyi-st" id="yiyi-722">当结果完成时就对它应用<em>callback</em>，在调用失败的情况下则应用<em>error_callback</em>。</span></p><p><span class="yiyi-st" id="yiyi-723">如果指定<em>error_callback</em>，那么它应该是一个接受单个参数的可调用对象。</span><span class="yiyi-st" id="yiyi-724">如果目标函数失败，则以该异常实例调用<em>error_callback</em>。</span></p><p><span class="yiyi-st" id="yiyi-725">回调应该立即完成，否则处理结果的线程将被阻塞。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.map"><span class="yiyi-st" id="yiyi-726"> <code class="descname">map</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-727">与<a class="reference internal" href="functions.html#map" title="map"><code class="xref py py-func docutils literal"><span class="pre">map()</span></code></a>内置函数（虽然它只支持一个<em>iterable</em>参数）等价的并行方式。</span><span class="yiyi-st" id="yiyi-728">它阻塞直到结果准备好。</span></p><p><span class="yiyi-st" id="yiyi-729">该方法将iterable分成多个块，然后将这些块作为单独的任务提交到进程池。</span><span class="yiyi-st" id="yiyi-730">这些块的（近似）大小可以通过将<em>chunksize</em>设置为正整数来指定。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.map_async"><span class="yiyi-st" id="yiyi-731"> <code class="descname">map_async</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">[</span>, <em>callback</em><span class="optional">[</span>, <em>error_callback</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-732">返回结果对象的<a class="reference internal" href="#multiprocessing.pool.Pool.map" title="multiprocessing.pool.Pool.map"><code class="xref py py-meth docutils literal"><span class="pre">map()</span></code></a>方法的变体。</span></p><p><span class="yiyi-st" id="yiyi-733">如果指定<em>callback</em>，那么它应该是一个可接受单个参数的可调用对象。</span><span class="yiyi-st" id="yiyi-734">当结果完成时就对它应用<em>callback</em>，在调用失败的情况下则应用<em>error_callback</em>。</span></p><p><span class="yiyi-st" id="yiyi-735">如果指定<em>error_callback</em>，那么它应该是一个接受单个参数的可调用对象。</span><span class="yiyi-st" id="yiyi-736">如果目标函数失败，则以该异常实例调用<em>error_callback</em>。</span></p><p><span class="yiyi-st" id="yiyi-737">回调应该立即完成，否则处理结果的线程将被阻塞。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.imap"><span class="yiyi-st" id="yiyi-738"> <code class="descname">imap</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-739"><a class="reference internal" href="functions.html#map" title="map"><code class="xref py py-meth docutils literal"><span class="pre">map()</span></code></a>的惰性版本。</span></p><p><span class="yiyi-st" id="yiyi-740"><em>chunksize</em>参数与<a class="reference internal" href="#multiprocessing.pool.Pool.map" title="multiprocessing.pool.Pool.map"><code class="xref py py-meth docutils literal"><span class="pre">map()</span></code></a>方法使用的参数相同。</span><span class="yiyi-st" id="yiyi-741">对于非常长的iterable，<em>chunksize</em>使用一个较大值的可以使得作业比使用默认值<code class="docutils literal"><span class="pre">1</span></code><strong>更快</strong>地完成。</span></p><p><span class="yiyi-st" id="yiyi-742">此外，如果<em>chunksize</em>是<code class="docutils literal"><span class="pre">1</span></code>，那么<a class="reference internal" href="#multiprocessing.pool.Pool.imap" title="multiprocessing.pool.Pool.imap"><code class="xref py py-meth docutils literal"><span class="pre">imap()</span></code></a>方法返回的迭代器的<code class="xref py py-meth docutils literal"><span class="pre">next()</span></code>方法有一个可选的参数<em>timeout</em>：如果在<em>timeout</em>秒内无法返回结果，那么<code class="docutils literal"><span class="pre">next(timeout)</span></code>将引发<a class="reference internal" href="#multiprocessing.TimeoutError" title="multiprocessing.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">multiprocessing.TimeoutError</span></code></a>。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.imap_unordered"><span class="yiyi-st" id="yiyi-743"> <code class="descname">imap_unordered</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-744">与<a class="reference internal" href="#multiprocessing.pool.Pool.imap" title="multiprocessing.pool.Pool.imap"><code class="xref py py-meth docutils literal"><span class="pre">imap()</span></code></a>相同，除了返回的迭代器的结果的顺序应该被认为是任意的。</span><span class="yiyi-st" id="yiyi-745">（只有当只有一个工作进程时，其顺序才保证是“正确的”。）</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.starmap"><span class="yiyi-st" id="yiyi-746"> <code class="descname">starmap</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-747">类似<a class="reference internal" href="functions.html#map" title="map"><code class="xref py py-meth docutils literal"><span class="pre">map()</span></code></a>，除了<em>iterable</em>的元素应该是可迭代对象，它们将分拆为参数。</span></p><p><span class="yiyi-st" id="yiyi-748">因此，<code class="docutils literal"><span class="pre">[(1,2),</span> <span class="pre">(3,</span> <span class="pre">4)]</span></code>这个<em>iterable</em>将导致<code class="docutils literal"><span class="pre">[func(1,2),</span> <span class="pre">func(3,4)]</span></code>。</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-749"><span class="versionmodified">版本3.3中的新功能。</span></span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.starmap_async"><span class="yiyi-st" id="yiyi-750"> <code class="descname">starmap_async</code><span class="sig-paren">(</span><em>func</em>, <em>iterable</em><span class="optional">[</span>, <em>chunksize</em><span class="optional">[</span>, <em>callback</em><span class="optional">[</span>, <em>error_back</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-751"><a class="reference internal" href="#multiprocessing.pool.Pool.starmap" title="multiprocessing.pool.Pool.starmap"><code class="xref py py-meth docutils literal"><span class="pre">starmap()</span></code></a>和<a class="reference internal" href="#multiprocessing.pool.Pool.map_async" title="multiprocessing.pool.Pool.map_async"><code class="xref py py-meth docutils literal"><span class="pre">map_async()</span></code></a>的组合，在可迭代对象组成的<em>iterable</em>上迭代并以分拆后的可迭代对象调用<em>func</em>。</span><span class="yiyi-st" id="yiyi-752">返回一个结果对象。</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-753"><span class="versionmodified">版本3.3中的新功能。</span></span></p></div></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.close"><span class="yiyi-st" id="yiyi-754"> <code class="descname">close</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-755">阻止任何更多的任务提交到进程池。</span><span class="yiyi-st" id="yiyi-756">一旦所有任务完成，工作进程将退出。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.terminate"><span class="yiyi-st" id="yiyi-757"> <code class="descname">terminate</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-758">立即停止工作进程，而不需要完成未完成的工作。</span><span class="yiyi-st" id="yiyi-759">当进程池对象被垃圾收集，<a class="reference internal" href="#multiprocessing.pool.Pool.terminate" title="multiprocessing.pool.Pool.terminate"><code class="xref py py-meth docutils literal"><span class="pre">terminate()</span></code></a>将被立即调用。</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.Pool.join"><span class="yiyi-st" id="yiyi-760"> <code class="descname">join</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-761">等待工作进程退出。</span><span class="yiyi-st" id="yiyi-762">在使用<a class="reference internal" href="#multiprocessing.pool.Pool.join" title="multiprocessing.pool.Pool.join"><code class="xref py py-meth docutils literal"><span class="pre">join()</span></code></a>之前，必须调用<a class="reference internal" href="#multiprocessing.pool.Pool.close" title="multiprocessing.pool.Pool.close"><code class="xref py py-meth docutils literal"><span class="pre">close()</span></code></a>或<a class="reference internal" href="#multiprocessing.pool.Pool.terminate" title="multiprocessing.pool.Pool.terminate"><code class="xref py py-meth docutils literal"><span class="pre">terminate()</span></code></a>。</span></p></dd></dl><div class="versionadded"><p><span class="yiyi-st" id="yiyi-763"><span class="versionmodified">版本3.3中的新功能：</span>进程池对象现在支持上下文管理协议 —— 请参阅<a class="reference internal" href="stdtypes.html#typecontextmanager"><span>上下文管理器类型</span></a>。</span><span class="yiyi-st" id="yiyi-764"><a class="reference internal" href="stdtypes.html#contextmanager.__enter__" title="contextmanager.__enter__"><code class="xref py py-meth docutils literal"><span class="pre">__enter__()</span></code></a>返回进程池对象，<a class="reference internal" href="stdtypes.html#contextmanager.__exit__" title="contextmanager.__exit__"><code class="xref py py-meth docutils literal"><span class="pre">__exit__()</span></code></a>调用<a class="reference internal" href="#multiprocessing.pool.Pool.terminate" title="multiprocessing.pool.Pool.terminate"><code class="xref py py-meth docutils literal"><span class="pre">terminate()</span></code></a>。</span></p></div></dd></dl><dl class="class"><dt id="multiprocessing.pool.AsyncResult"><span class="yiyi-st" id="yiyi-765"> <em class="property">class </em><code class="descclassname">multiprocessing.pool.</code><code class="descname">AsyncResult</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-766">The class of the result returned by <a class="reference internal" href="#multiprocessing.pool.Pool.apply_async" title="multiprocessing.pool.Pool.apply_async"><code class="xref py py-meth docutils literal"><span class="pre">Pool.apply_async()</span></code></a> and <a class="reference internal" href="#multiprocessing.pool.Pool.map_async" title="multiprocessing.pool.Pool.map_async"><code class="xref py py-meth docutils literal"><span class="pre">Pool.map_async()</span></code></a>.</span></p><dl class="method"><dt id="multiprocessing.pool.AsyncResult.get"><span class="yiyi-st" id="yiyi-767"> <code class="descname">get</code><span class="sig-paren">(</span><span class="optional">[</span><em>timeout</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-768">Return the result when it arrives. </span><span class="yiyi-st" id="yiyi-769">If <em>timeout</em> is not <code class="docutils literal"><span class="pre">None</span></code> and the result does not arrive within <em>timeout</em> seconds then <a class="reference internal" href="#multiprocessing.TimeoutError" title="multiprocessing.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">multiprocessing.TimeoutError</span></code></a> is raised. </span><span class="yiyi-st" id="yiyi-770">If the remote call raised an exception then that exception will be reraised by <a class="reference internal" href="#multiprocessing.pool.AsyncResult.get" title="multiprocessing.pool.AsyncResult.get"><code class="xref py py-meth docutils literal"><span class="pre">get()</span></code></a>.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.AsyncResult.wait"><span class="yiyi-st" id="yiyi-771"> <code class="descname">wait</code><span class="sig-paren">(</span><span class="optional">[</span><em>timeout</em><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-772">Wait until the result is available or until <em>timeout</em> seconds pass.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.AsyncResult.ready"><span class="yiyi-st" id="yiyi-773"> <code class="descname">ready</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-774">Return whether the call has completed.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.pool.AsyncResult.successful"><span class="yiyi-st" id="yiyi-775"> <code class="descname">successful</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-776">Return whether the call completed without raising an exception. </span><span class="yiyi-st" id="yiyi-777">Will raise <a class="reference internal" href="exceptions.html#AssertionError" title="AssertionError"><code class="xref py py-exc docutils literal"><span class="pre">AssertionError</span></code></a> if the result is not ready.</span></p></dd></dl></dd></dl><p><span class="yiyi-st" id="yiyi-778">The following example demonstrates the use of a pool:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Pool</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="k">with</span> <span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span> <span class="c1"># start 4 worker processes</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="p">(</span><span class="mi">10</span><span class="p">,))</span> <span class="c1"># evaluate "f(10)" asynchronously in a single process</span>
<span class="nb">print</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="c1"># prints "100" unless your computer is *very* slow</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pool</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)))</span> <span class="c1"># prints "[0, 1, 4,..., 81]"</span>
<span class="n">it</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">it</span><span class="p">))</span> <span class="c1"># prints "0"</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">it</span><span class="p">))</span> <span class="c1"># prints "1"</span>
<span class="nb">print</span><span class="p">(</span><span class="n">it</span><span class="o">.</span><span class="n">next</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="c1"># prints "4" unless your computer is *very* slow</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">,</span> <span class="p">(</span><span class="mi">10</span><span class="p">,))</span>
<span class="nb">print</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">timeout</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="c1"># raises multiprocessing.TimeoutError</span>
</code></pre></div><div class="section" id="module-multiprocessing.connection"><h3><span class="yiyi-st" id="yiyi-779">17.2.2.10. </span><span class="yiyi-st" id="yiyi-780">Listeners and Clients</span></h3><p><span class="yiyi-st" id="yiyi-781">Usually message passing between processes is done using queues or by using <a class="reference internal" href="#multiprocessing.Connection" title="multiprocessing.Connection"><code class="xref py py-class docutils literal"><span class="pre">Connection</span></code></a> objects returned by <a class="reference internal" href="#multiprocessing.Pipe" title="multiprocessing.Pipe"><code class="xref py py-func docutils literal"><span class="pre">Pipe()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-782">However, the <a class="reference internal" href="#module-multiprocessing.connection" title="multiprocessing.connection: API for dealing with sockets."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.connection</span></code></a> module allows some extra flexibility. </span><span class="yiyi-st" id="yiyi-783">It basically gives a high level message oriented API for dealing with sockets or Windows named pipes. </span><span class="yiyi-st" id="yiyi-784">It also has support for <em>digest authentication</em> using the <a class="reference internal" href="hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal"><span class="pre">hmac</span></code></a> module, and for polling multiple connections at the same time.</span></p><dl class="function"><dt id="multiprocessing.connection.deliver_challenge"><span class="yiyi-st" id="yiyi-785"> <code class="descclassname">multiprocessing.connection.</code><code class="descname">deliver_challenge</code><span class="sig-paren">(</span><em>connection</em>, <em>authkey</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-786">Send a randomly generated message to the other end of the connection and wait for a reply.</span></p><p><span class="yiyi-st" id="yiyi-787">If the reply matches the digest of the message using <em>authkey</em> as the key then a welcome message is sent to the other end of the connection. </span><span class="yiyi-st" id="yiyi-788">Otherwise <a class="reference internal" href="#multiprocessing.AuthenticationError" title="multiprocessing.AuthenticationError"><code class="xref py py-exc docutils literal"><span class="pre">AuthenticationError</span></code></a> is raised.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.connection.answer_challenge"><span class="yiyi-st" id="yiyi-789"> <code class="descclassname">multiprocessing.connection.</code><code class="descname">answer_challenge</code><span class="sig-paren">(</span><em>connection</em>, <em>authkey</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-790">Receive a message, calculate the digest of the message using <em>authkey</em> as the key, and then send the digest back.</span></p><p><span class="yiyi-st" id="yiyi-791">If a welcome message is not received, then <a class="reference internal" href="#multiprocessing.AuthenticationError" title="multiprocessing.AuthenticationError"><code class="xref py py-exc docutils literal"><span class="pre">AuthenticationError</span></code></a> is raised.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.connection.Client"><span class="yiyi-st" id="yiyi-792"> <code class="descclassname">multiprocessing.connection.</code><code class="descname">Client</code><span class="sig-paren">(</span><em>address</em><span class="optional">[</span>, <em>family</em><span class="optional">[</span>, <em>authenticate</em><span class="optional">[</span>, <em>authkey</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-793">Attempt to set up a connection to the listener which is using address <em>address</em>, returning a <a class="reference internal" href="#multiprocessing.Connection" title="multiprocessing.Connection"><code class="xref py py-class docutils literal"><span class="pre">Connection</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-794">The type of the connection is determined by <em>family</em> argument, but this can generally be omitted since it can usually be inferred from the format of <em>address</em>. </span><span class="yiyi-st" id="yiyi-795">(See <a class="reference internal" href="#multiprocessing-address-formats"><span>Address Formats</span></a>)</span></p><p><span class="yiyi-st" id="yiyi-796">If <em>authenticate</em> is <code class="docutils literal"><span class="pre">True</span></code> or <em>authkey</em> is a byte string then digest authentication is used. </span><span class="yiyi-st" id="yiyi-797">The key used for authentication will be either <em>authkey</em> or <code class="docutils literal"><span class="pre">current_process().authkey</span></code> if <em>authkey</em> is <code class="docutils literal"><span class="pre">None</span></code>. </span><span class="yiyi-st" id="yiyi-798">If authentication fails then <a class="reference internal" href="#multiprocessing.AuthenticationError" title="multiprocessing.AuthenticationError"><code class="xref py py-exc docutils literal"><span class="pre">AuthenticationError</span></code></a> is raised. </span><span class="yiyi-st" id="yiyi-799">See <a class="reference internal" href="#multiprocessing-auth-keys"><span>Authentication keys</span></a>.</span></p></dd></dl><dl class="class"><dt id="multiprocessing.connection.Listener"><span class="yiyi-st" id="yiyi-800"> <em class="property">class </em><code class="descclassname">multiprocessing.connection.</code><code class="descname">Listener</code><span class="sig-paren">(</span><span class="optional">[</span><em>address</em><span class="optional">[</span>, <em>family</em><span class="optional">[</span>, <em>backlog</em><span class="optional">[</span>, <em>authenticate</em><span class="optional">[</span>, <em>authkey</em><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="optional">]</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-801">A wrapper for a bound socket or Windows named pipe which is ‘listening’ for connections.</span></p><p><span class="yiyi-st" id="yiyi-802"><em>address</em> is the address to be used by the bound socket or named pipe of the listener object.</span></p><div class="admonition note"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-803">Note</span></p><p class="last"><span class="yiyi-st" id="yiyi-804">If an address of ‘0.0.0.0’ is used, the address will not be a connectable end point on Windows. </span><span class="yiyi-st" id="yiyi-805">If you require a connectable end-point, you should use ‘127.0.0.1’.</span></p></div><p><span class="yiyi-st" id="yiyi-806"><em>family</em> is the type of socket (or named pipe) to use. </span><span class="yiyi-st" id="yiyi-807">This can be one of the strings <code class="docutils literal"><span class="pre">'AF_INET'</span></code> (for a TCP socket), <code class="docutils literal"><span class="pre">'AF_UNIX'</span></code> (for a Unix domain socket) or <code class="docutils literal"><span class="pre">'AF_PIPE'</span></code> (for a Windows named pipe). </span><span class="yiyi-st" id="yiyi-808">Of these only the first is guaranteed to be available. </span><span class="yiyi-st" id="yiyi-809">If <em>family</em> is <code class="docutils literal"><span class="pre">None</span></code> then the family is inferred from the format of <em>address</em>. </span><span class="yiyi-st" id="yiyi-810">If <em>address</em> is also <code class="docutils literal"><span class="pre">None</span></code> then a default is chosen. </span><span class="yiyi-st" id="yiyi-811">This default is the family which is assumed to be the fastest available. </span><span class="yiyi-st" id="yiyi-812">See <a class="reference internal" href="#multiprocessing-address-formats"><span>Address Formats</span></a>. </span><span class="yiyi-st" id="yiyi-813">Note that if <em>family</em> is <code class="docutils literal"><span class="pre">'AF_UNIX'</span></code> and address is <code class="docutils literal"><span class="pre">None</span></code> then the socket will be created in a private temporary directory created using <a class="reference internal" href="tempfile.html#tempfile.mkstemp" title="tempfile.mkstemp"><code class="xref py py-func docutils literal"><span class="pre">tempfile.mkstemp()</span></code></a>.</span></p><p><span class="yiyi-st" id="yiyi-814">If the listener object uses a socket then <em>backlog</em> (1 by default) is passed to the <a class="reference internal" href="socket.html#socket.socket.listen" title="socket.socket.listen"><code class="xref py py-meth docutils literal"><span class="pre">listen()</span></code></a> method of the socket once it has been bound.</span></p><p><span class="yiyi-st" id="yiyi-815">If <em>authenticate</em> is <code class="docutils literal"><span class="pre">True</span></code> (<code class="docutils literal"><span class="pre">False</span></code> by default) or <em>authkey</em> is not <code class="docutils literal"><span class="pre">None</span></code> then digest authentication is used.</span></p><p><span class="yiyi-st" id="yiyi-816">If <em>authkey</em> is a byte string then it will be used as the authentication key; otherwise it must be <em>None</em>.</span></p><p><span class="yiyi-st" id="yiyi-817">If <em>authkey</em> is <code class="docutils literal"><span class="pre">None</span></code> and <em>authenticate</em> is <code class="docutils literal"><span class="pre">True</span></code> then <code class="docutils literal"><span class="pre">current_process().authkey</span></code> is used as the authentication key. </span><span class="yiyi-st" id="yiyi-818">If <em>authkey</em> is <code class="docutils literal"><span class="pre">None</span></code> and <em>authenticate</em> is <code class="docutils literal"><span class="pre">False</span></code> then no authentication is done. </span><span class="yiyi-st" id="yiyi-819">If authentication fails then <a class="reference internal" href="#multiprocessing.AuthenticationError" title="multiprocessing.AuthenticationError"><code class="xref py py-exc docutils literal"><span class="pre">AuthenticationError</span></code></a> is raised. </span><span class="yiyi-st" id="yiyi-820">See <a class="reference internal" href="#multiprocessing-auth-keys"><span>Authentication keys</span></a>.</span></p><dl class="method"><dt id="multiprocessing.connection.Listener.accept"><span class="yiyi-st" id="yiyi-821"> <code class="descname">accept</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-822">Accept a connection on the bound socket or named pipe of the listener object and return a <a class="reference internal" href="#multiprocessing.Connection" title="multiprocessing.Connection"><code class="xref py py-class docutils literal"><span class="pre">Connection</span></code></a> object. </span><span class="yiyi-st" id="yiyi-823">If authentication is attempted and fails, then <a class="reference internal" href="#multiprocessing.AuthenticationError" title="multiprocessing.AuthenticationError"><code class="xref py py-exc docutils literal"><span class="pre">AuthenticationError</span></code></a> is raised.</span></p></dd></dl><dl class="method"><dt id="multiprocessing.connection.Listener.close"><span class="yiyi-st" id="yiyi-824"> <code class="descname">close</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-825">Close the bound socket or named pipe of the listener object. </span><span class="yiyi-st" id="yiyi-826">This is called automatically when the listener is garbage collected. </span><span class="yiyi-st" id="yiyi-827">However it is advisable to call it explicitly.</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-828">Listener objects have the following read-only properties:</span></p><dl class="attribute"><dt id="multiprocessing.connection.Listener.address"><span class="yiyi-st" id="yiyi-829"> <code class="descname">address</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-830">The address which is being used by the Listener object.</span></p></dd></dl><dl class="attribute"><dt id="multiprocessing.connection.Listener.last_accepted"><span class="yiyi-st" id="yiyi-831"> <code class="descname">last_accepted</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-832">The address from which the last accepted connection came. </span><span class="yiyi-st" id="yiyi-833">If this is unavailable then it is <code class="docutils literal"><span class="pre">None</span></code>.</span></p></dd></dl><div class="versionadded"><p><span class="yiyi-st" id="yiyi-834"><span class="versionmodified">New in version 3.3: </span>Listener objects now support the context management protocol – see <a class="reference internal" href="stdtypes.html#typecontextmanager"><span>Context Manager Types</span></a>. </span><span class="yiyi-st" id="yiyi-835"><a class="reference internal" href="stdtypes.html#contextmanager.__enter__" title="contextmanager.__enter__"><code class="xref py py-meth docutils literal"><span class="pre">__enter__()</span></code></a> returns the listener object, and <a class="reference internal" href="stdtypes.html#contextmanager.__exit__" title="contextmanager.__exit__"><code class="xref py py-meth docutils literal"><span class="pre">__exit__()</span></code></a> calls <a class="reference internal" href="#multiprocessing.connection.Listener.close" title="multiprocessing.connection.Listener.close"><code class="xref py py-meth docutils literal"><span class="pre">close()</span></code></a>.</span></p></div></dd></dl><dl class="function"><dt id="multiprocessing.connection.wait"><span class="yiyi-st" id="yiyi-836"> <code class="descclassname">multiprocessing.connection.</code><code class="descname">wait</code><span class="sig-paren">(</span><em>object_list</em>, <em>timeout=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-837">Wait till an object in <em>object_list</em> is ready. </span><span class="yiyi-st" id="yiyi-838">Returns the list of those objects in <em>object_list</em> which are ready. </span><span class="yiyi-st" id="yiyi-839">If <em>timeout</em> is a float then the call blocks for at most that many seconds. </span><span class="yiyi-st" id="yiyi-840">If <em>timeout</em> is <code class="docutils literal"><span class="pre">None</span></code> then it will block for an unlimited period. </span><span class="yiyi-st" id="yiyi-841">A negative timeout is equivalent to a zero timeout.</span></p><p><span class="yiyi-st" id="yiyi-842">For both Unix and Windows, an object can appear in <em>object_list</em> if it is</span></p><ul class="simple"><li><span class="yiyi-st" id="yiyi-843">a readable <a class="reference internal" href="#multiprocessing.Connection" title="multiprocessing.Connection"><code class="xref py py-class docutils literal"><span class="pre">Connection</span></code></a> object;</span></li><li><span class="yiyi-st" id="yiyi-844">a connected and readable <a class="reference internal" href="socket.html#socket.socket" title="socket.socket"><code class="xref py py-class docutils literal"><span class="pre">socket.socket</span></code></a> object; or</span></li><li><span class="yiyi-st" id="yiyi-845">the <a class="reference internal" href="#multiprocessing.Process.sentinel" title="multiprocessing.Process.sentinel"><code class="xref py py-attr docutils literal"><span class="pre">sentinel</span></code></a> attribute of a <a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a> object.</span></li></ul><p><span class="yiyi-st" id="yiyi-846">A connection or socket object is ready when there is data available to be read from it, or the other end has been closed.</span></p><p><span class="yiyi-st" id="yiyi-847"><strong>Unix</strong>: <code class="docutils literal"><span class="pre">wait(object_list,</span> <span class="pre">timeout)</span></code> almost equivalent <code class="docutils literal"><span class="pre">select.select(object_list,</span> <span class="pre">[],</span> <span class="pre">[],</span> <span class="pre">timeout)</span></code>. </span><span class="yiyi-st" id="yiyi-848">The difference is that, if <a class="reference internal" href="select.html#select.select" title="select.select"><code class="xref py py-func docutils literal"><span class="pre">select.select()</span></code></a> is interrupted by a signal, it can raise <a class="reference internal" href="exceptions.html#OSError" title="OSError"><code class="xref py py-exc docutils literal"><span class="pre">OSError</span></code></a> with an error number of <code class="docutils literal"><span class="pre">EINTR</span></code>, whereas <a class="reference internal" href="#multiprocessing.connection.wait" title="multiprocessing.connection.wait"><code class="xref py py-func docutils literal"><span class="pre">wait()</span></code></a> will not.</span></p><p><span class="yiyi-st" id="yiyi-849"><strong>Windows</strong>: An item in <em>object_list</em> must either be an integer handle which is waitable (according to the definition used by the documentation of the Win32 function <code class="docutils literal"><span class="pre">WaitForMultipleObjects()</span></code>) or it can be an object with a <code class="xref py py-meth docutils literal"><span class="pre">fileno()</span></code> method which returns a socket handle or pipe handle. </span><span class="yiyi-st" id="yiyi-850">(Note that pipe handles and socket handles are <strong>not</strong> waitable handles.)</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-851"><span class="versionmodified">New in version 3.3.</span></span></p></div></dd></dl><p><span class="yiyi-st" id="yiyi-852"><strong>Examples</strong></span></p><p><span class="yiyi-st" id="yiyi-853">The following server code creates a listener which uses <code class="docutils literal"><span class="pre">'secret</span> <span class="pre">password'</span></code> as an authentication key. </span><span class="yiyi-st" id="yiyi-854">It then waits for a connection and sends some data to the client:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing.connection</span> <span class="k">import</span> <span class="n">Listener</span>
<span class="kn">from</span> <span class="nn">array</span> <span class="k">import</span> <span class="n">array</span>
<span class="n">address</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">6000</span><span class="p">)</span> <span class="c1"># family is deduced to be 'AF_INET'</span>
<span class="k">with</span> <span class="n">Listener</span><span class="p">(</span><span class="n">address</span><span class="p">,</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'secret password'</span><span class="p">)</span> <span class="k">as</span> <span class="n">listener</span><span class="p">:</span>
<span class="k">with</span> <span class="n">listener</span><span class="o">.</span><span class="n">accept</span><span class="p">()</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'connection accepted from'</span><span class="p">,</span> <span class="n">listener</span><span class="o">.</span><span class="n">last_accepted</span><span class="p">)</span>
<span class="n">conn</span><span class="o">.</span><span class="n">send</span><span class="p">([</span><span class="mf">2.25</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'junk'</span><span class="p">,</span> <span class="nb">float</span><span class="p">])</span>
<span class="n">conn</span><span class="o">.</span><span class="n">send_bytes</span><span class="p">(</span><span class="n">b</span><span class="s1">'hello'</span><span class="p">)</span>
<span class="n">conn</span><span class="o">.</span><span class="n">send_bytes</span><span class="p">(</span><span class="n">array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="mi">1729</span><span class="p">]))</span>
</code></pre><p><span class="yiyi-st" id="yiyi-855">The following code connects to the server and receives some data from the server:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing.connection</span> <span class="k">import</span> <span class="n">Client</span>
<span class="kn">from</span> <span class="nn">array</span> <span class="k">import</span> <span class="n">array</span>
<span class="n">address</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">6000</span><span class="p">)</span>
<span class="k">with</span> <span class="n">Client</span><span class="p">(</span><span class="n">address</span><span class="p">,</span> <span class="n">authkey</span><span class="o">=</span><span class="n">b</span><span class="s1">'secret password'</span><span class="p">)</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">recv</span><span class="p">())</span> <span class="c1"># =&gt; [2.25, None, 'junk', float]</span>
<span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">recv_bytes</span><span class="p">())</span> <span class="c1"># =&gt; 'hello'</span>
<span class="n">arr</span> <span class="o">=</span> <span class="n">array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="n">conn</span><span class="o">.</span><span class="n">recv_bytes_into</span><span class="p">(</span><span class="n">arr</span><span class="p">))</span> <span class="c1"># =&gt; 8</span>
<span class="nb">print</span><span class="p">(</span><span class="n">arr</span><span class="p">)</span> <span class="c1"># =&gt; array('i', [42, 1729, 0, 0, 0])</span>
</code></pre><p><span class="yiyi-st" id="yiyi-856">The following code uses <a class="reference internal" href="#multiprocessing.connection.wait" title="multiprocessing.connection.wait"><code class="xref py py-func docutils literal"><span class="pre">wait()</span></code></a> to wait for messages from multiple processes at once:</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">time</span><span class="o">,</span> <span class="nn">random</span>
<span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Pipe</span><span class="p">,</span> <span class="n">current_process</span>
<span class="kn">from</span> <span class="nn">multiprocessing.connection</span> <span class="k">import</span> <span class="n">wait</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">w</span><span class="p">):</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">w</span><span class="o">.</span><span class="n">send</span><span class="p">((</span><span class="n">i</span><span class="p">,</span> <span class="n">current_process</span><span class="p">()</span><span class="o">.</span><span class="n">name</span><span class="p">))</span>
<span class="n">w</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">readers</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">):</span>
<span class="n">r</span><span class="p">,</span> <span class="n">w</span> <span class="o">=</span> <span class="n">Pipe</span><span class="p">(</span><span class="n">duplex</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">readers</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">foo</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">w</span><span class="p">,))</span>
<span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="c1"># We close the writable end of the pipe now to be sure that</span>
<span class="c1"># p is the only process which owns a handle for it. This</span>
<span class="c1"># ensures that when p closes its handle for the writable end,</span>
<span class="c1"># wait() will promptly report the readable end as being ready.</span>
<span class="n">w</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">while</span> <span class="n">readers</span><span class="p">:</span>
<span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">wait</span><span class="p">(</span><span class="n">readers</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">msg</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">recv</span><span class="p">()</span>
<span class="k">except</span> <span class="ne">EOFError</span><span class="p">:</span>
<span class="n">readers</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">r</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
</code></pre><div class="section" id="address-formats"><h4><span class="yiyi-st" id="yiyi-857">17.2.2.10.1. </span><span class="yiyi-st" id="yiyi-858">Address Formats</span></h4><ul><li><p class="first"><span class="yiyi-st" id="yiyi-859">An <code class="docutils literal"><span class="pre">'AF_INET'</span></code> address is a tuple of the form <code class="docutils literal"><span class="pre">(hostname,</span> <span class="pre">port)</span></code> where <em>hostname</em> is a string and <em>port</em> is an integer.</span></p></li><li><p class="first"><span class="yiyi-st" id="yiyi-860">An <code class="docutils literal"><span class="pre">'AF_UNIX'</span></code> address is a string representing a filename on the filesystem.</span></p></li><li><dl class="first docutils"><dt><span class="yiyi-st" id="yiyi-861">An <code class="docutils literal"><span class="pre">'AF_PIPE'</span></code> address is a string of the form</span></dt><dd><p class="first last"><span class="yiyi-st" id="yiyi-862"><code class="samp docutils literal"><span class="pre">r'\\.\pipe\</span><em><span class="pre">PipeName</span></em><span class="pre">'</span></code>. </span><span class="yiyi-st" id="yiyi-863">To use <a class="reference internal" href="#multiprocessing.connection.Client" title="multiprocessing.connection.Client"><code class="xref py py-func docutils literal"><span class="pre">Client()</span></code></a> to connect to a named pipe on a remote computer called <em>ServerName</em> one should use an address of the form <code class="samp docutils literal"><span class="pre">r'\\</span><em><span class="pre">ServerName</span></em><span class="pre">\pipe\</span><em><span class="pre">PipeName</span></em><span class="pre">'</span></code> instead.</span></p></dd></dl></li></ul><p><span class="yiyi-st" id="yiyi-864">Note that any string beginning with two backslashes is assumed by default to be an <code class="docutils literal"><span class="pre">'AF_PIPE'</span></code> address rather than an <code class="docutils literal"><span class="pre">'AF_UNIX'</span></code> address.</span></p></div></div><div class="section" id="authentication-keys"><h3><span class="yiyi-st" id="yiyi-865">17.2.2.11. </span><span class="yiyi-st" id="yiyi-866">Authentication keys</span></h3><p><span class="yiyi-st" id="yiyi-867">When one uses <a class="reference internal" href="#multiprocessing.Connection.recv" title="multiprocessing.Connection.recv"><code class="xref py py-meth docutils literal"><span class="pre">Connection.recv</span></code></a>, the data received is automatically unpickled. </span><span class="yiyi-st" id="yiyi-868">Unfortunately unpickling data from an untrusted source is a security risk. </span><span class="yiyi-st" id="yiyi-869">Therefore <a class="reference internal" href="#multiprocessing.connection.Listener" title="multiprocessing.connection.Listener"><code class="xref py py-class docutils literal"><span class="pre">Listener</span></code></a> and <a class="reference internal" href="#multiprocessing.connection.Client" title="multiprocessing.connection.Client"><code class="xref py py-func docutils literal"><span class="pre">Client()</span></code></a> use the <a class="reference internal" href="hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal"><span class="pre">hmac</span></code></a> module to provide digest authentication.</span></p><p><span class="yiyi-st" id="yiyi-870">An authentication key is a byte string which can be thought of as a password: once a connection is established both ends will demand proof that the other knows the authentication key. </span><span class="yiyi-st" id="yiyi-871">(Demonstrating that both ends are using the same key does <strong>not</strong> involve sending the key over the connection.)</span></p><p><span class="yiyi-st" id="yiyi-872">If authentication is requested but no authentication key is specified then the return value of <code class="docutils literal"><span class="pre">current_process().authkey</span></code> is used (see <a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a>). </span><span class="yiyi-st" id="yiyi-873">This value will be automatically inherited by any <a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a> object that the current process creates. </span><span class="yiyi-st" id="yiyi-874">This means that (by default) all processes of a multi-process program will share a single authentication key which can be used when setting up connections between themselves.</span></p><p><span class="yiyi-st" id="yiyi-875">Suitable authentication keys can also be generated by using <a class="reference internal" href="os.html#os.urandom" title="os.urandom"><code class="xref py py-func docutils literal"><span class="pre">os.urandom()</span></code></a>.</span></p></div><div class="section" id="logging"><h3><span class="yiyi-st" id="yiyi-876">17.2.2.12. </span><span class="yiyi-st" id="yiyi-877">Logging</span></h3><p><span class="yiyi-st" id="yiyi-878">Some support for logging is available. </span><span class="yiyi-st" id="yiyi-879">Note, however, that the <a class="reference internal" href="logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal"><span class="pre">logging</span></code></a> package does not use process shared locks so it is possible (depending on the handler type) for messages from different processes to get mixed up.</span></p><dl class="function"><dt id="multiprocessing.get_logger"><span class="yiyi-st" id="yiyi-880"> <code class="descclassname">multiprocessing.</code><code class="descname">get_logger</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-881">Returns the logger used by <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>. </span><span class="yiyi-st" id="yiyi-882">If necessary, a new one will be created.</span></p><p><span class="yiyi-st" id="yiyi-883">When first created the logger has level <code class="xref py py-data docutils literal"><span class="pre">logging.NOTSET</span></code> and no default handler. </span><span class="yiyi-st" id="yiyi-884">Messages sent to this logger will not by default propagate to the root logger.</span></p><p><span class="yiyi-st" id="yiyi-885">Note that on Windows child processes will only inherit the level of the parent process’s logger – any other customization of the logger will not be inherited.</span></p></dd></dl><dl class="function"><dt id="multiprocessing.log_to_stderr"><span class="yiyi-st" id="yiyi-886"> <code class="descclassname">multiprocessing.</code><code class="descname">log_to_stderr</code><span class="sig-paren">(</span><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-887">This function performs a call to <a class="reference internal" href="#multiprocessing.get_logger" title="multiprocessing.get_logger"><code class="xref py py-func docutils literal"><span class="pre">get_logger()</span></code></a> but in addition to returning the logger created by get_logger, it adds a handler which sends output to <a class="reference internal" href="sys.html#sys.stderr" title="sys.stderr"><code class="xref py py-data docutils literal"><span class="pre">sys.stderr</span></code></a> using format <code class="docutils literal"><span class="pre">'[%(levelname)s/%(processName)s]</span> <span class="pre">%(message)s'</span></code>.</span></p></dd></dl><p><span class="yiyi-st" id="yiyi-888">Below is an example session with logging turned on:</span></p><pre><code class="language-python"><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">multiprocessing</span><span class="o">,</span> <span class="nn">logging</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">logger</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">log_to_stderr</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="s1">'doomed'</span><span class="p">)</span>
<span class="go">[WARNING/MainProcess] doomed</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">m</span> <span class="o">=</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Manager</span><span class="p">()</span>
<span class="go">[INFO/SyncManager-...] child process calling self.run()</span>
<span class="go">[INFO/SyncManager-...] created temp directory /.../pymp-...</span>
<span class="go">[INFO/SyncManager-...] manager serving at '/.../listener-...'</span>
<span class="gp">&gt;&gt;&gt; </span><span class="k">del</span> <span class="n">m</span>
<span class="go">[INFO/MainProcess] sending shutdown message to manager</span>
<span class="go">[INFO/SyncManager-...] manager exiting with exitcode 0</span>
</code></pre><p><span class="yiyi-st" id="yiyi-889">For a full table of logging levels, see the <a class="reference internal" href="logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal"><span class="pre">logging</span></code></a> module.</span></p></div><div class="section" id="module-multiprocessing.dummy"><h3><span class="yiyi-st" id="yiyi-890">17.2.2.13. </span><span class="yiyi-st" id="yiyi-891">The <a class="reference internal" href="#module-multiprocessing.dummy" title="multiprocessing.dummy: Dumb wrapper around threading."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.dummy</span></code></a> module</span></h3><p><span class="yiyi-st" id="yiyi-892"><a class="reference internal" href="#module-multiprocessing.dummy" title="multiprocessing.dummy: Dumb wrapper around threading."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing.dummy</span></code></a> replicates the API of <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> but is no more than a wrapper around the <a class="reference internal" href="threading.html#module-threading" title="threading: Thread-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">threading</span></code></a> module.</span></p></div></div><div class="section" id="programming-guidelines"><h2><span class="yiyi-st" id="yiyi-893">17.2.3. </span><span class="yiyi-st" id="yiyi-894">编程指南</span></h2><p><span class="yiyi-st" id="yiyi-895">There are certain guidelines and idioms which should be adhered to when using <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>.</span></p><div class="section" id="all-start-methods"><h3><span class="yiyi-st" id="yiyi-896">17.2.3.1. </span><span class="yiyi-st" id="yiyi-897">所有启动方法</span></h3><p><span class="yiyi-st" id="yiyi-898">The following applies to all start methods.</span></p><p><span class="yiyi-st" id="yiyi-899">Avoid shared state</span></p><span class="yiyi-st" id="yiyi-908"><blockquote><div><p>尽可能避免在进程之间移动大量数据。</p> <p>最好坚持使用队列或管道进行进程之间的通信，而不是使用较低的级别同步原语。</p></div></blockquote></span><p><span class="yiyi-st" id="yiyi-900">Picklability</span></p><span class="yiyi-st" id="yiyi-909"><blockquote><div>确保代理方法的参数是可选的。</div></blockquote></span><p><span class="yiyi-st" id="yiyi-901">Thread safety of proxies</span></p><span class="yiyi-st" id="yiyi-910"> <blockquote> <div><p>Do not use a proxy object from more than one thread unless you protect it with a lock.</p> <p>(There is never a problem with different processes using the <em>same</em> proxy.)</p> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-902">Joining zombie processes</span></p><span class="yiyi-st" id="yiyi-911"> <blockquote> <div>On Unix when a process finishes but has not been joined it becomes a zombie. There should never be very many because each time a new process starts (or <a class="reference internal" href="#multiprocessing.active_children" title="multiprocessing.active_children"><code class="xref py py-func docutils literal"><span class="pre">active_children()</span></code></a> is called) all completed processes which have not yet been joined will be joined. Also calling a finished process’s <a class="reference internal" href="#multiprocessing.Process.is_alive" title="multiprocessing.Process.is_alive"><code class="xref py py-meth docutils literal"><span class="pre">Process.is_alive</span></code></a> will join the process. Even so it is probably good practice to explicitly join all the processes that you start.</div></blockquote></span><p><span class="yiyi-st" id="yiyi-903">Better to inherit than pickle/unpickle</span></p><span class="yiyi-st" id="yiyi-912"> <blockquote> <div>When using the <em>spawn</em> or <em>forkserver</em> start methods many types from <a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead you should arrange the program so that a process which needs access to a shared resource created elsewhere can inherit it from an ancestor process.</div></blockquote></span><p><span class="yiyi-st" id="yiyi-904">Avoid terminating processes</span></p><span class="yiyi-st" id="yiyi-913"> <blockquote> <div><p>Using the <a class="reference internal" href="#multiprocessing.Process.terminate" title="multiprocessing.Process.terminate"><code class="xref py py-meth docutils literal"><span class="pre">Process.terminate</span></code></a> method to stop a process is liable to cause any shared resources (such as locks, semaphores, pipes and queues) currently being used by the process to become broken or unavailable to other processes.</p> <p>Therefore it is probably best to only consider using <a class="reference internal" href="#multiprocessing.Process.terminate" title="multiprocessing.Process.terminate"><code class="xref py py-meth docutils literal"><span class="pre">Process.terminate</span></code></a> on processes which never use any shared resources.</p> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-905">Joining processes that use queues</span></p><span class="yiyi-st" id="yiyi-914"> <blockquote> <div><p>Bear in mind that a process that has put items in a queue will wait before terminating until all the buffered items are fed by the “feeder” thread to the underlying pipe. (The child process can call the <a class="reference internal" href="#multiprocessing.Queue.cancel_join_thread" title="multiprocessing.Queue.cancel_join_thread"><code class="xref py py-meth docutils literal"><span class="pre">Queue.cancel_join_thread</span></code></a> method of the queue to avoid this behaviour.)</p> <p>This means that whenever you use a queue you need to make sure that all items which have been put on the queue will eventually be removed before the process is joined. Otherwise you cannot be sure that processes which have put items on the queue will terminate. Remember also that non-daemonic processes will be joined automatically.</p> <p>An example which will deadlock is the following:</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">q</span><span class="p">):</span> <span class="n">q</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'X'</span> <span class="o">*</span> <span class="mi">1000000</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">queue</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">queue</span><span class="p">,))</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="n">p</span><span class="o">.</span><span class="n">join</span><span class="p">()</span> <span class="c1"># this deadlocks</span> <span class="n">obj</span> <span class="o">=</span> <span class="n">queue</span><span class="o">.</span><span class="n">get</span><span class="p">()</span> </code></pre> <p>A fix here would be to swap the last two lines (or simply remove the <code class="docutils literal"><span class="pre">p.join()</span></code> line).</p> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-906">Explicitly pass resources to child processes</span></p><span class="yiyi-st" id="yiyi-915"> <blockquote> <div><p>On Unix using the <em>fork</em> start method, a child process can make use of a shared resource created in a parent process using a global resource. However, it is better to pass the object as an argument to the constructor for the child process.</p> <p>Apart from making the code (potentially) compatible with Windows and the other start methods this also ensures that as long as the child process is still alive the object will not be garbage collected in the parent process. This might be important if some resource is freed when the object is garbage collected in the parent process.</p> <p>So for instance</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Lock</span> <span class="k">def</span> <span class="nf">f</span><span class="p">():</span> <span class="o">...</span> <span class="n">do</span> <span class="n">something</span> <span class="n">using</span> <span class="s2">"lock"</span> <span class="o">...</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">lock</span> <span class="o">=</span> <span class="n">Lock</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">)</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> </code></pre> <p>should be rewritten as</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Lock</span> <span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">l</span><span class="p">):</span> <span class="o">...</span> <span class="n">do</span> <span class="n">something</span> <span class="n">using</span> <span class="s2">"l"</span> <span class="o">...</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">lock</span> <span class="o">=</span> <span class="n">Lock</span><span class="p">()</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">f</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">lock</span><span class="p">,))</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> </code></pre> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-907">Beware of replacing <a class="reference internal" href="sys.html#sys.stdin" title="sys.stdin"><code class="xref py py-data docutils literal"><span class="pre">sys.stdin</span></code></a> with a “file like object”</span></p><span class="yiyi-st" id="yiyi-916"> <blockquote> <div><p><a class="reference internal" href="#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a> originally unconditionally called:</p> <pre><code class="language-python"><span></span><span class="n">os</span><span class="o">.</span><span class="n">close</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">stdin</span><span class="o">.</span><span class="n">fileno</span><span class="p">())</span> </code></pre> <p>in the <code class="xref py py-meth docutils literal"><span class="pre">multiprocessing.Process._bootstrap()</span></code> method — this resulted in issues with processes-in-processes. This has been changed to:</p> <pre><code class="language-python"><span></span><span class="n">sys</span><span class="o">.</span><span class="n">stdin</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="n">sys</span><span class="o">.</span><span class="n">stdin</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">devnull</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">O_RDONLY</span><span class="p">),</span> <span class="n">closefd</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </code></pre> <p>Which solves the fundamental issue of processes colliding with each other resulting in a bad file descriptor error, but introduces a potential danger to applications which replace <a class="reference internal" href="sys.html#sys.stdin" title="sys.stdin"><code class="xref py py-func docutils literal"><span class="pre">sys.stdin()</span></code></a> with a “file-like object” with output buffering. This danger is that if multiple processes call <a class="reference internal" href="io.html#io.IOBase.close" title="io.IOBase.close"><code class="xref py py-meth docutils literal"><span class="pre">close()</span></code></a> on this file-like object, it could result in the same data being flushed to the object multiple times, resulting in corruption.</p> <p>If you write a file-like object and implement your own caching, you can make it fork-safe by storing the pid whenever you append to the cache, and discarding the cache when the pid changes. For example:</p> <pre><code class="language-python"><span></span><span class="nd">@property</span> <span class="k">def</span> <span class="nf">cache</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="n">pid</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getpid</span><span class="p">()</span> <span class="k">if</span> <span class="n">pid</span> <span class="o">!=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_pid</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">_pid</span> <span class="o">=</span> <span class="n">pid</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cache</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cache</span> </code></pre> <p>For more information, see <a class="reference external" href="https://bugs.python.org/issue5155">issue 5155</a>, <a class="reference external" href="https://bugs.python.org/issue5313">issue 5313</a> and <a class="reference external" href="https://bugs.python.org/issue5331">issue 5331</a></p> </div></blockquote></span></div><div class="section" id="the-spawn-and-forkserver-start-methods"><h3><span class="yiyi-st" id="yiyi-917">17.2.3.2. </span><span class="yiyi-st" id="yiyi-918">The <em>spawn</em> and <em>forkserver</em> start methods</span></h3><p><span class="yiyi-st" id="yiyi-919">There are a few extra restriction which don’t apply to the <em>fork</em> start method.</span></p><p><span class="yiyi-st" id="yiyi-920">More picklability</span></p><span class="yiyi-st" id="yiyi-923"> <blockquote> <div>Ensure that all arguments to <code class="xref py py-meth docutils literal"><span class="pre">Process.__init__()</span></code> are picklable. Also, if you subclass <a class="reference internal" href="#multiprocessing.Process" title="multiprocessing.Process"><code class="xref py py-class docutils literal"><span class="pre">Process</span></code></a> then make sure that instances will be picklable when the <a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">Process.start</span></code></a> method is called.</div></blockquote></span><p><span class="yiyi-st" id="yiyi-921">Global variables</span></p><span class="yiyi-st" id="yiyi-924"> <blockquote> <div><p>Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that <a class="reference internal" href="#multiprocessing.Process.start" title="multiprocessing.Process.start"><code class="xref py py-meth docutils literal"><span class="pre">Process.start</span></code></a> was called.</p> <p>However, global variables which are just module level constants cause no problems.</p> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-922">Safe importing of main module</span></p><span class="yiyi-st" id="yiyi-925"> <blockquote> <div><p>Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).</p> <p>For example, using the <em>spawn</em> or <em>forkserver</em> start method running the following module would fail with a <a class="reference internal" href="exceptions.html#RuntimeError" title="RuntimeError"><code class="xref py py-exc docutils literal"><span class="pre">RuntimeError</span></code></a>:</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span> <span class="k">def</span> <span class="nf">foo</span><span class="p">():</span> <span class="nb">print</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">foo</span><span class="p">)</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> </code></pre> <p>Instead one should protect the “entry point” of the program by using <code class="docutils literal"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">'__main__':</span></code> as follows:</p> <pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">freeze_support</span><span class="p">,</span> <span class="n">set_start_method</span> <span class="k">def</span> <span class="nf">foo</span><span class="p">():</span> <span class="nb">print</span><span class="p">(</span><span class="s1">'hello'</span><span class="p">)</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span> <span class="n">freeze_support</span><span class="p">()</span> <span class="n">set_start_method</span><span class="p">(</span><span class="s1">'spawn'</span><span class="p">)</span> <span class="n">p</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">foo</span><span class="p">)</span> <span class="n">p</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> </code></pre> <p>(The <code class="docutils literal"><span class="pre">freeze_support()</span></code> line can be omitted if the program will be run normally instead of frozen.)</p> <p>This allows the newly spawned Python interpreter to safely import the module and then run the module’s <code class="docutils literal"><span class="pre">foo()</span></code> function.</p> <p>Similar restrictions apply if a pool or manager is created in the main module.</p> </div></blockquote></span></div></div><div class="section" id="examples"><h2><span class="yiyi-st" id="yiyi-926">17.2.4. </span><span class="yiyi-st" id="yiyi-927">Examples</span></h2><p><span class="yiyi-st" id="yiyi-928">Demonstration of how to create and use customized managers and proxies:</span></p><pre><code class="language-python"><span></span><span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">freeze_support</span>
<span class="kn">from</span> <span class="nn">multiprocessing.managers</span> <span class="k">import</span> <span class="n">BaseManager</span><span class="p">,</span> <span class="n">BaseProxy</span>
<span class="kn">import</span> <span class="nn">operator</span>
<span class="c1">##</span>
<span class="k">class</span> <span class="nc">Foo</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'you called Foo.f()'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">g</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'you called Foo.g()'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_h</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'you called Foo._h()'</span><span class="p">)</span>
<span class="c1"># A simple generator function</span>
<span class="k">def</span> <span class="nf">baz</span><span class="p">():</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">i</span><span class="o">*</span><span class="n">i</span>
<span class="c1"># Proxy type for generator objects</span>
<span class="k">class</span> <span class="nc">GeneratorProxy</span><span class="p">(</span><span class="n">BaseProxy</span><span class="p">):</span>
<span class="n">_exposed_</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'__next__'</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span>
<span class="k">def</span> <span class="nf">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_callmethod</span><span class="p">(</span><span class="s1">'__next__'</span><span class="p">)</span>
<span class="c1"># Function to return the operator module</span>
<span class="k">def</span> <span class="nf">get_operator_module</span><span class="p">():</span>
<span class="k">return</span> <span class="n">operator</span>
<span class="c1">##</span>
<span class="k">class</span> <span class="nc">MyManager</span><span class="p">(</span><span class="n">BaseManager</span><span class="p">):</span>
<span class="k">pass</span>
<span class="c1"># register the Foo class; make `f()` and `g()` accessible via proxy</span>
<span class="n">MyManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'Foo1'</span><span class="p">,</span> <span class="n">Foo</span><span class="p">)</span>
<span class="c1"># register the Foo class; make `g()` and `_h()` accessible via proxy</span>
<span class="n">MyManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'Foo2'</span><span class="p">,</span> <span class="n">Foo</span><span class="p">,</span> <span class="n">exposed</span><span class="o">=</span><span class="p">(</span><span class="s1">'g'</span><span class="p">,</span> <span class="s1">'_h'</span><span class="p">))</span>
<span class="c1"># register the generator function baz; use `GeneratorProxy` to make proxies</span>
<span class="n">MyManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'baz'</span><span class="p">,</span> <span class="n">baz</span><span class="p">,</span> <span class="n">proxytype</span><span class="o">=</span><span class="n">GeneratorProxy</span><span class="p">)</span>
<span class="c1"># register get_operator_module(); make public functions accessible via proxy</span>
<span class="n">MyManager</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="s1">'operator'</span><span class="p">,</span> <span class="n">get_operator_module</span><span class="p">)</span>
<span class="c1">##</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">manager</span> <span class="o">=</span> <span class="n">MyManager</span><span class="p">()</span>
<span class="n">manager</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'-'</span> <span class="o">*</span> <span class="mi">20</span><span class="p">)</span>
<span class="n">f1</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">Foo1</span><span class="p">()</span>
<span class="n">f1</span><span class="o">.</span><span class="n">f</span><span class="p">()</span>
<span class="n">f1</span><span class="o">.</span><span class="n">g</span><span class="p">()</span>
<span class="k">assert</span> <span class="ow">not</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">f1</span><span class="p">,</span> <span class="s1">'_h'</span><span class="p">)</span>
<span class="k">assert</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">f1</span><span class="o">.</span><span class="n">_exposed_</span><span class="p">)</span> <span class="o">==</span> <span class="nb">sorted</span><span class="p">([</span><span class="s1">'f'</span><span class="p">,</span> <span class="s1">'g'</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'-'</span> <span class="o">*</span> <span class="mi">20</span><span class="p">)</span>
<span class="n">f2</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">Foo2</span><span class="p">()</span>
<span class="n">f2</span><span class="o">.</span><span class="n">g</span><span class="p">()</span>
<span class="n">f2</span><span class="o">.</span><span class="n">_h</span><span class="p">()</span>
<span class="k">assert</span> <span class="ow">not</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">f2</span><span class="p">,</span> <span class="s1">'f'</span><span class="p">)</span>
<span class="k">assert</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">f2</span><span class="o">.</span><span class="n">_exposed_</span><span class="p">)</span> <span class="o">==</span> <span class="nb">sorted</span><span class="p">([</span><span class="s1">'g'</span><span class="p">,</span> <span class="s1">'_h'</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'-'</span> <span class="o">*</span> <span class="mi">20</span><span class="p">)</span>
<span class="n">it</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">baz</span><span class="p">()</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">it</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'&lt;</span><span class="si">%d</span><span class="s1">&gt;'</span> <span class="o">%</span> <span class="n">i</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">' '</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'-'</span> <span class="o">*</span> <span class="mi">20</span><span class="p">)</span>
<span class="n">op</span> <span class="o">=</span> <span class="n">manager</span><span class="o">.</span><span class="n">operator</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'op.add(23, 45) ='</span><span class="p">,</span> <span class="n">op</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="mi">23</span><span class="p">,</span> <span class="mi">45</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'op.pow(2, 94) ='</span><span class="p">,</span> <span class="n">op</span><span class="o">.</span><span class="n">pow</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">94</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'op._exposed_ ='</span><span class="p">,</span> <span class="n">op</span><span class="o">.</span><span class="n">_exposed_</span><span class="p">)</span>
<span class="c1">##</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">freeze_support</span><span class="p">()</span>
<span class="n">test</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-929">Using <a class="reference internal" href="#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal"><span class="pre">Pool</span></code></a>:</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">multiprocessing</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">random</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="c1">#</span>
<span class="c1"># Functions used by test code</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="n">args</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">return</span> <span class="s1">'</span><span class="si">%s</span><span class="s1"> says that </span><span class="si">%s%s</span><span class="s1"> = </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span>
<span class="n">multiprocessing</span><span class="o">.</span><span class="n">current_process</span><span class="p">()</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
<span class="n">func</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">result</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">calculatestar</span><span class="p">(</span><span class="n">args</span><span class="p">):</span>
<span class="k">return</span> <span class="n">calculate</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">mul</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span> <span class="o">*</span> <span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">())</span>
<span class="k">return</span> <span class="n">a</span> <span class="o">*</span> <span class="n">b</span>
<span class="k">def</span> <span class="nf">plus</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span> <span class="o">*</span> <span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">())</span>
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="mf">1.0</span> <span class="o">/</span> <span class="p">(</span><span class="n">x</span> <span class="o">-</span> <span class="mf">5.0</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">pow3</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">**</span> <span class="mi">3</span>
<span class="k">def</span> <span class="nf">noop</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">pass</span>
<span class="c1">#</span>
<span class="c1"># Test code</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">PROCESSES</span> <span class="o">=</span> <span class="mi">4</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Creating pool with </span><span class="si">%d</span><span class="s1"> processes</span><span class="se">\n</span><span class="s1">'</span> <span class="o">%</span> <span class="n">PROCESSES</span><span class="p">)</span>
<span class="k">with</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">Pool</span><span class="p">(</span><span class="n">PROCESSES</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span>
<span class="c1">#</span>
<span class="c1"># Tests</span>
<span class="c1">#</span>
<span class="n">TASKS</span> <span class="o">=</span> <span class="p">[(</span><span class="n">mul</span><span class="p">,</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="mi">7</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span> <span class="o">+</span> \
<span class="p">[(</span><span class="n">plus</span><span class="p">,</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="mi">8</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="n">results</span> <span class="o">=</span> <span class="p">[</span><span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">calculate</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">TASKS</span><span class="p">]</span>
<span class="n">imap_it</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">calculatestar</span><span class="p">,</span> <span class="n">TASKS</span><span class="p">)</span>
<span class="n">imap_unordered_it</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap_unordered</span><span class="p">(</span><span class="n">calculatestar</span><span class="p">,</span> <span class="n">TASKS</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Ordered results using pool.apply_async():'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">results</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">r</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Ordered results using pool.imap():'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">imap_it</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Unordered results using pool.imap_unordered():'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">imap_unordered_it</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Ordered results using pool.map() --- will block till complete:'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">pool</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">calculatestar</span><span class="p">,</span> <span class="n">TASKS</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="c1">#</span>
<span class="c1"># Test error handling</span>
<span class="c1">#</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Testing error handling:'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pool</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,)))</span>
<span class="k">except</span> <span class="ne">ZeroDivisionError</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">Got ZeroDivisionError as expected from pool.apply()'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">AssertionError</span><span class="p">(</span><span class="s1">'expected ZeroDivisionError'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pool</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))))</span>
<span class="k">except</span> <span class="ne">ZeroDivisionError</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">Got ZeroDivisionError as expected from pool.map()'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">AssertionError</span><span class="p">(</span><span class="s1">'expected ZeroDivisionError'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)))))</span>
<span class="k">except</span> <span class="ne">ZeroDivisionError</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">Got ZeroDivisionError as expected from list(pool.imap())'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">AssertionError</span><span class="p">(</span><span class="s1">'expected ZeroDivisionError'</span><span class="p">)</span>
<span class="n">it</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">x</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">it</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">ZeroDivisionError</span><span class="p">:</span>
<span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">5</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span>
<span class="k">break</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">5</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">AssertionError</span><span class="p">(</span><span class="s1">'expected ZeroDivisionError'</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">9</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">Got ZeroDivisionError as expected from IMapIterator.next()'</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="c1">#</span>
<span class="c1"># Testing timeouts</span>
<span class="c1">#</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Testing ApplyResult.get() with timeout:'</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">' '</span><span class="p">)</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">calculate</span><span class="p">,</span> <span class="n">TASKS</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="k">while</span> <span class="mi">1</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'</span><span class="se">\n\t</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="mf">0.02</span><span class="p">))</span>
<span class="k">break</span>
<span class="k">except</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">TimeoutError</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Testing IMapIterator.next() with timeout:'</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s1">' '</span><span class="p">)</span>
<span class="n">it</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">calculatestar</span><span class="p">,</span> <span class="n">TASKS</span><span class="p">)</span>
<span class="k">while</span> <span class="mi">1</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'</span><span class="se">\n\t</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">it</span><span class="o">.</span><span class="n">next</span><span class="p">(</span><span class="mf">0.02</span><span class="p">))</span>
<span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span>
<span class="k">break</span>
<span class="k">except</span> <span class="n">multiprocessing</span><span class="o">.</span><span class="n">TimeoutError</span><span class="p">:</span>
<span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'.'</span><span class="p">)</span>
<span class="nb">print</span><span class="p">()</span>
<span class="nb">print</span><span class="p">()</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">multiprocessing</span><span class="o">.</span><span class="n">freeze_support</span><span class="p">()</span>
<span class="n">test</span><span class="p">()</span>
</code></pre><p><span class="yiyi-st" id="yiyi-930">An example showing how to use queues to feed tasks to a collection of worker processes and collect the results:</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">random</span>
<span class="kn">from</span> <span class="nn">multiprocessing</span> <span class="k">import</span> <span class="n">Process</span><span class="p">,</span> <span class="n">Queue</span><span class="p">,</span> <span class="n">current_process</span><span class="p">,</span> <span class="n">freeze_support</span>
<span class="c1">#</span>
<span class="c1"># Function run by worker processes</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">worker</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">output</span><span class="p">):</span>
<span class="k">for</span> <span class="n">func</span><span class="p">,</span> <span class="n">args</span> <span class="ow">in</span> <span class="nb">iter</span><span class="p">(</span><span class="nb">input</span><span class="o">.</span><span class="n">get</span><span class="p">,</span> <span class="s1">'STOP'</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">calculate</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="n">output</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
<span class="c1">#</span>
<span class="c1"># Function used to calculate result</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="n">args</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">return</span> <span class="s1">'</span><span class="si">%s</span><span class="s1"> says that </span><span class="si">%s%s</span><span class="s1"> = </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> \
<span class="p">(</span><span class="n">current_process</span><span class="p">()</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">func</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">result</span><span class="p">)</span>
<span class="c1">#</span>
<span class="c1"># Functions referenced by tasks</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">mul</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span><span class="o">*</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">())</span>
<span class="k">return</span> <span class="n">a</span> <span class="o">*</span> <span class="n">b</span>
<span class="k">def</span> <span class="nf">plus</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span><span class="o">*</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">())</span>
<span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span>
<span class="c1">#</span>
<span class="c1">#</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="n">NUMBER_OF_PROCESSES</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">TASKS1</span> <span class="o">=</span> <span class="p">[(</span><span class="n">mul</span><span class="p">,</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="mi">7</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">)]</span>
<span class="n">TASKS2</span> <span class="o">=</span> <span class="p">[(</span><span class="n">plus</span><span class="p">,</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="mi">8</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="c1"># Create queues</span>
<span class="n">task_queue</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span>
<span class="n">done_queue</span> <span class="o">=</span> <span class="n">Queue</span><span class="p">()</span>
<span class="c1"># Submit tasks</span>
<span class="k">for</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">TASKS1</span><span class="p">:</span>
<span class="n">task_queue</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
<span class="c1"># Start worker processes</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">NUMBER_OF_PROCESSES</span><span class="p">):</span>
<span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">worker</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">task_queue</span><span class="p">,</span> <span class="n">done_queue</span><span class="p">))</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="c1"># Get and print results</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Unordered results:'</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">TASKS1</span><span class="p">)):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">done_queue</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="c1"># Add more tasks using `put()`</span>
<span class="k">for</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">TASKS2</span><span class="p">:</span>
<span class="n">task_queue</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
<span class="c1"># Get and print some more results</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">TASKS2</span><span class="p">)):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="se">\t</span><span class="s1">'</span><span class="p">,</span> <span class="n">done_queue</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="c1"># Tell child processes to stop</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">NUMBER_OF_PROCESSES</span><span class="p">):</span>
<span class="n">task_queue</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="s1">'STOP'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
<span class="n">freeze_support</span><span class="p">()</span>
<span class="n">test</span><span class="p">()</span>
</code></pre></div></div></div>
Reference in New Issue View Git Blame Copy Permalink