mirror of
https://github.com/fofolee/uTools-Manuals.git
synced 2025-06-08 23:14:06 +08:00
78 lines
49 KiB
HTML
78 lines
49 KiB
HTML
<div class="body" role="main"><div class="section" id="module-concurrent.futures"><h1><span class="yiyi-st" id="yiyi-10">17.4。 <a class="reference internal" href="#module-concurrent.futures" title="concurrent.futures: Execute computations concurrently using threads or processes."><code class="xref py py-mod docutils literal"><span class="pre">concurrent.futures</span></code></a> - 启动并行任务</span></h1><div class="versionadded"><p><span class="yiyi-st" id="yiyi-11"><span class="versionmodified">版本3.2中的新功能。</span></span></p></div><p><span class="yiyi-st" id="yiyi-12"><strong>源代码:</strong> <a class="reference external" href="https://hg.python.org/cpython/file/3.5/Lib/concurrent/futures/thread.py">Lib / concurrent / futures / thread.py</a>和<a class="reference external" href="https://hg.python.org/cpython/file/3.5/Lib/concurrent/futures/process.py">Lib / concurrent / futures / process.py</a></span></p><p><span class="yiyi-st" id="yiyi-13"><a class="reference internal" href="#module-concurrent.futures" title="concurrent.futures: Execute computations concurrently using threads or processes."><code class="xref py py-mod docutils literal"><span class="pre">concurrent.futures</span></code></a>模块为异步执行可调用项提供了一个高级接口。</span></p><p><span class="yiyi-st" id="yiyi-14">异步操作可以由线程实现(通过<a class="reference internal" href="#concurrent.futures.ThreadPoolExecutor" title="concurrent.futures.ThreadPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ThreadPoolExecutor</span></code></a>),或者由进程实现(通过<a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>)。</span><span class="yiyi-st" id="yiyi-15">两者实现了相同的接口,这些接口是定义在抽象类<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>中的。</span></p><div class="section" id="executor-objects"><h2><span class="yiyi-st" id="yiyi-16">17.4.1. </span><span class="yiyi-st" id="yiyi-17">执行程序对象</span></h2><dl class="class"><dt id="concurrent.futures.Executor"><span class="yiyi-st" id="yiyi-18"> <em class="property">class </em><code class="descclassname">concurrent.futures.</code><code class="descname">Executor</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-19">一个抽象类,提供了异步执行调用的方法。</span><span class="yiyi-st" id="yiyi-20">它不应该直接使用,而是通过其具体的子类。</span></p><span class="yiyi-st" id="yiyi-21"> <blockquote> <div><dl class="method"> <dt id="concurrent.futures.Executor.submit"> <code class="descname">submit</code><span class="sig-paren">(</span><em>fn</em>, <em>*args</em>, <em>**kwargs</em><span class="sig-paren">)</span></dt> <dd><p>Schedules the callable, <em>fn</em>, to be executed as <code class="docutils literal"><span class="pre">fn(*args</span> <span class="pre">**kwargs)</span></code> and returns a <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> object representing the execution of the callable.</p> <pre><code class="language-python"><span></span><span class="k">with</span> <span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="k">as</span> <span class="n">executor</span><span class="p">:</span> <span class="n">future</span> <span class="o">=</span> <span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="nb">pow</span><span class="p">,</span> <span class="mi">323</span><span class="p">,</span> <span class="mi">1235</span><span class="p">)</span> <span class="nb">print</span><span class="p">(</span><span class="n">future</span><span class="o">.</span><span class="n">result</span><span class="p">())</span> </code></pre> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Executor.map"> <code class="descname">map</code><span class="sig-paren">(</span><em>func</em>, <em>*iterables</em>, <em>timeout=None</em>, <em>chunksize=1</em><span class="sig-paren">)</span></dt> <dd><p>Equivalent to <a class="reference internal" href="functions.html#map" title="map"><code class="xref py py-func docutils literal"><span class="pre">map(func,</span> <span class="pre">*iterables)</span></code></a> except <em>func</em> is executed asynchronously and several calls to <em>func</em> may be made concurrently. The returned iterator raises a <a class="reference internal" href="#concurrent.futures.TimeoutError" title="concurrent.futures.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">concurrent.futures.TimeoutError</span></code></a> if <a class="reference internal" href="stdtypes.html#iterator.__next__" title="iterator.__next__"><code class="xref py py-meth docutils literal"><span class="pre">__next__()</span></code></a> is called and the result isn’t available after <em>timeout</em> seconds from the original call to <a class="reference internal" href="#concurrent.futures.Executor.map" title="concurrent.futures.Executor.map"><code class="xref py py-meth docutils literal"><span class="pre">Executor.map()</span></code></a>. <em>timeout</em> can be an int or a float. If <em>timeout</em> is not specified or <code class="docutils literal"><span class="pre">None</span></code>, there is no limit to the wait time. If a call raises an exception, then that exception will be raised when its value is retrieved from the iterator. When using <a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>, this method chops <em>iterables</em> into a number of chunks which it submits to the pool as separate tasks. The (approximate) size of these chunks can be specified by setting <em>chunksize</em> to a positive integer. For very long iterables, using a large value for <em>chunksize</em> can significantly improve performance compared to the default size of 1. With <a class="reference internal" href="#concurrent.futures.ThreadPoolExecutor" title="concurrent.futures.ThreadPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ThreadPoolExecutor</span></code></a>, <em>chunksize</em> has no effect.</p> <div class="versionchanged"> <p><span class="versionmodified">Changed in version 3.5: </span>Added the <em>chunksize</em> argument.</p> </div> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Executor.shutdown"> <code class="descname">shutdown</code><span class="sig-paren">(</span><em>wait=True</em><span class="sig-paren">)</span></dt> <dd><p>Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to <a class="reference internal" href="#concurrent.futures.Executor.submit" title="concurrent.futures.Executor.submit"><code class="xref py py-meth docutils literal"><span class="pre">Executor.submit()</span></code></a> and <a class="reference internal" href="#concurrent.futures.Executor.map" title="concurrent.futures.Executor.map"><code class="xref py py-meth docutils literal"><span class="pre">Executor.map()</span></code></a> made after shutdown will raise <a class="reference internal" href="exceptions.html#RuntimeError" title="RuntimeError"><code class="xref py py-exc docutils literal"><span class="pre">RuntimeError</span></code></a>.</p> <p>If <em>wait</em> is <code class="docutils literal"><span class="pre">True</span></code> then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If <em>wait</em> is <code class="docutils literal"><span class="pre">False</span></code> then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of <em>wait</em>, the entire Python program will not exit until all pending futures are done executing.</p> <p>You can avoid having to call this method explicitly if you use the <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal"><span class="pre">with</span></code></a> statement, which will shutdown the <a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a> (waiting as if <a class="reference internal" href="#concurrent.futures.Executor.shutdown" title="concurrent.futures.Executor.shutdown"><code class="xref py py-meth docutils literal"><span class="pre">Executor.shutdown()</span></code></a> were called with <em>wait</em> set to <code class="docutils literal"><span class="pre">True</span></code>):</p> <pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">shutil</span> <span class="k">with</span> <span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="n">e</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">,</span> <span class="s1">'src1.txt'</span><span class="p">,</span> <span class="s1">'dest1.txt'</span><span class="p">)</span> <span class="n">e</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">,</span> <span class="s1">'src2.txt'</span><span class="p">,</span> <span class="s1">'dest2.txt'</span><span class="p">)</span> <span class="n">e</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">,</span> <span class="s1">'src3.txt'</span><span class="p">,</span> <span class="s1">'dest3.txt'</span><span class="p">)</span> <span class="n">e</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">,</span> <span class="s1">'src4.txt'</span><span class="p">,</span> <span class="s1">'dest4.txt'</span><span class="p">)</span> </code></pre> </dd></dl> </div></blockquote></span></dd></dl></div><div class="section" id="threadpoolexecutor"><h2><span class="yiyi-st" id="yiyi-22">17.4.2. </span><span class="yiyi-st" id="yiyi-23">ThreadPoolExecutor</span></h2><p><span class="yiyi-st" id="yiyi-24"><a class="reference internal" href="#concurrent.futures.ThreadPoolExecutor" title="concurrent.futures.ThreadPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ThreadPoolExecutor</span></code></a>是一个使用线程池异步执行调用的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>子类。</span></p><p><span class="yiyi-st" id="yiyi-25">当与<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>相关联的可调用函数等待另一个<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>的结果时,可能会发生死锁。</span><span class="yiyi-st" id="yiyi-26">例如:</span></p><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">time</span>
|
||
<span class="k">def</span> <span class="nf">wait_on_b</span><span class="p">():</span>
|
||
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">result</span><span class="p">())</span> <span class="c1"># b will never complete because it is waiting on a.</span>
|
||
<span class="k">return</span> <span class="mi">5</span>
|
||
|
||
<span class="k">def</span> <span class="nf">wait_on_a</span><span class="p">():</span>
|
||
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">result</span><span class="p">())</span> <span class="c1"># a will never complete because it is waiting on b.</span>
|
||
<span class="k">return</span> <span class="mi">6</span>
|
||
|
||
|
||
<span class="n">executor</span> <span class="o">=</span> <span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
|
||
<span class="n">a</span> <span class="o">=</span> <span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">wait_on_b</span><span class="p">)</span>
|
||
<span class="n">b</span> <span class="o">=</span> <span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">wait_on_a</span><span class="p">)</span>
|
||
</code></pre><p><span class="yiyi-st" id="yiyi-27">和:</span></p><pre><code class="language-python"><span></span><span class="k">def</span> <span class="nf">wait_on_future</span><span class="p">():</span>
|
||
<span class="n">f</span> <span class="o">=</span> <span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="nb">pow</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
|
||
<span class="c1"># This will never complete because there is only one worker thread and</span>
|
||
<span class="c1"># it is executing this function.</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">result</span><span class="p">())</span>
|
||
|
||
<span class="n">executor</span> <span class="o">=</span> <span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
|
||
<span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">wait_on_future</span><span class="p">)</span>
|
||
</code></pre><dl class="class"><dt id="concurrent.futures.ThreadPoolExecutor"><span class="yiyi-st" id="yiyi-28"> <em class="property">class </em><code class="descclassname">concurrent.futures.</code><code class="descname">ThreadPoolExecutor</code><span class="sig-paren">(</span><em>max_workers=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-29">使用至多<em>max_workers</em>线程的池来异步执行调用的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>子类。</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-30"><span class="versionmodified">在版本3.5中更改:</span>如果<em>max_workers</em>是<code class="docutils literal"><span class="pre">None</span></code>或未指定,则默认为机器上的处理器数乘以<code class="docutils literal"><span class="pre">5</span></code>,假设<a class="reference internal" href="#concurrent.futures.ThreadPoolExecutor" title="concurrent.futures.ThreadPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ThreadPoolExecutor</span></code></a>通常用于重叠I / O而不是CPU工作,并且工作程序的数量应该高于<a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a></span></p></div></dd></dl><div class="section" id="threadpoolexecutor-example"><h3><span class="yiyi-st" id="yiyi-31">17.4.2.1. </span><span class="yiyi-st" id="yiyi-32">ThreadPoolExecutor示例</span></h3><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">concurrent.futures</span>
|
||
<span class="kn">import</span> <span class="nn">urllib.request</span>
|
||
|
||
<span class="n">URLS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'http://www.foxnews.com/'</span><span class="p">,</span>
|
||
<span class="s1">'http://www.cnn.com/'</span><span class="p">,</span>
|
||
<span class="s1">'http://europe.wsj.com/'</span><span class="p">,</span>
|
||
<span class="s1">'http://www.bbc.co.uk/'</span><span class="p">,</span>
|
||
<span class="s1">'http://some-made-up-domain.com/'</span><span class="p">]</span>
|
||
|
||
<span class="c1"># Retrieve a single page and report the URL and contents</span>
|
||
<span class="k">def</span> <span class="nf">load_url</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="p">):</span>
|
||
<span class="k">with</span> <span class="n">urllib</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="n">timeout</span><span class="p">)</span> <span class="k">as</span> <span class="n">conn</span><span class="p">:</span>
|
||
<span class="k">return</span> <span class="n">conn</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
|
||
|
||
<span class="c1"># We can use a with statement to ensure threads are cleaned up promptly</span>
|
||
<span class="k">with</span> <span class="n">concurrent</span><span class="o">.</span><span class="n">futures</span><span class="o">.</span><span class="n">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span> <span class="k">as</span> <span class="n">executor</span><span class="p">:</span>
|
||
<span class="c1"># Start the load operations and mark each future with its URL</span>
|
||
<span class="n">future_to_url</span> <span class="o">=</span> <span class="p">{</span><span class="n">executor</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="n">load_url</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="mi">60</span><span class="p">):</span> <span class="n">url</span> <span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">URLS</span><span class="p">}</span>
|
||
<span class="k">for</span> <span class="n">future</span> <span class="ow">in</span> <span class="n">concurrent</span><span class="o">.</span><span class="n">futures</span><span class="o">.</span><span class="n">as_completed</span><span class="p">(</span><span class="n">future_to_url</span><span class="p">):</span>
|
||
<span class="n">url</span> <span class="o">=</span> <span class="n">future_to_url</span><span class="p">[</span><span class="n">future</span><span class="p">]</span>
|
||
<span class="k">try</span><span class="p">:</span>
|
||
<span class="n">data</span> <span class="o">=</span> <span class="n">future</span><span class="o">.</span><span class="n">result</span><span class="p">()</span>
|
||
<span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">exc</span><span class="p">:</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="si">%r</span><span class="s1"> generated an exception: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">exc</span><span class="p">))</span>
|
||
<span class="k">else</span><span class="p">:</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="si">%r</span><span class="s1"> page is </span><span class="si">%d</span><span class="s1"> bytes'</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">)))</span>
|
||
</code></pre></div></div><div class="section" id="processpoolexecutor"><h2><span class="yiyi-st" id="yiyi-33">17.4.3. </span><span class="yiyi-st" id="yiyi-34">ProcessPoolExecutor </span></h2><p><span class="yiyi-st" id="yiyi-35"><a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>类是使用进程池异步执行调用的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>子类。</span><span class="yiyi-st" id="yiyi-36"><a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>使用<a class="reference internal" href="multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></code></a>模块,它允许它侧移<a class="reference internal" href="../glossary.html#term-global-interpreter-lock"><span class="xref std std-term">Global Interpreter Lock</span></a>,但也意味着只有可拾取对象可以被执行,回。</span></p><p><span class="yiyi-st" id="yiyi-37"><code class="docutils literal"><span class="pre">__main__</span></code>模块必须可以通过worker子进程进行导入。</span><span class="yiyi-st" id="yiyi-38">这意味着<a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>在交互式解释器中不起作用。</span></p><p><span class="yiyi-st" id="yiyi-39">从提交到<a class="reference internal" href="#concurrent.futures.ProcessPoolExecutor" title="concurrent.futures.ProcessPoolExecutor"><code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code></a>的可调用中调用<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>或<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>方法将导致死锁。</span></p><dl class="class"><dt id="concurrent.futures.ProcessPoolExecutor"><span class="yiyi-st" id="yiyi-40"><em class="property">class</em> <code class="descclassname">concurrent.futures。</code> <code class="descname">ProcessPoolExecutor tt> <span class="sig-paren">(</span> <em>max_workers = None</em> <span class="sig-paren">/ T5> </span></code></span></dt><dd><p><span class="yiyi-st" id="yiyi-41">使用最多<em>max_workers</em>进程的池异步执行调用的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>子类。</span><span class="yiyi-st" id="yiyi-42">如果<em>max_workers</em>是<code class="docutils literal"><span class="pre">None</span></code>或未给出,则它将默认为机器上的处理器数。</span><span class="yiyi-st" id="yiyi-43">如果<em>max_workers</em>低于或等于<code class="docutils literal"><span class="pre">0</span></code>,则会引发<a class="reference internal" href="exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal"><span class="pre">ValueError</span></code></a>。</span></p><div class="versionchanged"><p><span class="yiyi-st" id="yiyi-44"><span class="versionmodified">在版本3.3中更改:</span>当其中一个工作进程突然终止时,现在会出现<code class="xref py py-exc docutils literal"><span class="pre">BrokenProcessPool</span></code>错误。</span><span class="yiyi-st" id="yiyi-45">以前,行为是未定义的,但对执行者或其未来的操作通常会冻结或死锁。</span></p></div></dd></dl><div class="section" id="processpoolexecutor-example"><h3><span class="yiyi-st" id="yiyi-46">17.4.3.1. </span><span class="yiyi-st" id="yiyi-47">ProcessPoolExecutor示例</span></h3><pre><code class="language-python"><span></span><span class="kn">import</span> <span class="nn">concurrent.futures</span>
|
||
<span class="kn">import</span> <span class="nn">math</span>
|
||
|
||
<span class="n">PRIMES</span> <span class="o">=</span> <span class="p">[</span>
|
||
<span class="mi">112272535095293</span><span class="p">,</span>
|
||
<span class="mi">112582705942171</span><span class="p">,</span>
|
||
<span class="mi">112272535095293</span><span class="p">,</span>
|
||
<span class="mi">115280095190773</span><span class="p">,</span>
|
||
<span class="mi">115797848077099</span><span class="p">,</span>
|
||
<span class="mi">1099726899285419</span><span class="p">]</span>
|
||
|
||
<span class="k">def</span> <span class="nf">is_prime</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
|
||
<span class="k">if</span> <span class="n">n</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||
<span class="k">return</span> <span class="kc">False</span>
|
||
|
||
<span class="n">sqrt_n</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">math</span><span class="o">.</span><span class="n">floor</span><span class="p">(</span><span class="n">math</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">n</span><span class="p">)))</span>
|
||
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">sqrt_n</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">):</span>
|
||
<span class="k">if</span> <span class="n">n</span> <span class="o">%</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
|
||
<span class="k">return</span> <span class="kc">False</span>
|
||
<span class="k">return</span> <span class="kc">True</span>
|
||
|
||
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
|
||
<span class="k">with</span> <span class="n">concurrent</span><span class="o">.</span><span class="n">futures</span><span class="o">.</span><span class="n">ProcessPoolExecutor</span><span class="p">()</span> <span class="k">as</span> <span class="n">executor</span><span class="p">:</span>
|
||
<span class="k">for</span> <span class="n">number</span><span class="p">,</span> <span class="n">prime</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">PRIMES</span><span class="p">,</span> <span class="n">executor</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">is_prime</span><span class="p">,</span> <span class="n">PRIMES</span><span class="p">)):</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="s1">'</span><span class="si">%d</span><span class="s1"> is prime: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">number</span><span class="p">,</span> <span class="n">prime</span><span class="p">))</span>
|
||
|
||
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>
|
||
<span class="n">main</span><span class="p">()</span>
|
||
</code></pre></div></div><div class="section" id="future-objects"><h2><span class="yiyi-st" id="yiyi-48">17.4.4. </span><span class="yiyi-st" id="yiyi-49">未来的对象</span></h2><p><span class="yiyi-st" id="yiyi-50"><a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>类封装了可调用的异步执行。</span><span class="yiyi-st" id="yiyi-51"><a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>实例是由<a class="reference internal" href="#concurrent.futures.Executor.submit" title="concurrent.futures.Executor.submit"><code class="xref py py-meth docutils literal"><span class="pre">Executor.submit()</span></code></a>创建的。</span></p><dl class="class"><dt id="concurrent.futures.Future"><span class="yiyi-st" id="yiyi-52"> <em class="property">class </em><code class="descclassname">concurrent.futures.</code><code class="descname">Future</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-53">封装一个可调用的异步执行。</span><span class="yiyi-st" id="yiyi-54"><a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>实例由<a class="reference internal" href="#concurrent.futures.Executor.submit" title="concurrent.futures.Executor.submit"><code class="xref py py-meth docutils literal"><span class="pre">Executor.submit()</span></code></a>创建,不应直接创建,除非测试。</span></p><span class="yiyi-st" id="yiyi-56"> <blockquote> <div><dl class="method"> <dt id="concurrent.futures.Future.cancel"> <code class="descname">cancel</code><span class="sig-paren">(</span><span class="sig-paren">)</span></dt> <dd><p>Attempt to cancel the call. If the call is currently being executed and cannot be cancelled then the method will return <code class="docutils literal"><span class="pre">False</span></code>, otherwise the call will be cancelled and the method will return <code class="docutils literal"><span class="pre">True</span></code>.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.cancelled"> <code class="descname">cancelled</code><span class="sig-paren">(</span><span class="sig-paren">)</span></dt> <dd><p>Return <code class="docutils literal"><span class="pre">True</span></code> if the call was successfully cancelled.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.running"> <code class="descname">running</code><span class="sig-paren">(</span><span class="sig-paren">)</span></dt> <dd><p>Return <code class="docutils literal"><span class="pre">True</span></code> if the call is currently being executed and cannot be cancelled.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.done"> <code class="descname">done</code><span class="sig-paren">(</span><span class="sig-paren">)</span></dt> <dd><p>Return <code class="docutils literal"><span class="pre">True</span></code> if the call was successfully cancelled or finished running.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.result"> <code class="descname">result</code><span class="sig-paren">(</span><em>timeout=None</em><span class="sig-paren">)</span></dt> <dd><p>Return the value returned by the call. If the call hasn’t yet completed then this method will wait up to <em>timeout</em> seconds. If the call hasn’t completed in <em>timeout</em> seconds, then a <a class="reference internal" href="#concurrent.futures.TimeoutError" title="concurrent.futures.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">concurrent.futures.TimeoutError</span></code></a> will be raised. <em>timeout</em> can be an int or float. If <em>timeout</em> is not specified or <code class="docutils literal"><span class="pre">None</span></code>, there is no limit to the wait time.</p> <p>If the future is cancelled before completing then <a class="reference internal" href="#concurrent.futures.CancelledError" title="concurrent.futures.CancelledError"><code class="xref py py-exc docutils literal"><span class="pre">CancelledError</span></code></a> will be raised.</p> <p>If the call raised, this method will raise the same exception.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.exception"> <code class="descname">exception</code><span class="sig-paren">(</span><em>timeout=None</em><span class="sig-paren">)</span></dt> <dd><p>Return the exception raised by the call. If the call hasn’t yet completed then this method will wait up to <em>timeout</em> seconds. If the call hasn’t completed in <em>timeout</em> seconds, then a <a class="reference internal" href="#concurrent.futures.TimeoutError" title="concurrent.futures.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">concurrent.futures.TimeoutError</span></code></a> will be raised. <em>timeout</em> can be an int or float. If <em>timeout</em> is not specified or <code class="docutils literal"><span class="pre">None</span></code>, there is no limit to the wait time.</p> <p>If the future is cancelled before completing then <a class="reference internal" href="#concurrent.futures.CancelledError" title="concurrent.futures.CancelledError"><code class="xref py py-exc docutils literal"><span class="pre">CancelledError</span></code></a> will be raised.</p> <p>If the call completed without raising, <code class="docutils literal"><span class="pre">None</span></code> is returned.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.add_done_callback"> <code class="descname">add_done_callback</code><span class="sig-paren">(</span><em>fn</em><span class="sig-paren">)</span></dt> <dd><p>Attaches the callable <em>fn</em> to the future. <em>fn</em> will be called, with the future as its only argument, when the future is cancelled or finishes running.</p> <p>Added callables are called in the order that they were added and are always called in a thread belonging to the process that added them. If the callable raises an <a class="reference internal" href="exceptions.html#Exception" title="Exception"><code class="xref py py-exc docutils literal"><span class="pre">Exception</span></code></a> subclass, it will be logged and ignored. If the callable raises a <a class="reference internal" href="exceptions.html#BaseException" title="BaseException"><code class="xref py py-exc docutils literal"><span class="pre">BaseException</span></code></a> subclass, the behavior is undefined.</p> <p>If the future has already completed or been cancelled, <em>fn</em> will be called immediately.</p> </dd></dl> </div></blockquote></span><p><span class="yiyi-st" id="yiyi-55">以下<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>方法适用于单元测试和<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>实施。</span></p><span class="yiyi-st" id="yiyi-57"> <blockquote> <div><dl class="method"> <dt id="concurrent.futures.Future.set_running_or_notify_cancel"> <code class="descname">set_running_or_notify_cancel</code><span class="sig-paren">(</span><span class="sig-paren">)</span></dt> <dd><p>This method should only be called by <a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a> implementations before executing the work associated with the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> and by unit tests.</p> <p>If the method returns <code class="docutils literal"><span class="pre">False</span></code> then the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> was cancelled, i.e. <a class="reference internal" href="#concurrent.futures.Future.cancel" title="concurrent.futures.Future.cancel"><code class="xref py py-meth docutils literal"><span class="pre">Future.cancel()</span></code></a> was called and returned <cite>True</cite>. Any threads waiting on the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> completing (i.e. through <a class="reference internal" href="#concurrent.futures.as_completed" title="concurrent.futures.as_completed"><code class="xref py py-func docutils literal"><span class="pre">as_completed()</span></code></a> or <a class="reference internal" href="#concurrent.futures.wait" title="concurrent.futures.wait"><code class="xref py py-func docutils literal"><span class="pre">wait()</span></code></a>) will be woken up.</p> <p>If the method returns <code class="docutils literal"><span class="pre">True</span></code> then the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> was not cancelled and has been put in the running state, i.e. calls to <a class="reference internal" href="#concurrent.futures.Future.running" title="concurrent.futures.Future.running"><code class="xref py py-meth docutils literal"><span class="pre">Future.running()</span></code></a> will return <cite>True</cite>.</p> <p>This method can only be called once and cannot be called after <a class="reference internal" href="#concurrent.futures.Future.set_result" title="concurrent.futures.Future.set_result"><code class="xref py py-meth docutils literal"><span class="pre">Future.set_result()</span></code></a> or <a class="reference internal" href="#concurrent.futures.Future.set_exception" title="concurrent.futures.Future.set_exception"><code class="xref py py-meth docutils literal"><span class="pre">Future.set_exception()</span></code></a> have been called.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.set_result"> <code class="descname">set_result</code><span class="sig-paren">(</span><em>result</em><span class="sig-paren">)</span></dt> <dd><p>Sets the result of the work associated with the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> to <em>result</em>.</p> <p>This method should only be used by <a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a> implementations and unit tests.</p> </dd></dl> <dl class="method"> <dt id="concurrent.futures.Future.set_exception"> <code class="descname">set_exception</code><span class="sig-paren">(</span><em>exception</em><span class="sig-paren">)</span></dt> <dd><p>Sets the result of the work associated with the <a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a> to the <a class="reference internal" href="exceptions.html#Exception" title="Exception"><code class="xref py py-class docutils literal"><span class="pre">Exception</span></code></a> <em>exception</em>.</p> <p>This method should only be used by <a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a> implementations and unit tests.</p> </dd></dl> </div></blockquote></span></dd></dl></div><div class="section" id="module-functions"><h2><span class="yiyi-st" id="yiyi-58">17.4.5. </span><span class="yiyi-st" id="yiyi-59">模块函数</span></h2><dl class="function"><dt id="concurrent.futures.wait"><span class="yiyi-st" id="yiyi-60"> <code class="descclassname">concurrent.futures.</code><code class="descname">wait</code><span class="sig-paren">(</span><em>fs</em>, <em>timeout=None</em>, <em>return_when=ALL_COMPLETED</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-61">等待由<em>fs</em>给出的<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>实例(可能由不同的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>实例创建)完成。</span><span class="yiyi-st" id="yiyi-62">返回一个命名的2元组的集合。</span><span class="yiyi-st" id="yiyi-63">第一个集合,名为<code class="docutils literal"><span class="pre">done</span></code>,包含在等待完成之前完成(完成或被取消)的期货。</span><span class="yiyi-st" id="yiyi-64">第二个集合,名为<code class="docutils literal"><span class="pre">not_done</span></code>,包含未完成的期货。</span></p><p><span class="yiyi-st" id="yiyi-65"><em>timeout</em>可用于控制返回前等待的最大秒数。</span><span class="yiyi-st" id="yiyi-66"><em>timeout</em>可以是一个int或float。</span><span class="yiyi-st" id="yiyi-67">如果未指定<em>超时</em>或<code class="docutils literal"><span class="pre">None</span></code>,则等待时间没有限制。</span></p><p><span class="yiyi-st" id="yiyi-68"><em>return_when</em>指示此函数应返回的时间。</span><span class="yiyi-st" id="yiyi-69">它必须是以下常量之一:</span></p><table border="1" class="docutils"><thead valign="bottom"><tr class="row-odd"><th class="head"><span class="yiyi-st" id="yiyi-70">不变</span></th><th class="head"><span class="yiyi-st" id="yiyi-71">描述</span></th></tr></thead><tbody valign="top"><tr class="row-even"><td><span class="yiyi-st" id="yiyi-72"><code class="xref py py-const docutils literal"><span class="pre">FIRST_COMPLETED</span></code></span></td><td><span class="yiyi-st" id="yiyi-73">当任何未来完成或被取消时,该函数将返回。</span></td></tr><tr class="row-odd"><td><span class="yiyi-st" id="yiyi-74"><code class="xref py py-const docutils literal"><span class="pre">FIRST_EXCEPTION</span></code></span></td><td><span class="yiyi-st" id="yiyi-75">当任何未来通过提出异常完成时,函数将返回。</span><span class="yiyi-st" id="yiyi-76">如果没有未来引发异常,则它等效于<code class="xref py py-const docutils literal"><span class="pre">ALL_COMPLETED</span></code>。</span></td></tr><tr class="row-even"><td><span class="yiyi-st" id="yiyi-77"><code class="xref py py-const docutils literal"><span class="pre">ALL_COMPLETED</span></code></span></td><td><span class="yiyi-st" id="yiyi-78">当所有期货完成或被取消时,该函数将返回。</span></td></tr></tbody></table></dd></dl><dl class="function"><dt id="concurrent.futures.as_completed"><span class="yiyi-st" id="yiyi-79"> <code class="descclassname">concurrent.futures.</code><code class="descname">as_completed</code><span class="sig-paren">(</span><em>fs</em>, <em>timeout=None</em><span class="sig-paren">)</span></span></dt><dd><p><span class="yiyi-st" id="yiyi-80">在多个<a class="reference internal" href="#concurrent.futures.Future" title="concurrent.futures.Future"><code class="xref py py-class docutils literal"><span class="pre">Future</span></code></a>实例(可能由不同的<a class="reference internal" href="#concurrent.futures.Executor" title="concurrent.futures.Executor"><code class="xref py py-class docutils literal"><span class="pre">Executor</span></code></a>实例创建)上的迭代器(iterator)将会被返回,这些Future实例由<em>fs</em>完成(结束或者被停止)时产生(译者注:fs可以是隐含有一系列Future实例以及其他普通值的组合,返回的迭代器迭代的是那些Future实例,不包含普通的值)。</span><span class="yiyi-st" id="yiyi-81">由<em>fs</em>给出的任何重复的futures将只会返回一次。</span><span class="yiyi-st" id="yiyi-82">在调用<a class="reference internal" href="#concurrent.futures.as_completed" title="concurrent.futures.as_completed"><code class="xref py py-func docutils literal"><span class="pre">as_completed()</span></code></a>之前完成的任何futures将首先生成。</span><span class="yiyi-st" id="yiyi-83">返回的迭代器引发<a class="reference internal" href="#concurrent.futures.TimeoutError" title="concurrent.futures.TimeoutError"><code class="xref py py-exc docutils literal"><span class="pre">concurrent.futures.TimeoutError</span></code></a>如果<a class="reference internal" href="stdtypes.html#iterator.__next__" title="iterator.__next__"><code class="xref py py-meth docutils literal"><span class="pre">__next__()</span></code></a>被调用,并且结果在从原始的<em>超时</em>秒后不可用调用<a class="reference internal" href="#concurrent.futures.as_completed" title="concurrent.futures.as_completed"><code class="xref py py-func docutils literal"><span class="pre">as_completed()</span></code></a>。</span><span class="yiyi-st" id="yiyi-84"><em>timeout</em>可以是一个int或float。</span><span class="yiyi-st" id="yiyi-85">如果未指定<em>超时</em>或<code class="docutils literal"><span class="pre">None</span></code>,则等待时间没有限制。</span></p></dd></dl><div class="admonition seealso"><p class="first admonition-title"><span class="yiyi-st" id="yiyi-86">也可以看看</span></p><dl class="last docutils"><dt><span class="yiyi-st" id="yiyi-87"><span class="target" id="index-0"></span> <a class="pep reference external" href="https://www.python.org/dev/peps/pep-3148"><strong>PEP 3148</strong></a> - futures - 异步执行计算</span></dt><dd><span class="yiyi-st" id="yiyi-88">描述此功能以包含在Python标准库中的提议。</span></dd></dl></div></div><div class="section" id="exception-classes"><h2><span class="yiyi-st" id="yiyi-89">17.4.6. </span><span class="yiyi-st" id="yiyi-90">异常类</span></h2><dl class="exception"><dt id="concurrent.futures.CancelledError"><span class="yiyi-st" id="yiyi-91"> <em class="property">exception </em><code class="descclassname">concurrent.futures.</code><code class="descname">CancelledError</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-92">在未来取消时引发。</span></p></dd></dl><dl class="exception"><dt id="concurrent.futures.TimeoutError"><span class="yiyi-st" id="yiyi-93"> <em class="property">exception </em><code class="descclassname">concurrent.futures.</code><code class="descname">TimeoutError</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-94">在未来操作超过给定超时时触发。</span></p></dd></dl><dl class="exception"><dt id="concurrent.futures.process.BrokenProcessPool"><span class="yiyi-st" id="yiyi-95"> <em class="property">exception </em><code class="descclassname">concurrent.futures.process.</code><code class="descname">BrokenProcessPool</code></span></dt><dd><p><span class="yiyi-st" id="yiyi-96">派生自<a class="reference internal" href="exceptions.html#RuntimeError" title="RuntimeError"><code class="xref py py-exc docutils literal"><span class="pre">RuntimeError</span></code></a>,此异常类在<code class="xref py py-class docutils literal"><span class="pre">ProcessPoolExecutor</span></code>的工作程序之一以非干净方式终止时生成(例如,如果它从外部杀死)。</span></p><div class="versionadded"><p><span class="yiyi-st" id="yiyi-97"><span class="versionmodified">版本3.3中的新功能。</span></span></p></div></dd></dl></div></div></div> |