.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "11_demos\python_packages\parallelization_threading\demo_ThreadPoolExecutor.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py:

Thread pool executor
=======================

``Asyncio``/``concurrent`` heavily changed from python ``3.4`` to ``3.7``, better read the docs
and do some tutorials. Asyncio is preferred over plain concurrent module. 

.. GENERATED FROM PYTHON SOURCE LINES 6-18

.. code-block:: default


    import concurrent.futures
    import urllib.request

    URLS = [
        "http://www.foxnews.com/",
        "http://www.cnn.com/",
        "http://europe.wsj.com/",
        "http://www.bbc.co.uk/",
        "http://some-made-up-domain.com/",
    ]








.. GENERATED FROM PYTHON SOURCE LINES 20-21

Retrieve a single page and report the url and contents

.. GENERATED FROM PYTHON SOURCE LINES 21-26

.. code-block:: default

    def load_url(url, timeout):
        with urllib.request.urlopen(url, timeout=timeout) as conn:
            return conn.read()









.. GENERATED FROM PYTHON SOURCE LINES 27-28

We can use a with statement to ensure threads are cleaned up promptly

.. GENERATED FROM PYTHON SOURCE LINES 28-39

.. code-block:: default

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        # Start the load operations and mark each future with its URL
        future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
        for future in concurrent.futures.as_completed(future_to_url):
            url = future_to_url[future]
            try:
                data = future.result()
            except Exception as exc:
                print("%r generated an exception: %s" % (url, exc))
            else:
                print("%r page is %d bytes" % (url, len(data)))




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    'http://www.foxnews.com/' page is 311539 bytes
    'http://www.bbc.co.uk/' page is 449682 bytes
    'http://www.cnn.com/' page is 1143482 bytes
    'http://some-made-up-domain.com/' page is 1127 bytes
    'http://europe.wsj.com/' generated an exception: HTTP Error 403: Forbidden





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.870 seconds)


.. _sphx_glr_download_11_demos_python_packages_parallelization_threading_demo_ThreadPoolExecutor.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: demo_ThreadPoolExecutor.py <demo_ThreadPoolExecutor.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: demo_ThreadPoolExecutor.ipynb <demo_ThreadPoolExecutor.ipynb>`