:orphan:

Tutorials
=========

Below is a gallery of tutorials for writing various basic operations with Triton. It is recommended that you read through the tutorials in order, starting with the simplest one.

To install the dependencies for the tutorials:

.. code-block:: bash

    cd triton
    pip install -e './python[tutorials]'



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, you will write a simple vector addition using Triton.">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_01-vector-add_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_01-vector-add.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Vector Addition</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, you will write a fused softmax operation that is significantly faster than Py...">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_02-fused-softmax_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_02-fused-softmax.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Fused Softmax</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="You will specifically learn about:">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_03-matrix-multiplication_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_03-matrix-multiplication.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Matrix Multiplication</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, you will write a memory-efficient implementation of dropout whose state will ...">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_04-low-memory-dropout_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_04-low-memory-dropout.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Low-Memory Dropout</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In doing so, you will learn about:">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_05-layer-norm_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_05-layer-norm.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Layer Normalization</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This is a Triton implementation of the Flash Attention v2 algorithm from Tri Dao (https://trida...">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_06-fused-attention_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_06-fused-attention.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Fused Attention</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Libdevice (`tl.extra.cuda.libdevice`) function">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_07-extern-functions_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_07-extern-functions.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Libdevice (`tl.extra.cuda.libdevice`) function</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Group GEMM">

.. only:: html

  .. image:: /getting-started/tutorials/images/thumb/sphx_glr_08-grouped-gemm_thumb.png
    :alt:

  :ref:`sphx_glr_getting-started_tutorials_08-grouped-gemm.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Group GEMM</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /getting-started/tutorials/01-vector-add
   /getting-started/tutorials/02-fused-softmax
   /getting-started/tutorials/03-matrix-multiplication
   /getting-started/tutorials/04-low-memory-dropout
   /getting-started/tutorials/05-layer-norm
   /getting-started/tutorials/06-fused-attention
   /getting-started/tutorials/07-extern-functions
   /getting-started/tutorials/08-grouped-gemm


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: tutorials_python.zip </getting-started/tutorials/tutorials_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: tutorials_jupyter.zip </getting-started/tutorials/tutorials_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_