:orphan:

Code samples
==================


.. raw:: html

  <div id='sg-tag-list' class='sphx-glr-tag-list'></div>


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial contains a basic demonstration how to set up LinearOperators for the Hessian and the GGN and how to multiply them to a vector.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_matrix_vector_products_thumb.png
    :alt:

  :doc:`/basic_usage/example_matrix_vector_products`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Matrix-vector products</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates how to work with Huggingface (HF) language models.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_huggingface_thumb.png
    :alt:

  :doc:`/basic_usage/example_huggingface`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Usage with Huggingface LLMs</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial explains how to create linear operators that correspond to a sub-matrix of another linear operator.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_submatrices_thumb.png
    :alt:

  :doc:`/basic_usage/example_submatrices`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Sub-matrices of linear operators</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates how to compute a subset of eigenvalues of a linear operator, using scipy.sparse.linalg.eigsh. Concretely, we will compute leading eigenvalues of the Hessian.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_eigenvalues_thumb.png
    :alt:

  :doc:`/basic_usage/example_eigenvalues`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Eigenvalues</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates how to work with inverses of linear operators.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_inverses_thumb.png
    :alt:

  :doc:`/basic_usage/example_inverses`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Inverses (natural gradient)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example we implement Fisher-weighted model averaging, a technique described in this NeurIPS 2022 paper. It requires Fisher-vector products, and multiplication with the inverse of a sum of Fisher matrices. The paper uses a diagonal approximation of the Fisher matrices. In addition, we will also use the exact Fisher matrices and rely on matrix-free methods for applying the inverse.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_model_merging_thumb.png
    :alt:

  :doc:`/basic_usage/example_model_merging`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Fisher-weighted Model Averaging</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial visualizes different curvature matrices for a model with sufficiently small parameter space.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_visual_tour_thumb.png
    :alt:

  :doc:`/basic_usage/example_visual_tour`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visual tour of curvature matrices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we will compare two approaches to compute the Fisher information matrix:">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_fisher_monte_carlo_thumb.gif
    :alt:

  :doc:`/basic_usage/example_fisher_monte_carlo`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Monte-Carlo approximation of the Fisher</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example we will use the spectral density estimation techniques of Papyan, 2020 to reproduce the plots for synthetic spectra shown in Figure 15 of the paper.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_verification_spectral_density_thumb.png
    :alt:

  :doc:`/basic_usage/example_verification_spectral_density`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Spectral density estimation (verification)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we demonstrate how to evaluate the run time and memory performance of linear operators. This allows to get a feeling for how expensive each operator is, compared to a gradient computation.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_benchmark_thumb.png
    :alt:

  :doc:`/basic_usage/example_benchmark`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Benchmarking linear operators</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example we will explore estimators for the trace and diagonal of a matrix. curvlinops implements different methods, and we will reproduce the results from their original papers, using toy matrices with a power law spectrum.">

.. only:: html

  .. image:: /basic_usage/images/thumb/sphx_glr_example_trace_diagonal_estimation_thumb.png
    :alt:

  :doc:`/basic_usage/example_trace_diagonal_estimation`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Trace and diagonal estimation</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /basic_usage/example_matrix_vector_products
   /basic_usage/example_huggingface
   /basic_usage/example_submatrices
   /basic_usage/example_eigenvalues
   /basic_usage/example_inverses
   /basic_usage/example_model_merging
   /basic_usage/example_visual_tour
   /basic_usage/example_fisher_monte_carlo
   /basic_usage/example_verification_spectral_density
   /basic_usage/example_benchmark
   /basic_usage/example_trace_diagonal_estimation


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: basic_usage_python.zip </basic_usage/basic_usage_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: basic_usage_jupyter.zip </basic_usage/basic_usage_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_