diff -Nru pygments-2.10.0+dfsg/AUTHORS pygments-2.11.2+dfsg/AUTHORS --- pygments-2.10.0+dfsg/AUTHORS 2021-08-08 14:31:49.000000000 +0000 +++ pygments-2.11.2+dfsg/AUTHORS 2022-01-06 11:12:07.000000000 +0000 @@ -6,6 +6,7 @@ Other contributors, listed alphabetically, are: * Sam Aaron -- Ioke lexer +* Jean Abou Samra -- LilyPond lexer * João Abecasis -- JSLT lexer * Ali Afshar -- image formatter * Thomas Aglassinger -- Easytrieve, JCL, Rexx, Transact-SQL and VBScript @@ -117,6 +118,7 @@ * Gerwin Klein -- Isabelle lexer * Eric Knibbe -- Lasso lexer * Stepan Koltsov -- Clay lexer +* Oliver Kopp - Friendly grayscale style * Adam Koprowski -- Opa lexer * Benjamin Kowarsch -- Modula-2 lexer * Domen Kožar -- Nix lexer @@ -144,6 +146,7 @@ * Brian McKenna -- F# lexer * Charles McLaughlin -- Puppet lexer * Kurt McKee -- Tera Term macro lexer, PostgreSQL updates, MySQL overhaul +* Joe Eli McIlvain -- Savi lexer * Lukas Meuser -- BBCode formatter, Lua lexer * Cat Miller -- Pig lexer * Paul Miller -- LiveScript lexer @@ -219,6 +222,7 @@ * Brian Tiffin -- OpenCOBOL lexer * Bob Tolbert -- Hy lexer * Matthias Trute -- Forth lexer +* Tuoa Spi T4 -- Bdd lexer * Erick Tryzelaar -- Felix lexer * Alexander Udalov -- Kotlin lexer improvements * Thomas Van Doren -- Chapel lexer @@ -246,5 +250,8 @@ * Fabian Neumann -- CDDL lexer * Thomas Duboucher -- CDDL lexer * Philipp Imhof -- Pango Markup formatter +* Thomas Voss -- Sed lexer +* Martin Fischer -- WCAG contrast testing +* Marc Auberer -- Spice lexer Many thanks for all contributions! diff -Nru pygments-2.10.0+dfsg/CHANGES pygments-2.11.2+dfsg/CHANGES --- pygments-2.10.0+dfsg/CHANGES 2021-08-15 10:45:08.000000000 +0000 +++ pygments-2.11.2+dfsg/CHANGES 2022-01-06 11:12:07.000000000 +0000 @@ -1,10 +1,111 @@ Pygments changelog ================== -Since 2.5.0, issue numbers refer to the tracker at -, -pull request numbers to the requests at -. +Pull request numbers before 2.4.2 are not linked as they refer to the now defunct Bitbucket project. + +Version 2.12.0 +-------------- +(not released yet) + +Version 2.11.2 +-------------- +(released January 6th, 2022) + +- Updated lexers: + + * C-family: Fix incorrect handling of labels (#2022, #1996, #1182) + * Java: Fixed an issue with ``record`` keywords result in ``Error`` tokens in some cases (#2018) + +- Fix links to line numbers not working correctly (#2014) +- Remove ``underline`` from ``Whitespace`` style in the ``Tango`` theme (#2020) +- Fix ``IRC`` and ``Terminal256`` formatters not backtracking correctly for custom token types, resulting in some unstyled tokens (#1986) + +Version 2.11.1 +-------------- +(released December 31st, 2021) + +- Updated lexers: + + * C-family: Handle return types with multiple tokens (e.g. ``unsigned int``) (#2008) + * JSON: Fix a regression which caused whitespace before ``:`` to result in ``Error`` tokens (#2010) + * SPICE: Various improvements (#2009) + +Version 2.11.0 +-------------- +(released December 30th, 2021) + +- Added lexers: + + * BDD (#1803) + * Elpi (#1894) + * LilyPond (#1845, #1968, #1971, #2001). This comes with a custom style as well. + * Maxima (#1885) + * Rita (#1541, #2003) + * Savi (#1863) + * Sed (#1935) + * Sophia contracts (#1974) + * Spice (#1980) + * ``.SRCINFO`` (#1951) + +- Updated lexers: + + * ABNF: Allow one-character rules (#1804) + * Assembly: Fix incorrect token endings (#1895, #1961) + * Bibtex: Distinguish between ``comment`` and ``commentary`` (#1899, #1806) + * C family: Support unicode identifiers (#1848) + * CDDL: Fix slow lexing speed (#1959) + * Debian control: Add missing fields (#1946) + * Devicetree: Recognize hexadecimal addresses for nodes (#1949) + * GDScript: Add ``void`` data type (#1948) + * GSQL + + - Fix comment handling (#2002) + - Fix catastrophic backtracking (#2006) + + * HTML, XML: Improve comment handling (#1896) + * Java: Add ``yield`` (#1941) and sealed classes/record (#1902) + * Makefiles (#1860, #1898) + * objdump-nasm: Improve handling of ``--no-show-raw-insn`` dumps (#1981) + * Prolog: Support escaped ``\`` inside quoted strings (#1479) + * Python: + + - Support ``~`` in tracebacks (#2004) + - Support the pattern matching keywords (#1797, #1994) + + * RobotFramework: Improve empty brace handling (#1921, #1922) + * Terraform + + - Add the 'set' type (#1909) + - Support heredocs (#1909) + +- Added styles: + + * Dracula (#1796) + * Friendly Grayscale (#1040, #1273) + * LilyPond (#1845) -- to be used for the ``LilyPond`` language. + * One-Dark (#1924, #1979) + +.. note:: + + All of the new styles unfortunately do not conform to WCAG recommendations. + +- There is new infrastructure in place to improve style accessibility. The default style has been updated to conform to WCAG recommendations. All styles are now checked for sufficient contrast by default to prevent regressions. (#1919, #1937, #1938, #1940) +- Clean up unused imports (#1887) +- Fix multiple lexers producing repeated single-character tokens +- Fix multiple lexers marking whitespace as ``Text`` (#1237, #1905, #1908, #1914, #1911, #1923, #1939, #1957, #1978) +- Remove duplicated assignments in the Paraiso style (#1934) +- ``pygmentize`` supports JSON output for the various list functions now, making it easier to consume them from scripts. (#1437, #1890) +- Use the ``shell`` lexer for ``kshrc`` files (#1947) +- Use the ``ruby`` lexer for ``Vagrantfile`` files (#1936) +- Use the C lexer for ``.xbm`` and ``.xpm`` files (#1802) +- Add a ``groff`` formatter (#1873) +- Update documentation (#1928) +- Line anchors now link to themselves (#1973) +- Add official support for Python 3.10 (#1917) +- Fix several missing colors in dark styles: Gruvbox dark, Monokai, Rrt, Sas, Strata dark (#1955) +- Associate more file types with ``man`` pages +- The ``HtmlFormatter`` can now emit tooltips for each token to ease debugging of lexers (#1822) +- Add ``f90`` as an alias for ``fortran`` (#2000) Version 2.10.0 @@ -27,6 +128,7 @@ * Fennel (#1862) * JavaScript (#1741, #1814) * LLVM (#1824) + * Python (#1852) * Rust - Fix lexing of "break" and "continue" (#1843) @@ -35,6 +137,7 @@ * Scala: Add support for the ``\`` operator (#1857) * Swift (#1767, #1842) * Tcl: Allow ``,`` and ``@`` in strings (#1834, #1742) + * TOML (#1870, #1872) - Fix assert statements in TNT lexer. - Token types across all lexers have been unified (using the most common token @@ -70,6 +173,7 @@ * Chapel (#1743) * Coq (#1721) * Cython (#853) + * DeviceTree (#1755) * Groovy (#1765) * Julia (#1715) * Octave: Allow multiline and block-percent comments (#1726) @@ -89,6 +193,7 @@ - Improve line number colors (#1779, #1778) - Fix CTag related issue (#1724) - Recognize ``.leex`` as Elixir templates +- Fix incorrect variable being accessed (#1748) - Updated `filename` handling in HTML formatter if `linenos='table'` (#1757) @@ -139,6 +244,7 @@ - Update builtin functions (#1705) - Various cleanups (#1673) + * OpenEdge (#1696) * Python: Improve handling of raw f-strings (#1681, #1683) * Ruby: Better method name handling (#1531) * Stata: Updated keywords (#1470) @@ -213,9 +319,9 @@ - SQL - Fix keywords (#1668) - Typescript: Fix incorrect punctuation handling (#1510, #1511) -- Fix infinite loop in SML lexer (#1625) +- Fix infinite loop in SML lexer (#1625), `CVE-2021-20270 `_ - Fix backtracking string regexes in JavaScript/TypeScript, Modula2 - and many other lexers (#1637) + and many other lexers (#1637) `CVE-2021-27291 `_ - Limit recursion with nesting Ruby heredocs (#1638) - Fix a few inefficient regexes for guessing lexers - Fix the raw token lexer handling of Unicode (#1616) diff -Nru pygments-2.10.0+dfsg/Contributing.md pygments-2.11.2+dfsg/Contributing.md --- pygments-2.10.0+dfsg/Contributing.md 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/Contributing.md 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,67 @@ +Licensing +========= + +The code is distributed under the BSD 2-clause license. Contributors making pull +requests must agree that they are able and willing to put their contributions +under that license. + +Contribution checklist +====================== + +* Check the documentation for how to write + [a new lexer](https://pygments.org/docs/lexerdevelopment/), + [a new formatter](https://pygments.org/docs/formatterdevelopment/) or + [a new filter](https://pygments.org/docs/filterdevelopment/) + +* When writing rules, try to merge simple rules. For instance, combine: + + ```python + _PUNCTUATION = [ + (r"\(", token.Punctuation), + (r"\)", token.Punctuation), + (r"\[", token.Punctuation), + (r"\]", token.Punctuation), + ("{", token.Punctuation), + ("}", token.Punctuation), + ] + ``` + + into: + + ```python + (r"[\(\)\[\]{}]", token.Punctuation) + ``` + +* Be careful with ``.*``. This matches greedily as much as it can. For instance, + rule like ``@.*@`` will match the whole string ``@first@ second @third@``, + instead of matching ``@first@`` and ``@second@``. You can use ``@.*?@`` in + this case to stop early. The ``?`` tries to match _as few times_ as possible. + +* Don't add imports of your lexer anywhere in the codebase. (In case you're + curious about ``compiled.py`` -- this file exists for backwards compatibility + reasons.) + +* Use the standard importing convention: ``from token import Punctuation`` + +* For test cases that assert on the tokens produced by a lexer, use tools: + + * You can use the ``testcase`` formatter to produce a piece of code that + can be pasted into a unittest file: + ``python -m pygments -l lua -f testcase <<< "local a = 5"`` + + * Most snippets should instead be put as a sample file under + ``tests/snippets//*.txt``. These files are automatically + picked up as individual tests, asserting that the input produces the + expected tokens. + + To add a new test, create a file with just your code snippet under a + subdirectory based on your lexer's main alias. Then run + ``pytest --update-goldens `` to auto-populate the currently + expected tokens. Check that they look good and check in the file. + + Also run the same command whenever you need to update the test if the + actual produced tokens change (assuming the change is expected). + + * Large test files should go in ``tests/examplefiles``. This works + similar to ``snippets``, but the token output is stored in a separate + file. Output can also be regenerated with ``--update-goldens``. diff -Nru pygments-2.10.0+dfsg/.coveragerc pygments-2.11.2+dfsg/.coveragerc --- pygments-2.10.0+dfsg/.coveragerc 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/.coveragerc 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,3 @@ +[run] +include = + pygments/* diff -Nru pygments-2.10.0+dfsg/debian/changelog pygments-2.11.2+dfsg/debian/changelog --- pygments-2.10.0+dfsg/debian/changelog 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/changelog 2022-02-15 11:37:08.000000000 +0000 @@ -1,3 +1,29 @@ +pygments (2.11.2+dfsg-2) unstable; urgency=medium + + * Team upload + * autopkgtest: Adjust B-D for autopkgtest + + -- Carsten Schoenert Tue, 15 Feb 2022 12:37:08 +0100 + +pygments (2.11.2+dfsg-1) unstable; urgency=medium + + * Team upload + * d/gbp.conf: Add content to filter out + * New upstream version 2.11.2+dfsg + * d/control: Extend packages for Build-Depends + - The new upstream version requires now python3-lxml and + python3-wcag-contrast-ratio. + * d/*control: Running wrap-and-sort -ast + - No functional modifications. + * Rebuild patch queue from patch-queue branch + - Added patch: + 0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch + * d/copyright: Update copyright year + * d/control: Remove obsolete Conflicts for python-pygments + * d/python-pygments-doc.doc-base: Add new sequencer file + + -- Carsten Schoenert Mon, 14 Feb 2022 18:50:39 +0100 + pygments (2.10.0+dfsg-1) unstable; urgency=medium * New upstream release; Closes: #984625 diff -Nru pygments-2.10.0+dfsg/debian/control pygments-2.11.2+dfsg/debian/control --- pygments-2.10.0+dfsg/debian/control 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/control 2022-02-14 17:43:34.000000000 +0000 @@ -2,15 +2,20 @@ Section: python Priority: optional Maintainer: Piotr Ożarowski -Uploaders: Debian Python Team -Build-Depends: debhelper-compat (= 13) -Build-Depends-Indep: dh-python, - python3-all, - python3-docutils, - python3-jinja2, - python3-pytest , - python3-setuptools, - python3-sphinx, +Uploaders: + Debian Python Team , +Build-Depends: + debhelper-compat (= 13), +Build-Depends-Indep: + dh-python, + python3-all, + python3-docutils, + python3-jinja2, + python3-lxml, + python3-pytest , + python3-setuptools, + python3-sphinx, + python3-wcag-contrast-ratio, Standards-Version: 4.6.0.1 Rules-Requires-Root: no Homepage: http://pygments.org/ @@ -20,13 +25,17 @@ Package: python3-pygments Architecture: all Multi-Arch: foreign -Depends: python3-pkg-resources, - ${misc:Depends}, - ${python3:Depends}, -Suggests: python-pygments-doc, - ttf-bitstream-vera, -Breaks: python-pygments (<< 2.3.1+dfsg-4~) -Replaces: python-pygments (<< 2.3.1+dfsg-4~) +Depends: + python3-pkg-resources, + ${misc:Depends}, + ${python3:Depends}, +Suggests: + python-pygments-doc, + ttf-bitstream-vera, +Breaks: + python-pygments (<< 2.3.1+dfsg-4~), +Replaces: + python-pygments (<< 2.3.1+dfsg-4~), Description: syntax highlighting package written in Python 3 Pygments aims to be a generic syntax highlighter for general use in all kinds of software such as forum systems, wikis or other applications that need to @@ -43,10 +52,11 @@ Section: doc Architecture: all Multi-Arch: foreign -Depends: ${misc:Depends}, - ${sphinxdoc:Depends}, -Recommends: python3-pygments, -Conflicts: python-pygments (<= 2.0~), +Depends: + ${misc:Depends}, + ${sphinxdoc:Depends}, +Recommends: + python3-pygments, Description: documentation for the Pygments Pygments is syntax highlighting package. . diff -Nru pygments-2.10.0+dfsg/debian/copyright pygments-2.11.2+dfsg/debian/copyright --- pygments-2.10.0+dfsg/debian/copyright 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/copyright 2022-02-14 17:33:00.000000000 +0000 @@ -11,7 +11,7 @@ Furthermore, the pre-built documentation and egg info has been removed. Files: * -Copyright: 2006-2021 Pygments team, see AUTHORS +Copyright: 2006-2022 Pygments team, see AUTHORS License: BSD-2-clause Files: scripts/vim2pygments.py diff -Nru pygments-2.10.0+dfsg/debian/gbp.conf pygments-2.11.2+dfsg/debian/gbp.conf --- pygments-2.10.0+dfsg/debian/gbp.conf 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/gbp.conf 2022-02-14 16:32:24.000000000 +0000 @@ -1,3 +1,13 @@ [DEFAULT] debian-branch = debian/master upstream-branch = upstream/latest +pristine-tar = true + +[import-orig] +filter = [ + 'Pygments.egg-info', + 'doc/_build', + 'tests/examplefiles', + 'tests/test_examplefiles.py', + ] +filter-pristine-tar = True diff -Nru pygments-2.10.0+dfsg/debian/patches/0001-docs-moved-to-python-pygments-doc-binary-package.patch pygments-2.11.2+dfsg/debian/patches/0001-docs-moved-to-python-pygments-doc-binary-package.patch --- pygments-2.10.0+dfsg/debian/patches/0001-docs-moved-to-python-pygments-doc-binary-package.patch 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/patches/0001-docs-moved-to-python-pygments-doc-binary-package.patch 2022-02-14 17:36:34.000000000 +0000 @@ -0,0 +1,22 @@ +From: =?utf-8?q?Piotr_O=C5=BCarowski?= +Date: Mon, 31 Dec 2018 12:36:56 +0100 +Subject: docs: moved to python-pygments-doc binary package + +Forwarded: not-needed +--- + doc/pygmentize.1 | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/doc/pygmentize.1 b/doc/pygmentize.1 +index 5ac8fe6..85d44f8 100644 +--- a/doc/pygmentize.1 ++++ b/doc/pygmentize.1 +@@ -104,7 +104,7 @@ Show help screen. + .B \-V + Show version of the Pygments package. + .SH SEE ALSO +-/usr/share/doc/python-pygments/index.html ++/usr/share/doc/python-pygments-doc/index.html + .SH AUTHOR + pygmentize was written by Georg Brandl . + .PP diff -Nru pygments-2.10.0+dfsg/debian/patches/0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch pygments-2.11.2+dfsg/debian/patches/0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch --- pygments-2.10.0+dfsg/debian/patches/0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/patches/0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch 2022-02-14 17:36:34.000000000 +0000 @@ -0,0 +1,19 @@ +From: Carsten Schoenert +Date: Mon, 14 Feb 2022 17:51:56 +0100 +Subject: pygments.bashcomp: Remove hashbang within bash-completion + +This completion script isn't needing any hashbang as it's not intended +to get executed directly. +--- + external/pygments.bashcomp | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/external/pygments.bashcomp b/external/pygments.bashcomp +index 1299fdb..2f802d6 100644 +--- a/external/pygments.bashcomp ++++ b/external/pygments.bashcomp +@@ -1,4 +1,3 @@ +-#!bash + # + # Bash completion support for Pygments (the 'pygmentize' command). + # diff -Nru pygments-2.10.0+dfsg/debian/patches/0003-docs-moved-to-python-pygments-doc-binary-package.patch pygments-2.11.2+dfsg/debian/patches/0003-docs-moved-to-python-pygments-doc-binary-package.patch --- pygments-2.10.0+dfsg/debian/patches/0003-docs-moved-to-python-pygments-doc-binary-package.patch 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/patches/0003-docs-moved-to-python-pygments-doc-binary-package.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,19 +0,0 @@ -From: =?utf-8?q?Piotr_O=C5=BCarowski?= -Date: Mon, 31 Dec 2018 12:36:56 +0100 -Subject: docs moved to python-pygments-doc binary package - ---- - doc/pygmentize.1 | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - ---- a/doc/pygmentize.1 -+++ b/doc/pygmentize.1 -@@ -104,7 +104,7 @@ Show help screen. - .B \-V - Show version of the Pygments package. - .SH SEE ALSO --/usr/share/doc/python-pygments/index.html -+/usr/share/doc/python-pygments-doc/index.html - .SH AUTHOR - pygmentize was written by Georg Brandl . - .PP diff -Nru pygments-2.10.0+dfsg/debian/patches/series pygments-2.11.2+dfsg/debian/patches/series --- pygments-2.10.0+dfsg/debian/patches/series 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/patches/series 2022-02-14 17:36:34.000000000 +0000 @@ -1 +1,2 @@ -0003-docs-moved-to-python-pygments-doc-binary-package.patch +0001-docs-moved-to-python-pygments-doc-binary-package.patch +0002-pygments.bashcomp-Remove-hashbang-within-bash-comple.patch diff -Nru pygments-2.10.0+dfsg/debian/python-pygments-doc.doc-base pygments-2.11.2+dfsg/debian/python-pygments-doc.doc-base --- pygments-2.10.0+dfsg/debian/python-pygments-doc.doc-base 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/python-pygments-doc.doc-base 2022-02-14 17:02:09.000000000 +0000 @@ -0,0 +1,10 @@ +Document: python-pygments-doc +Title: Pygments Documentation +Abstract: This is the documentation of Pygments, a generic syntax highlighter + suitable for use in code hosting, forums, wikis or other applications that need + to prettify source code. +Section: Programming/Python + +Format: HTML +Index: /usr/share/doc/python-pygments-doc/html/index.html +Files: /usr/share/doc/python-pygments-doc/html/*.html diff -Nru pygments-2.10.0+dfsg/debian/tests/control pygments-2.11.2+dfsg/debian/tests/control --- pygments-2.10.0+dfsg/debian/tests/control 2021-12-10 05:46:37.000000000 +0000 +++ pygments-2.11.2+dfsg/debian/tests/control 2022-02-15 08:56:26.000000000 +0000 @@ -1,2 +1,7 @@ Tests: python3-pygments -Depends: python3-all, python3-pygments, python3-pytest +Depends: + python3-all, + python3-lxml, + python3-pygments, + python3-pytest, + python3-wcag-contrast-ratio, diff -Nru pygments-2.10.0+dfsg/description.rst pygments-2.11.2+dfsg/description.rst --- pygments-2.10.0+dfsg/description.rst 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/description.rst 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,18 @@ +Pygments +~~~~~~~~ + +Pygments is a syntax highlighting package written in Python. + +It is a generic syntax highlighter suitable for use in code hosting, forums, +wikis or other applications that need to prettify source code. Highlights +are: + +* a wide range of over 500 languages and other text formats is supported +* special attention is paid to details, increasing quality by a fair amount +* support for new languages and formats are added easily +* a number of output formats, presently HTML, LaTeX, RTF, SVG, all image + formats that PIL supports and ANSI sequences +* it is usable as a command-line tool and as a library + +Copyright 2006-2021 by the Pygments team, see ``AUTHORS``. +Licensed under the BSD, see ``LICENSE`` for details. \ No newline at end of file diff -Nru pygments-2.10.0+dfsg/doc/conf.py pygments-2.11.2+dfsg/doc/conf.py --- pygments-2.10.0+dfsg/doc/conf.py 2021-02-14 13:34:58.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/conf.py 2022-01-06 11:12:07.000000000 +0000 @@ -2,7 +2,7 @@ # Pygments documentation build configuration file # -import sys, os +import re, sys, os, itertools # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -10,6 +10,10 @@ sys.path.insert(0, os.path.abspath('..')) import pygments +import pygments.formatters +import pygments.lexers +import pygments.styles +import tests.contrast.test_contrasts as test_contrasts # -- General configuration ----------------------------------------------------- @@ -128,10 +132,10 @@ # Additional templates that should be rendered to pages, maps page names to # template names. +html_additional_pages = {'styles': 'styles.html'} + if os.environ.get('WEBSITE_BUILD'): - html_additional_pages = { - 'demo': 'demo.html', - } + html_additional_pages['demo'] = 'demo.html' # If false, no module index is generated. #html_domain_indices = True @@ -220,10 +224,64 @@ # Example configuration for intersphinx: refer to the Python standard library. #intersphinx_mapping = {'http://docs.python.org/': None} +rst_prolog = '.. |language_count| replace:: {}'.format(len(list(pygments.lexers.get_all_lexers()))) def pg_context(app, pagename, templatename, ctx, event_arg): ctx['demo_active'] = bool(os.environ.get('WEBSITE_BUILD')) + if pagename == 'demo': + ctx['lexers'] = sorted(pygments.lexers.get_all_lexers(), key=lambda x: x[0].lower()) + + if pagename in ('styles', 'demo'): + with open('examples/example.py') as f: + html = f.read() + lexer = pygments.lexers.get_lexer_for_filename('example.py') + min_contrasts = test_contrasts.min_contrasts() + ctx['styles_aa'] = [] + ctx['styles_sub_aa'] = [] + for style in pygments.styles.get_all_styles(): + if not pygments.styles.get_style_by_name(style).web_style_gallery_exclude: + aa = min_contrasts[style] >= test_contrasts.WCAG_AA_CONTRAST + bg_r, bg_g, bg_b = test_contrasts.hex2rgb(pygments.styles.get_style_by_name(style).background_color) + ctx['styles_aa' if aa else 'styles_sub_aa'].append( + dict( + name=style, + html=pygments.highlight( + html, + lexer, + pygments.formatters.HtmlFormatter(noclasses=True, style=style), + ), + # from https://en.wikipedia.org/wiki/Relative_luminance + bg_luminance=(0.2126*bg_r + 0.7152*bg_g + 0.0722*bg_b) + ) + ) + + # sort styles according to their background luminance (light styles first) + # if styles have the same background luminance sort them by their name + sortkey = lambda s: (-s['bg_luminance'], s['name']) + # the default style is always displayed first + default_style = ctx['styles_aa'].pop(0) + ctx['styles_aa'].sort(key=sortkey) + ctx['styles_aa'].insert(0, default_style) + ctx['styles_sub_aa'].sort(key=sortkey) + + +def source_read(app, docname, source): + # linkify issue / PR numbers in changelog + if docname == 'docs/changelog': + with open('../CHANGES') as f: + changelog = f.read() + + idx = changelog.find('\nVersion 2.4.2\n') + + def linkify(match): + url = 'https://github.com/pygments/pygments/issues/' + match[1] + return '`{} <{}>`_'.format(match[0], url) + + linkified = re.sub(r'(?:PR)?#([0-9]+)\b', linkify, changelog[:idx]) + source[0] = linkified + changelog[idx:] + def setup(app): app.connect('html-page-context', pg_context) + app.connect('source-read', source_read) diff -Nru pygments-2.10.0+dfsg/doc/docs/api.rst pygments-2.11.2+dfsg/doc/docs/api.rst --- pygments-2.10.0+dfsg/doc/docs/api.rst 2020-09-19 06:28:25.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/api.rst 2022-01-06 11:12:07.000000000 +0000 @@ -319,8 +319,11 @@ Option processing ================= -The :mod:`pygments.util` module has some utility functions usable for option -processing: +The :mod:`pygments.util` module has some utility functions usable for processing +command line options. All of the following functions get values from a +dictionary of options. If the value is already in the type expected by the +option, it is returned as-is. Otherwise, if the value is a string, it is first +converted to the expected type if possible. .. exception:: OptionError @@ -329,8 +332,11 @@ .. function:: get_bool_opt(options, optname, default=None) - Interpret the key `optname` from the dictionary `options` as a boolean and - return it. Return `default` if `optname` is not in `options`. + Intuitively, this is `options.get(optname, default)`, but restricted to + Boolean value. The Booleans can be represented as string, in order to accept + Boolean value from the command line arguments. If the key `optname` is + present in the dictionnary `options` and is not associated with a Boolean, + raise an `OptionError`. If it is absent, `default` is returned instead. The valid string values for ``True`` are ``1``, ``yes``, ``true`` and ``on``, the ones for ``False`` are ``0``, ``no``, ``false`` and ``off`` diff -Nru pygments-2.10.0+dfsg/doc/docs/cmdline.rst pygments-2.11.2+dfsg/doc/docs/cmdline.rst --- pygments-2.10.0+dfsg/doc/docs/cmdline.rst 2021-05-03 15:06:38.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/cmdline.rst 2022-01-06 11:12:07.000000000 +0000 @@ -99,6 +99,37 @@ will print out ``python``. It won't highlight anything yet. If no specific lexer is known for that filename, ``text`` is printed. +Additionally, there is the ``-C`` option, which is just like like ``-N``, except +that it prints out a lexer name based solely on a given content from standard +input. + + +Guessing the lexer from the file contents +----------------------------------------- + +The ``-g`` option will try to guess the correct lexer from the file contents, +or pass through as plain text if nothing can be guessed. This option also looks +for Vim modelines in the text, and for *some* languages, shebangs. Usage is as +follows:: + + $ pygmentize -g setup.py + +Note though, that this option is not very relaiable, and probably should be +used only if Pygments is not able to guess the correct lexer from the file's +extension. + + +Highlighting stdin until EOF +---------------------------- + +The ``-s`` option processes lines one at a time until EOF, rather than waiting +to process the entire file. This only works for stdin, only for lexers with no +line-spanning constructs, and is intended for streaming input such as you get +from `tail -f`. Usage is as follows:: + + $ tail -f sql.log | pygmentize -s -l sql + + Custom Lexers and Formatters ---------------------------- @@ -117,6 +148,7 @@ For more information, see :doc:`the Pygments documentation on Lexer development `. + Getting help ------------ @@ -128,6 +160,14 @@ will list only all installed filters. +.. versionadded:: 2.11 + +The ``--json`` option can be used in conjunction with the ``-L`` option to +output it's contents as JSON. Thus, to print all the installed styles and their +description in JSON, use the command:: + + $ pygmentize -L styles --json + The ``-H`` option will give you detailed information (the same that can be found in this documentation) about a lexer, formatter or filter. Usage is as follows:: diff -Nru pygments-2.10.0+dfsg/doc/docs/filterdevelopment.rst pygments-2.11.2+dfsg/doc/docs/filterdevelopment.rst --- pygments-2.10.0+dfsg/doc/docs/filterdevelopment.rst 2020-09-19 06:28:25.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/filterdevelopment.rst 2022-01-06 11:12:07.000000000 +0000 @@ -66,6 +66,10 @@ ttype = Name yield ttype, value -The decorator automatically subclasses an internal filter class and uses the -decorated function as a method for filtering. (That's why there is a `self` -argument that you probably won't end up using in the method.) + +You can instantiate this filter by calling `uncolor(classtoo=True)`, the same +way that you would have instantiated the previous filter by calling +`UncolorFilter(classtoo=True)`. Indeed, The decorator automatically ensures that +`uncolor` is a class which subclasses an internal filter class. The class +`uncolo` uses the decorated function as a method for filtering. (That's why +there is a `self` argument that you probably won't end up using in the method.) diff -Nru pygments-2.10.0+dfsg/doc/docs/filters.rst pygments-2.11.2+dfsg/doc/docs/filters.rst --- pygments-2.10.0+dfsg/doc/docs/filters.rst 2020-09-19 06:28:25.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/filters.rst 2022-01-06 11:12:07.000000000 +0000 @@ -6,11 +6,18 @@ .. versionadded:: 0.7 -You can filter token streams coming from lexers to improve or annotate the -output. For example, you can highlight special words in comments, convert -keywords to upper or lowercase to enforce a style guide etc. +Transforming a stream of tokens into another stream is called "filtering" and is +done by filters. The most common example of filters transform each token by +applying a simple rules such as highlighting the token if it is a TODO or +another special word, or converting keywords to uppercase to enforce a style +guide. More complex filters can transform the stream of tokens, such as removing +the line identation or merging tokens together. It should be noted that pygments +filters are entirely unrelated to Python's `filter +`_. -To apply a filter, you can use the `add_filter()` method of a lexer: +An arbitrary number of filters can be applied to token streams coming from +lexers to improve or annotate the output. To apply a filter, you can use the +`add_filter()` method of a lexer: .. sourcecode:: pycon diff -Nru pygments-2.10.0+dfsg/doc/docs/lexerdevelopment.rst pygments-2.11.2+dfsg/doc/docs/lexerdevelopment.rst --- pygments-2.10.0+dfsg/doc/docs/lexerdevelopment.rst 2021-02-14 13:34:58.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/lexerdevelopment.rst 2022-01-06 11:12:07.000000000 +0000 @@ -44,7 +44,8 @@ token with the match text and type `tokentype` and push `new_state` on the state stack. If the new state is ``'#pop'``, the topmost state is popped from the stack instead. To pop more than one state, use ``'#pop:2'`` and so on. -``'#push'`` is a synonym for pushing the current state on the stack. +``'#push'`` is a synonym for pushing a second time the current state on top of +the stack. The following example shows the `DiffLexer` from the builtin lexers. Note that it contains some additional attributes `name`, `aliases` and `filenames` which @@ -113,9 +114,6 @@ $ python -m pygments -x -l your_lexer.py:SomeLexer -Use the ``-f`` flag to select a different output format than terminal -escape sequences. - Or, using the Python API: .. code-block:: python @@ -132,6 +130,17 @@ If you only want to use your lexer with the Pygments API, you can import and instantiate the lexer yourself, then pass it to :func:`pygments.highlight`. +Use the ``-f`` flag to select a different output format than terminal +escape sequences. The :class:`pygments.formatters.html.HtmlFormatter` helps +you with debugging your lexer. You can use the ``debug_token_types`` option +to display the token types assigned to each part of your input file: + +.. code-block:: console + + $ python -m pygments -x -f html -Ofull,debug_token_types -l your_lexer.py:SomeLexer + +Hover over each token to see the token type displayed as a tooltip. + To prepare your new lexer for inclusion in the Pygments distribution, so that it will be found when passing filenames or lexer aliases from the command line, you have to perform the following steps. diff -Nru pygments-2.10.0+dfsg/doc/docs/styles.rst pygments-2.11.2+dfsg/doc/docs/styles.rst --- pygments-2.10.0+dfsg/doc/docs/styles.rst 2021-05-03 15:06:38.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/docs/styles.rst 2022-01-06 11:12:07.000000000 +0000 @@ -4,8 +4,8 @@ Styles ====== -Pygments comes with some builtin styles that work for both the HTML and -LaTeX formatter. +Pygments comes with :doc:`some builtin styles ` that work for both the +HTML and LaTeX formatter. The builtin styles can be looked up with the `get_style_by_name` function: diff -Nru pygments-2.10.0+dfsg/doc/examples/example.py pygments-2.11.2+dfsg/doc/examples/example.py --- pygments-2.10.0+dfsg/doc/examples/example.py 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/examples/example.py 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,14 @@ +from typing import Iterator + +# This is an example +class Math: + @staticmethod + def fib(n: int) -> Iterator[int]: + """ Fibonacci series up to n """ + a, b = 0, 1 + while a < n: + yield a + a, b = b, a + b + +result = sum(Math.fib(42)) +print("The answer is {}".format(result)) diff -Nru pygments-2.10.0+dfsg/doc/index.rst pygments-2.11.2+dfsg/doc/index.rst --- pygments-2.10.0+dfsg/doc/index.rst 2021-01-12 06:49:17.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/index.rst 2022-01-06 11:12:07.000000000 +0000 @@ -5,7 +5,7 @@ use in code hosting, forums, wikis or other applications that need to prettify source code. Highlights are: -* a wide range of over 500 languages and other text formats is supported +* a wide range of |language_count| languages and other text formats is supported * special attention is paid to details that increase highlighting quality * support for new languages and formats are added easily; most languages use a simple regex-based lexing mechanism diff -Nru pygments-2.10.0+dfsg/doc/languages.rst pygments-2.11.2+dfsg/doc/languages.rst --- pygments-2.10.0+dfsg/doc/languages.rst 2021-08-08 12:54:08.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/languages.rst 2022-01-06 11:12:07.000000000 +0000 @@ -56,6 +56,7 @@ * `Dylan `_ (incl. console) * `Eiffel `_ * `Elm `_ +* `Elpi `_ * `Emacs Lisp `_ * Email * `Erlang `_ (incl. shell sessions) @@ -153,6 +154,7 @@ * `Rust `_ * S, S-Plus, `R `_ * `Scala `_ +* `Savi `_ * `Scdoc `_ * `Scheme `_ * `Scilab `_ @@ -167,6 +169,7 @@ * `Snowball `_ * `Solidity `_ * `SourcePawn `_ +* `Spice `_ * `Stan `_ * `Standard ML `_ * `Stata `_ @@ -220,6 +223,7 @@ * Apache config files * Apache Pig * BBCode +* Bdd * CapDL * `Cap'n Proto `_ * `CDDL `_ @@ -250,6 +254,7 @@ * JSON, JSON-LD * Lean theorem prover * Lighttpd config files +* `LilyPond `_ * Linux kernel log (dmesg) * LLVM assembly * LSL scripts @@ -273,6 +278,7 @@ * Ragel * Redcode * ReST +* `Rita `_ * `Roboconf `_ * Robot Framework * RPM spec files @@ -282,6 +288,7 @@ * Sieve * Singularity * `Smithy `_ +* `Sophia `_ * SPARQL * SQL, also MySQL, SQLite * Squid configuration diff -Nru pygments-2.10.0+dfsg/doc/_static/demo.css pygments-2.11.2+dfsg/doc/_static/demo.css --- pygments-2.10.0+dfsg/doc/_static/demo.css 2021-01-12 06:49:17.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/_static/demo.css 2022-01-06 11:12:07.000000000 +0000 @@ -3,6 +3,7 @@ border-radius: 0; border: 1px solid #ccc; margin-top: 15px; + margin-bottom: 10px; padding: 10px 15px 5px 10px; position: relative; } @@ -19,29 +20,69 @@ } #hlcode { - padding: 10px 0; + margin: 10px 0; + max-height: 500px; + overflow: auto; + border: 1px solid #ccc; +} + +#hlcode:empty { + display: none; } #hlcode pre { background-color: transparent; - border-radius: 0; + border: 0; + margin: 0; +} +#hlcode table { + /* unset negative margin from pygments14.css */ + margin: unset; } -#hlcodedl { - text-align: right; +#code-header:not([hidden]) { + display: flex; + gap: 1em; padding: 0 15px; } +.flex-grow-1 { + flex-grow: 1; +} +#lexer { + margin-right: 0.5em; +} +#guessed-lexer:not(:empty):before { + content: '(guessed '; +} +#guessed-lexer:not(:empty):after { + content: ')'; +} + +#loading[hidden] { + visibility: hidden; + display: flex; +} #loading { - position: absolute; - top: 0; - left: 0; - width: 100%; - height: 100%; - margin: auto auto; - background-color: #cccccccc; display: flex; - flex-direction: column; - justify-content: center; - text-align: center; + align-items: center; + gap: 1em; +} + +#format-settings { + display: flex; + gap: 1em; + border-top: 1px solid #ccc; + padding-top: 0.5em; + margin-top: 0.5em; +} + +.tokens code { + /* make whitespace visible */ + white-space: pre; + background: #d9d9d9; +} + +#contrast-warning { + color: darkred; } diff -Nru pygments-2.10.0+dfsg/doc/_static/demo.js pygments-2.11.2+dfsg/doc/_static/demo.js --- pygments-2.10.0+dfsg/doc/_static/demo.js 2021-01-12 06:49:17.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/_static/demo.js 2022-01-06 11:12:07.000000000 +0000 @@ -1,126 +1,186 @@ -languagePluginLoader.then(() => { - // pyodide is now ready to use... - pyodide.loadPackage('Pygments').then(() => { - pyodide.runPython('import pygments.lexers, pygments.formatters.html, pygments.styles'); - - let qvars = getQueryVariables(); - - var lexerlist = pyodide.runPython('list(pygments.lexers.get_all_lexers())'); - var sel = document.getElementById("lang"); - for (lex of lexerlist) { - if (lex[1][0] === undefined) { - continue; - } - var opt = document.createElement("option"); - opt.text = lex[0]; - opt.value = lex[1][0]; - sel.add(opt); - if (lex[1].indexOf(qvars['lexer']) >= 0) { - opt.selected = true; - } - } +const loadingDiv = document.getElementById("loading"); +const langSelect = document.getElementById("lang"); +const styleSelect = document.getElementById("style"); +const formatterSelect = document.getElementById("formatter"); +const outputDiv = document.getElementById("hlcode"); +const codeHeader = document.getElementById("code-header"); +const copyLink = document.getElementById("copylink"); +const style = document.getElementById("css-style"); +const textarea = document.getElementById("code"); +const uriTooLongMsg = document.getElementById('uri-too-long'); +const contrastWarning = document.getElementById('contrast-warning'); +const fileInput = document.getElementById("file"); +const fileInputResetButton = document.getElementById('reset-file'); + +const qvars = Object.fromEntries(new URLSearchParams(window.location.search)); +if (qvars.lexer) { + langSelect.value = qvars.lexer; +} +if (qvars.code !== undefined) { + textarea.value = qvars.code; + loadingDiv.hidden = false; +} +if (qvars.style !== undefined) { + styleSelect.value = qvars.style; + updateContrastWarning(); +} +if (qvars.formatter !== undefined) { + formatterSelect.value = qvars.formatter; +} + +styleSelect.addEventListener('change', () => { + style.textContent = styles[styleSelect.value]; + updateCopyLink(); + updateContrastWarning(); +}); - var stylelist = pyodide.runPython('list(pygments.styles.get_all_styles())'); - var sel = document.getElementById("style"); - for (sty of stylelist) { - if (sty != "default") { - var opt = document.createElement("option"); - opt.text = sty; - opt.value = sty; - sel.add(opt); - } - } +function updateContrastWarning() { + contrastWarning.hidden = styleSelect.selectedOptions[0].dataset.wcag == 'aa'; +} - document.getElementById("hlbtn").disabled = false; - document.getElementById("loading").style.display = "none"; +function debounce(func, timeout) { + let timer; + return (...args) => { + clearTimeout(timer); + timer = setTimeout(() => func.apply(this, args), timeout); + }; +} - if (qvars['code'] !== undefined) { - document.getElementById("code").value = qvars['code']; - highlight(); - } - }); +const highlightShortDebounce = debounce(highlight, 50); +const highlightLongDebounce = debounce(highlight, 500); + +function debouncedUpdate() { + if (fileInput.files.length > 0) + return; + + if (textarea.value.length < 1000) { + highlightShortDebounce(); + } else { + highlightLongDebounce(); + } +} + +langSelect.addEventListener('change', debouncedUpdate); +textarea.addEventListener('input', debouncedUpdate); +formatterSelect.addEventListener('change', debouncedUpdate); +fileInput.addEventListener('change', () => { + fileInputResetButton.hidden = false; + highlight(); +}); +fileInputResetButton.hidden = fileInput.files.length == 0; +fileInputResetButton.addEventListener('click', () => { + fileInputResetButton.hidden = true; + fileInput.value = ''; + highlight(); }); -function getQueryVariables() { - var query = window.location.search.substring(1); - var vars = query.split('&'); - var var_obj = {}; - for (var i = 0; i < vars.length; i++) { - var pair = vars[i].split('='); - var_obj[decodeURIComponent(pair[0])] = decodeURIComponent(pair[1]); - } - return var_obj; -} - -function new_file() { - pyodide.globals['fname'] = document.getElementById("file").files[0].name; - var alias = pyodide.runPython('pygments.lexers.find_lexer_class_for_filename(fname).aliases[0]'); - var sel = document.getElementById("lang"); - for (var i = 0; i < sel.length; i++) { - if (sel.options[i].value == alias) { - sel.selectedIndex = i; - reset_err_hl(); - break; +let styles; + +const highlightWorker = new Worker("/_static/demo-worker.js"); +highlightWorker.onmessage = (msg) => { + if (msg.data.loaded) { + styles = msg.data.loaded.styles; + + if (qvars.code !== undefined || textarea.value) { + loadingDiv.hidden = true; + highlight(); + } + } else if (msg.data.html) { + outputDiv.innerHTML = msg.data.html; + codeHeader.hidden = false; + loadingDiv.hidden = true; + style.textContent = styles[styleSelect.value]; + } else if (msg.data.tokens) { + const table = document.createElement('table'); + table.className = 'tokens'; + for (const [tokenType, value] of msg.data.tokens) { + const tr = document.createElement('tr'); + const td1 = document.createElement('td'); + td1.textContent = tokenType.join('.'); + const td2 = document.createElement('td'); + const inlineCode = document.createElement('code'); + inlineCode.textContent = value; + td2.appendChild(inlineCode); + tr.appendChild(td1); + tr.appendChild(td2); + table.appendChild(tr); } + outputDiv.innerHTML = ''; + outputDiv.appendChild(table); + + codeHeader.hidden = false; + loadingDiv.hidden = true; + } else if (msg.data.lexer) { + highlight(msg.data.lexer); + } else { + console.warn('unexpected message from highlight worker', msg); } -} +}; -function reset_err_hl() { - document.getElementById("aroundlang").style.backgroundColor = null; +function updateCopyLink() { + var url = document.location.origin + document.location.pathname + + "?" + new URLSearchParams({ + lexer: langSelect.value, + style: styleSelect.value, + formatter: formatterSelect.value, + code: textarea.value, + }).toString() + if (url.length > 8201) { + // pygments.org is hosted on GitHub pages which does not support URIs longer than 8201 + copyLink.hidden = true; + uriTooLongMsg.hidden = false; + } else { + copyLink.href = url; + copyLink.textContent = 'Copy link'; + copyLink.hidden = false; + uriTooLongMsg.hidden = true; + } } -function highlight() { - var select = document.getElementById("lang"); - var alias = select.options.item(select.selectedIndex).value +async function highlight(guessedLexer) { + var lexer = langSelect.value || guessedLexer; + var file = fileInput.files[0]; + + let code; + if (file) { + code = await file.arrayBuffer(); + } else { + code = textarea.value; + } + + loadingDiv.hidden = false; - if (alias == "") { - document.getElementById("aroundlang").style.backgroundColor = "#ffcccc"; + if (!lexer) { + const guess_lexer = {code}; + if (file) + guess_lexer.filename = file.name; + highlightWorker.postMessage({guess_lexer}); + document.getElementById('loading-text').textContent = 'guessing lexer...'; return; } - pyodide.globals['alias'] = alias; - var select = document.getElementById("style"); - pyodide.globals['style'] = select.options.item(select.selectedIndex).value; + document.getElementById('loading-text').textContent = 'highlighting code...'; - pyodide.runPython('lexer = pygments.lexers.get_lexer_by_name(alias)'); - pyodide.runPython('fmter = pygments.formatters.html.HtmlFormatter(noclasses=True, style=style)'); + document.getElementById('guessed-lexer').textContent = guessedLexer; - var file = document.getElementById("file").files[0]; - if (file) { - file.arrayBuffer().then(function(buf) { - pyodide.globals['code_mem'] = buf; - pyodide.runPython('code = bytes(code_mem)'); - document.getElementById("copy_btn").style.display = "none"; - highlight_now(); - }); + highlightWorker.postMessage({highlight: {code, lexer, formatter: formatterSelect.value}}); + + if (code instanceof ArrayBuffer) { + copyLink.hidden = true; + uriTooLongMsg.hidden = true; } else { - var code = document.getElementById("code").value; - pyodide.globals['code'] = code; - var link = document.location.origin + document.location.pathname + - "?lexer=" + encodeURIComponent(alias) + "&code=" + encodeURIComponent(code); - document.getElementById("copy_field").value = link; - document.getElementById("copy_btn").style.display = ""; - highlight_now(); + updateCopyLink(); } } -function highlight_now() { - var out = document.getElementById("hlcode"); - out.innerHTML = pyodide.runPython('pygments.highlight(code, lexer, fmter)'); - document.location.hash = "#try"; - document.getElementById("hlcodedl").style.display = "block"; -} - -function copy_link() { - var copy_field = document.getElementById("copy_field"); - copy_field.select(); - copy_field.setSelectionRange(0, 99999); - document.execCommand("copy"); -} +copyLink.addEventListener('click', async (e) => { + e.preventDefault(); + await navigator.clipboard.writeText(e.target.href); +}); function download_code() { var filename = "highlighted.html"; - var hlcode = document.getElementById("hlcode").innerHTML; + var hlcode = document.getElementById("hlcode").innerHTML + style.outerHTML; var blob = new Blob([hlcode], {type: 'text/html'}); if (window.navigator.msSaveOrOpenBlob) { window.navigator.msSaveBlob(blob, filename); diff -Nru pygments-2.10.0+dfsg/doc/_static/demo-worker.js pygments-2.11.2+dfsg/doc/_static/demo-worker.js --- pygments-2.10.0+dfsg/doc/_static/demo-worker.js 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/_static/demo-worker.js 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,72 @@ +self.languagePluginUrl = '/_static/pyodide/'; +importScripts('/_static/pyodide/pyodide.js'); + +(async function() { + await languagePluginLoader; + await self.pyodide.loadPackage(["Pygments"]); + const styles = self.pyodide.runPython(` + from pygments.formatters.html import HtmlFormatter + from pygments.styles import STYLE_MAP + {s: HtmlFormatter(style=s).get_style_defs('.demo-highlight') for s in STYLE_MAP} + `); + self.postMessage({loaded: {styles}}) +})(); + +self.onmessage = async (event) => { + if (event.data.highlight) { + self.pyodide.globals['code'] = event.data.highlight.code; + self.pyodide.globals['lexer_name'] = event.data.highlight.lexer; + + self.pyodide.runPython(` + import pygments.lexers + + lexer = pygments.lexers.get_lexer_by_name(lexer_name) + if type(code) == memoryview: + code = bytes(code) + tokens = lexer.get_tokens(code) + `); + + const formatter = event.data.highlight.formatter; + if (formatter == 'html') { + + const html = self.pyodide.runPython(` + import io + from pygments.formatters.html import HtmlFormatter + + fmter = HtmlFormatter(cssclass='demo-highlight') + buf = io.StringIO() + fmter.format(tokens, buf) + buf.getvalue() + `); + self.postMessage({html}); + } else if (formatter == 'tokens') { + const tokens = self.pyodide.runPython('list(tokens)'); + self.postMessage({tokens}); + } else { + console.warn('unknown formatter:', formatter); + } + } else if (event.data.guess_lexer) { + self.pyodide.globals['code'] = event.data.guess_lexer.code; + self.pyodide.globals['filename'] = event.data.guess_lexer.filename; + const lexer = self.pyodide.runPython(` + import sys + sys.setrecursionlimit(1000) + # TODO: remove after upgrading to Pyodide 0.19 + + import pygments.lexers + import pygments.util + + if type(code) == memoryview: + code = bytes(code) + + if filename: + lexer = pygments.lexers.guess_lexer_for_filename(filename, code) + else: + lexer = pygments.lexers.guess_lexer(code) + lexer.aliases[0] + `); + self.postMessage({lexer}); + } else { + console.warn('unknown command: expected highlight or guess_lexer but received ', event.data); + } +} diff -Nru pygments-2.10.0+dfsg/doc/styles.rst pygments-2.11.2+dfsg/doc/styles.rst --- pygments-2.10.0+dfsg/doc/styles.rst 1970-01-01 00:00:00.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/styles.rst 2022-01-06 11:12:07.000000000 +0000 @@ -0,0 +1,5 @@ +:orphan: + +This file is overridden by _templates/styles.html and just exists to allow the +Styles gallery to be reliably linked from the documentation +(since its location varies between `make html` and `make dirhtml`). diff -Nru pygments-2.10.0+dfsg/doc/_templates/demo.html pygments-2.11.2+dfsg/doc/_templates/demo.html --- pygments-2.10.0+dfsg/doc/_templates/demo.html 2021-01-12 06:49:17.000000000 +0000 +++ pygments-2.11.2+dfsg/doc/_templates/demo.html 2022-01-06 11:12:07.000000000 +0000 @@ -4,9 +4,6 @@ {% block extrahead %} {{ super() }} - - - {% endblock %} {% block htmltitle %}Demo{{ titlesuffix }}{% endblock %} @@ -14,43 +11,87 @@ {% block body %} {{ body }} -

Demo - Try it out!

-

The highlighting here is performed in-browser using - a WebAssembly translation of the latest Pygments master branch, courtesy of - Pyodide.

-

Your content is neither sent over the web nor stored anywhere.

+

Try out Pygments!

+ +
-

Enter code and select a language

-
-

-   + + {% for name, info, _, _ in lexers %} + + {% endfor %} -    - ·   -  

-

  -   or enter code below:

-

-

- -    

-
-
-

-

Loading Python...

+ + +

+

+ +

+

+ + +

+
+ + +
+
+ + + +
-
+
{% endblock %} {% block footer %} +
{# closes "flexwrapper" div #}