# Marked2 refresh breaks with pandoc/katex

#### Steven Obua

08 Sep, 2020 01:01 PM

Hi,

I am using a custom preprocessor (pandoc) with the following options:

• Path: /usr/local/bin/pandoc
• Args: -f markdown -t html5 -s --katex

This leads to the refresh not working properly anymore. That is, when I change the .md file, the inline math is not displayed properly in Marked 2. Explicitly right-clicking and selecting "Refresh" in the Marked2 window fixes it, but that's obviously not a great workflow.

I am using Marked 2 V2.6.1

1. Support Staff Posted by Brett on 08 Sep, 2020 01:45 PM

Remove the —katex flag and let me know if that helps.

- Brett

2. Posted by Steven Obua on 08 Sep, 2020 02:24 PM

Hi Brett,

thank you for looking into this. Removing the --katex flag fixes the refresh problem.

Unfortunately I need the --katex flag to force pandoc to use proper latex output instead of unicode. I guess if pandoc had a flag --dontUseUnicode my problem would be solved.

3. Support Staff Posted by Brett on 08 Sep, 2020 02:34 PM

I'm not familiar enough with Pandoc to be certain why the katex flag is
messing up the refresh to begin with. It's odd that it works on first
load (or refresh) but not on update. I don't currently have an
explanation for that.

-Brett

4. Posted by Steven Obua on 08 Sep, 2020 02:41 PM

No worries, I'll find a workaround for that. I think the problem is somehow that pandoc includes a link to the katex-script with the --katex option.

5. Support Staff Posted by Brett on 08 Sep, 2020 02:46 PM

Yeah, that would botch Marked's built-in script as it loads a
conflicting script with every refresh… I might be able to test for the
existence of an embedded script and compensate for it, but I'll have to
look into it.

-Brett

6. Support Staff Posted by Brett on 08 Sep, 2020 05:02 PM

So without the --katex flag, Pandoc appears to do its own mathML
rendering, but with the flag it simply outputs a span with the class
"math inline". Which _should_ be exactly what the built-in katex
processor needs to do its own work, so it appears it's working in
reverse. I'm presently very confused, but will continue looking into
this as you're the third person to report the issue with the exact same
circumstances.

As far as the unicode processing issue, can you provide me with a sample
document that shows the type of input you're giving it and getting
incorrect output when --katex isn't included in the arguments? (Not
being a "math" guy myself, I might need an explanation of what's
incorrect about the rendering, but we'll see :)).

Thanks,
Brett

7. Posted by Steven Obua on 08 Sep, 2020 06:20 PM

Yes, I see a little bit clearer now myself. I've been using Pandoc seriously just since yesterday, so just getting used to its features myself.

Here is an example markdown input: $c \in \mathcal{C}$

You are right, without the --katex flag, Pandoc is doing its own rendering using built-in features of the browser, producing output like

<span class="math inline"><em>c</em> ∈ 풞</span>


This works with Marked2 without problems.

With the --katex flag, it produces output meant for post processing with the Katex script, like this:

<span class="math inline">$$c \in \mathcal{C}$$</span>


It seems to me almost that the problem is that the post-processing that is supposed to happen via the KaTeX engine doesn't happen on automatic refresh somehow, but just on manual refresh. Are you actually loading the site new on automatic refresh, or are you doing something maybe that updates the page without triggering the KaTeX postprocessing of the DOM?

8. Support Staff Posted by Brett on 08 Sep, 2020 06:43 PM

Automatic updates happen by injecting new content into the DOM rather than a full refresh (as of version 1.6). However, on every update it should be triggering MathJax/Katex rendering (depending on what’s enabled in preferences). So I just need to figure out why this works with the built-in processors but not with pandoc…

- Brett

9. Posted by Steven Obua on 08 Sep, 2020 08:46 PM

As far as I can tell, Pandoc doesn't do anything special except insert the inline-math as usual and then inserting code in the html header to load the script. So I guess the difference must be in how the script loading works in the various circumstances.

I have Katex rendering enabled in Marked2 preferences, if that makes a difference?

With the standard Pandoc installation this header is generated:

<script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/katex.min.js"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/contrib/auto-render.min.js"></script><script>document.addEventListener("DOMContentLoaded", function() {
renderMathInElement(document.body);
});</script><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/katex.min.css" />


This leads to the observed behaviour. I've also tried changing the template that Pandoc uses so that instead the following header is generated to include the newest katex:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/katex.min.css" integrity="sha384-AfEj0r4/OFrOo5t7NnNe46zW/tFgW6x/bCJG8FqQCEo3+Aro6EYUG4+cU+KJWu/X" crossorigin="anonymous">

<!-- The loading of KaTeX is deferred to speed up page rendering -->
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/katex.min.js" integrity="sha384-g7c+Jr9ZivxKLnZTDUhnkOnsh30B4H0rpLUpJ4jAIKs4fnJI+sEnkvrMWph2EDg4" crossorigin="anonymous"></script>

<!-- To automatically render math in text elements, include the auto-render extension: -->
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/contrib/auto-render.min.js" integrity="sha384-mll67QQFJfxn0IYznZYonOWZ644AWYC+Pt2cHqMaRhXVrursRwvLnLaebdGIlYNa" crossorigin="anonymous"
onload="renderMathInElement(document.body);"></script>


Doesn't work either, even worse: now even manual refresh doesn't work anymore.

When I switch off custom processing, the update works without problems. Unfortunately I cannot use it as I am relying on some pandoc features like the ability to define new latex macros.

I've looked at the html code that is exported by Marked 2, and adapted the pandoc template so that it looks the same. No difference, still doesn't work.

10. Support Staff Posted by Brett on 08 Sep, 2020 09:11 PM

So the scripts you show are running renderMathInElement only on
DOMContentLoaded events, which wouldn't be triggered by the update.
However, I note that manually running renderMathInElement doesn't work
with the markup that Pandoc is inserting when --katex is specified.
Something is missing from that resulting markup that isn't allowing
Katex to perform its rendering, whether loaded by Marked or by Pandoc.
Still investigating...

-Brett

11. Support Staff Posted by Brett on 08 Sep, 2020 09:27 PM

Ok, so test this out for me, if you don't mind:

1. Include the --katex flag in the custom processor arguments
2. Load a file where the math no longer renders now
3. Right click in the preview and choose "Inspect Element"
4. In the inspector, click in the bottom (the "Terminal") and paste in
the following:

document.querySelectorAll('.math').forEach(function(i) {
katex.render(i.innerText, i); })

See if that causes everything to render the way you expect it to. If so,
I'll just need to figure out a way to trigger that when needed... which
will probably be just as confusing, but at least I'll have a start.

-Brett

12. Posted by Steven Obua on 08 Sep, 2020 09:45 PM

Of course, happy to test!

I've tried it, and it looks much better, the inline math is correct after running the code. But the display math (the one with [ ... ]) is shown inline instead of in display mode. That's probably because not only the display math element itself must be updated, but also the paragraph it is in.

13. Posted by Steven Obua on 08 Sep, 2020 10:19 PM

I think the problem is that somehow the  or  that surround the math text get lost. If in the terminal I execute

document.querySelectorAll('.math').forEach(function(i) {
i.innerText = "\$" + i.innerText + "\$";});


and then

renderMathInElement(document.body);


every math element is shown in display mode.

14. Posted by Steven Obua on 08 Sep, 2020 10:57 PM

So if I type this into the terminal, everything displays perfectly:

document.querySelectorAll('.math.inline').forEach(function(i) {
i.innerText = "\$$" + i.innerText + "\$$";});
document.querySelectorAll('.math.display').forEach(function(i) {
i.innerText = "\$" + i.innerText + "\$";});
renderMathInElement(document.body);


But instead of running that code it's probably better to make sure that the escaped brackets don't get lost in the first place.

15. Support Staff Posted by Brett on 08 Sep, 2020 11:05 PM

Yes, I was noticing that the delimiters were stripped by pandoc when —katex is used. Will have to work around that.

- Brett

16. Posted by Steven Obua on 08 Sep, 2020 11:24 PM

Oh, but they are not stripped by pandoc! KaTex does not work when the delimiters are not there. When I redirect the pandoc output into a file, they are there. So its probably happening when you take the input from the custom processor and do your thing with it.

17. Support Staff Posted by Brett on 08 Sep, 2020 11:29 PM

Interesting, I’ll look into that.

- Brett

18. Support Staff Posted by Brett on 09 Sep, 2020 02:21 AM

When I run pandoc --katex on the following (from the command line):

$\Delta\delta = \delta_2 - \delta_1$

\$$\Delta\delta = \delta_2 - \delta_1\$$


I get:

<p><span class="math inline">\Delta\delta = \delta_2 - \delta_1</span></p>
<p>$$= _2 - _1$$</p>


Using $delimiters, it creates the math span but removes the delimters. Using the escaped parens preserves the delimiters, but removes the \deltas. Neither renders properly. This appears to be a pandoc issue, though I can't understand how it's supposed to function. 19. Posted by Steven Obua on 09 Sep, 2020 05:12 AM Mysterious stuff!! I am trying the same input. I've attached the file test.md which I run it on, and then I run pandoc -f markdown -t html5 --katex test.md and I get the output <p><span class="math inline">$$\Delta\delta = \delta_2 - \delta_1$$</span></p> <p>$$= _2 - _1$$</p>  So what is happening here is that the first one works OK, and the second one is probably just not valid pandoc math syntax? 20. Posted by Steven Obua on 09 Sep, 2020 06:57 AM I just checked and the missing brackets is definitely a pandoc issue. I was using a different version on the command line vs. in Marked 2. On the command line I was using the pandoc from my anaconda3 installation which was version 2.2.3.2. In Marked 2 I was using 2.10.1. The newer version omits the escaped brackets and that breaks the current version of KaTeX. 21. Support Staff Posted by Brett on 09 Sep, 2020 07:02 AM I wonder if it's intentional somehow. Like did KaTeX change the way it operates in a newer version or something? -Brett 22. Posted by Steven Obua on 09 Sep, 2020 07:16 AM In the newer version they are now doing this to load katex: <script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.js"></script> <script>document.addEventListener("DOMContentLoaded", function () { var mathElements = document.getElementsByClassName("math"); var macros = []; for (var i = 0; i < mathElements.length; i++) { var texText = mathElements[i].firstChild; if (mathElements[i].tagName == "SPAN") { katex.render(texText.data, mathElements[i], { displayMode: mathElements[i].classList.contains('display'), throwOnError: false, macros: macros, fleqn: false }); }}}); </script> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css" />  So that explains why they are not doing the escaped brackets anymore, because they are calling katex.render manually now. 23. Posted by Steven Obua on 09 Sep, 2020 07:53 AM Ok, what I do now is: I use the • --mathjax flag for pandoc • switch on Mathjax in the Marked2 preferences Now it works flawlessly  I thought I tried that earlier, but probably it didn't work because of some combination of the changes I tried out with the wrong pandoc version and my custom pandoc template modifications. 24. Posted by Steven Obua on 09 Sep, 2020 08:01 AM Oh, I was too quick, now there are some other issues. I don't think it is worth for you to invest any more time into that, I guess it is just a can of worms. I will just use the built-in rendering without --mathjax or --katex flag which is good enough for prototyping and just check the result from time to time in the browser with --katex switched on. Thank you so much for your help! 25. Support Staff Posted by Brett on 10 Sep, 2020 08:15 AM I have a build that solves a lot of the issues, though there's still some weirdness with the way Pandoc strips out some characters when --katex is enabled. But give this a test and see if it helps overall: Marked 2.6.2 (1008) Thanks, Brett 26. Posted by Steven Obua on 10 Sep, 2020 09:18 AM I see you are not one to easily give up! Here are my observations for the new version: • when using --katex and Katex flag in Marked2 preferences on, the update now works, and scrolling to the right position also works. But it is garbled up, it seems that macros are lost, for example in $(c_1, \ldots, c_n)$ the c_1 and c_2 render fine, but the \ldots is missing. • when using --mathjax and Mathjax flag on, update works, but it scrolls always to the top of the document. Also subindices do not work, as in $S_1\$. There are some minor other problems as well.

• when using no -- option and neither Mathjax flag or Katex flag on, everything works great except for the fact that the built-in browser renderers are so atrocious, and display formulas appear inline and not display. When the Katex flag is on, it is all garbled up though, and when the Mathjax flag is on the subindices problem appears.

27. Support Staff Posted by Brett on 10 Sep, 2020 01:20 PM

Ok, please try the same tests with this version. I think we're making progress! (I hope.)

Marked 2.6.2 (1009)

28. Support Staff Posted by Brett on 10 Sep, 2020 01:37 PM

On your second point, the subindices were breaking because I was
manually escaping underscores, which apparently I don't need to do? I
know there was a reason for it at some point, but I've lost track. As
far as the scrolling goes when MathJax is enabled, I am seeing it scroll
up a bit on refresh, but if "Scroll to first Edit" is enabled in
Preferences->Preview, it restores its position properly. I'm not certain
why MathJax is causing that and KaTeX isn't, will look into it.

Let me know how the latest test build goes.

-Brett

29. Posted by Steven Obua on 10 Sep, 2020 02:26 PM

It works much much better now! As far as I can see, the only problem left is the scrolling to the top, which for me happens when the "Scroll to first Edit" is enabled. The edit-marker is also displayed at the very top.

30. Posted by Steven Obua on 10 Sep, 2020 02:32 PM

I had provided pandoc with a custom template with --template custom.html, and when I disable this, then "Scroll to first Edit" works properly! That's fine for me, as I don't need the custom template for editing, just later for publishing.

Thank you for digging into this and making it work!

Comments are currently closed for this discussion. You can start a new one.

# Keyboard shortcuts

### Generic

? Show this help Blurs the current field

### Comment Form

r Focus the comment reply box Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

## Recent Discussions

 24 Sep, 2020 04:22 PM Exporting to PDF Highlights Words 22 Sep, 2020 02:48 PM Bear & Markdown Metadata 18 Sep, 2020 03:53 PM images breaking on preview automatic refresh 17 Sep, 2020 11:20 PM Markdown table formatting 12 Sep, 2020 06:05 PM webp format

## Recent Articles

 Fenced code blocks inside indented code blocks Using JavaScript in Marked Custom CSS: Writing custom CSS for Marked License code has already been utilized Highlight sentences longer than a certain number of words