tag:support.markedapp.com,2011-09-11:/discussions/questions/10283-marked2-refresh-breaks-with-pandockatexMarked: Discussion 2020-09-10T15:09:21Ztag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T13:01:57Z2020-09-08T13:01:59ZMarked2 refresh breaks with pandoc/katex<div><p>Hi,</p>
<p>I am using a custom preprocessor (pandoc) with the following options:</p>
<ul>
<li>Path: <code>/usr/local/bin/pandoc</code></li>
<li>Args: <code>-f markdown -t html5 -s --katex</code></li>
</ul>
<p>This leads to the refresh not working properly anymore. That is, when I change the .md file, the inline math is not displayed properly in Marked 2. Explicitly right-clicking and selecting "Refresh" in the Marked2 window fixes it, but that's obviously not a great workflow.</p>
<p>I am using Marked 2 V2.6.1</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T13:45:04Z2020-09-08T13:45:04ZMarked2 refresh breaks with pandoc/katex<div><p>Remove the —katex flag and let me know if that helps.</p>
<ul>
<li>Brett</li>
</ul></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T14:24:46Z2020-09-08T14:24:47ZMarked2 refresh breaks with pandoc/katex<div><p>Hi Brett,</p>
<p>thank you for looking into this. Removing the <code>--katex</code> flag fixes the refresh problem.</p>
<p>Unfortunately I need the <code>--katex</code> flag to force pandoc to use proper latex output instead of unicode. I guess if pandoc had a flag <code>--dontUseUnicode</code> my problem would be solved.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T14:34:50Z2020-09-08T14:34:50ZMarked2 refresh breaks with pandoc/katex<div><p>I'm not familiar enough with Pandoc to be certain why the katex flag is<br>
messing up the refresh to begin with. It's odd that it works on first<br>
load (or refresh) but not on update. I don't currently have an<br>
explanation for that.</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T14:41:54Z2020-09-08T14:41:58ZMarked2 refresh breaks with pandoc/katex<div><p>No worries, I'll find a workaround for that. I think the problem is somehow that pandoc includes a link to the katex-script with the <code>--katex</code> option.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T14:46:09Z2020-09-08T14:46:09ZMarked2 refresh breaks with pandoc/katex<div><p>Yeah, that would botch Marked's built-in script as it loads a<br>
conflicting script with every refresh… I might be able to test for the<br>
existence of an embedded script and compensate for it, but I'll have to<br>
look into it.</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T17:02:52Z2020-09-08T17:02:52ZMarked2 refresh breaks with pandoc/katex<div><p>So without the --katex flag, Pandoc appears to do its own mathML<br>
rendering, but with the flag it simply outputs a span with the class<br>
"math inline". Which <em>should</em> be exactly what the built-in katex processor needs to do its own work, so it appears it's working in<br>
reverse. I'm presently very confused, but will continue looking into<br>
this as you're the third person to report the issue with the exact same<br>
circumstances.</p>
<p>As far as the unicode processing issue, can you provide me with a sample<br>
document that shows the type of input you're giving it and getting<br>
incorrect output when <code>--katex</code> isn't included in the arguments? (Not<br>
being a "math" guy myself, I might need an explanation of what's<br>
incorrect about the rendering, but we'll see :)).</p>
<p>Thanks,<br>
Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T18:20:56Z2020-09-08T18:20:57ZMarked2 refresh breaks with pandoc/katex<div><p>Yes, I see a little bit clearer now myself. I've been using Pandoc seriously just since yesterday, so just getting used to its features myself.</p>
<p>Here is an example markdown input: <code>$c \in \mathcal{C}$</code></p>
<p>You are right, without the <code>--katex</code> flag, Pandoc is doing its own rendering using built-in features of the browser, producing output like<br></p>
<pre>
<code><span class="math inline"><em>c</em> ∈ 풞</span></code>
</pre>
<p>This works with Marked2 without problems.</p>
<p><em>With</em> the <code>--katex</code> flag, it produces output meant for post processing with the Katex script, like this:</p>
<pre>
<code><span class="math inline">\(c \in \mathcal{C}\)</span></code>
</pre>
<p>It seems to me almost that the problem is that the post-processing that is supposed to happen via the KaTeX engine doesn't happen on automatic refresh somehow, but just on manual refresh. Are you actually loading the site new on automatic refresh, or are you doing something maybe that updates the page without triggering the KaTeX postprocessing of the DOM?</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T18:43:31Z2020-09-08T18:43:31ZMarked2 refresh breaks with pandoc/katex<div><p>Automatic updates happen by injecting new content into the DOM rather than a full refresh (as of version 1.6). However, on every update it should be triggering MathJax/Katex rendering (depending on what’s enabled in preferences). So I just need to figure out why this works with the built-in processors but not with pandoc…</p>
<ul>
<li>Brett</li>
</ul></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T20:46:28Z2020-09-08T20:46:30ZMarked2 refresh breaks with pandoc/katex<div><p>As far as I can tell, Pandoc doesn't do anything special except insert the inline-math as usual and then inserting code in the html header to load the script. So I guess the difference must be in how the script loading works in the various circumstances.</p>
<p>I have Katex rendering enabled in Marked2 preferences, if that makes a difference?</p>
<p>With the standard Pandoc installation this header is generated:</p>
<pre>
<code><script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/katex.min.js"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/contrib/auto-render.min.js"></script><script>document.addEventListener("DOMContentLoaded", function() {
renderMathInElement(document.body);
});</script><link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.8.3/katex.min.css" /></code>
</pre>
<p>This leads to the observed behaviour. I've also tried changing the template that Pandoc uses so that instead the following header is generated to include the newest katex:</p>
<pre>
<code><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/katex.min.css" integrity="sha384-AfEj0r4/OFrOo5t7NnNe46zW/tFgW6x/bCJG8FqQCEo3+Aro6EYUG4+cU+KJWu/X" crossorigin="anonymous">
<!-- The loading of KaTeX is deferred to speed up page rendering -->
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/katex.min.js" integrity="sha384-g7c+Jr9ZivxKLnZTDUhnkOnsh30B4H0rpLUpJ4jAIKs4fnJI+sEnkvrMWph2EDg4" crossorigin="anonymous"></script>
<!-- To automatically render math in text elements, include the auto-render extension: -->
<script defer src="https://cdn.jsdelivr.net/npm/katex@0.12.0/dist/contrib/auto-render.min.js" integrity="sha384-mll67QQFJfxn0IYznZYonOWZ644AWYC+Pt2cHqMaRhXVrursRwvLnLaebdGIlYNa" crossorigin="anonymous"
onload="renderMathInElement(document.body);"></script></code>
</pre>
<p>Doesn't work either, even worse: now even manual refresh doesn't work anymore.</p>
<p>When I switch off custom processing, the update works without problems. Unfortunately I cannot use it as I am relying on some pandoc features like the ability to define new latex macros.</p>
<p>I've looked at the html code that is exported by Marked 2, and adapted the pandoc template so that it looks the same. No difference, still doesn't work.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T21:11:56Z2020-09-08T21:11:56ZMarked2 refresh breaks with pandoc/katex<div><p>So the scripts you show are running renderMathInElement only on<br>
DOMContentLoaded events, which wouldn't be triggered by the update.<br>
However, I note that manually running renderMathInElement doesn't work<br>
with the markup that Pandoc is inserting when <code>--katex</code> is specified.<br>
Something is missing from that resulting markup that isn't allowing<br>
Katex to perform its rendering, whether loaded by Marked or by Pandoc.<br>
Still investigating...</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T21:27:40Z2020-09-08T21:27:40ZMarked2 refresh breaks with pandoc/katex<div><p>Ok, so test this out for me, if you don't mind:</p>
<ol>
<li>Include the --katex flag in the custom processor arguments<br></li>
<li>Load a file where the math no longer renders now<br></li>
<li>Right click in the preview and choose "Inspect Element"<br></li>
<li>
<p>In the inspector, click in the bottom (the "Terminal") and paste in<br>
the following:</p>
<p>document.querySelectorAll('.math').forEach(function(i) { katex.render(i.innerText, i); })</p>
</li>
</ol>
<p>See if that causes everything to render the way you expect it to. If so,<br>
I'll just need to figure out a way to trigger that when needed... which<br>
will probably be just as confusing, but at least I'll have a start.</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T21:45:13Z2020-09-08T21:45:14ZMarked2 refresh breaks with pandoc/katex<div><p>Of course, happy to test!</p>
<p>I've tried it, and it looks much better, the inline math is correct after running the code. But the display math (the one with [ ... ]) is shown inline instead of in display mode. That's probably because not only the display math element itself must be updated, but also the paragraph it is in.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T22:19:00Z2020-09-08T22:19:01ZMarked2 refresh breaks with pandoc/katex<div><p>I think the problem is that somehow the <code>\( \)</code> or <code>\[ \]</code> that surround the math text get lost. If in the terminal I execute</p>
<pre>
<code>document.querySelectorAll('.math').forEach(function(i) {
i.innerText = "\\[" + i.innerText + "\\]";});</code>
</pre>
<p>and then</p>
<pre>
<code>renderMathInElement(document.body);</code>
</pre>
<p>every math element is shown in display mode.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T22:57:46Z2020-09-08T22:57:47ZMarked2 refresh breaks with pandoc/katex<div><p>So if I type this into the terminal, everything displays perfectly:</p>
<pre>
<code>document.querySelectorAll('.math.inline').forEach(function(i) {
i.innerText = "\\(" + i.innerText + "\\)";});
document.querySelectorAll('.math.display').forEach(function(i) {
i.innerText = "\\[" + i.innerText + "\\]";});
renderMathInElement(document.body);</code>
</pre>
<p>But instead of running that code it's probably better to make sure that the escaped brackets don't get lost in the first place.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T23:05:38Z2020-09-08T23:05:38ZMarked2 refresh breaks with pandoc/katex<div><p>Yes, I was noticing that the delimiters were stripped by pandoc when —katex is used. Will have to work around that.</p>
<ul>
<li>Brett</li>
</ul></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T23:24:54Z2020-09-08T23:24:54ZMarked2 refresh breaks with pandoc/katex<div><p>Oh, but they are not stripped by pandoc! KaTex does not work when the delimiters are not there. When I redirect the pandoc output into a file, they are there. So its probably happening when you take the input from the custom processor and do your thing with it.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-08T23:29:54Z2020-09-08T23:29:54ZMarked2 refresh breaks with pandoc/katex<div><p>Interesting, I’ll look into that.</p>
<ul>
<li>Brett</li>
</ul></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T02:21:57Z2020-09-09T02:21:57ZMarked2 refresh breaks with pandoc/katex<div><p>When I run <code>pandoc --katex</code> on the following (from the command line):</p>
<pre>
<code>$\Delta\delta = \delta_2 - \delta_1$
\\(\Delta\delta = \delta_2 - \delta_1\\)</code>
</pre>
<p>I get:</p>
<pre>
<code><p><span class="math inline">\Delta\delta = \delta_2 - \delta_1</span></p>
<p>\(= _2 - _1\)</p></code>
</pre>
<p>Using $ delimiters, it creates the math span but removes the delimters. Using the escaped parens preserves the delimiters, but removes the <code>\delta</code>s. Neither renders properly. This appears to be a pandoc issue, though I can't understand how it's <em>supposed</em> to function.</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T05:12:02Z2020-09-09T05:12:03ZMarked2 refresh breaks with pandoc/katex<div><p>Mysterious stuff!! I am trying the same input. I've attached the file <code>test.md</code> which I run it on, and then I run <code>pandoc -f markdown -t html5 --katex test.md</code> and I get the output</p>
<pre>
<code><p><span class="math inline">\(\Delta\delta = \delta_2 - \delta_1\)</span></p>
<p>\(= _2 - _1\)</p></code>
</pre>
<p>So what is happening here is that the first one works OK, and the second one is probably just not valid pandoc math syntax?</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T06:57:34Z2020-09-09T06:57:35ZMarked2 refresh breaks with pandoc/katex<div><p>I just checked and the missing brackets is definitely a pandoc issue. I was using a different version on the command line vs. in Marked 2. On the command line I was using the pandoc from my anaconda3 installation which was version <code>2.2.3.2</code>. In Marked 2 I was using <code>2.10.1</code>.</p>
<p>The newer version omits the escaped brackets and that breaks the current version of KaTeX.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T07:02:27Z2020-09-09T07:02:27ZMarked2 refresh breaks with pandoc/katex<div><p>I wonder if it's intentional somehow. Like did KaTeX change the way it<br>
operates in a newer version or something?</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T07:16:33Z2020-09-09T07:16:36ZMarked2 refresh breaks with pandoc/katex<div><p>In the newer version they are now doing this to load katex:</p>
<pre>
<code><script src="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.js"></script>
<script>document.addEventListener("DOMContentLoaded", function () {
var mathElements = document.getElementsByClassName("math");
var macros = [];
for (var i = 0; i < mathElements.length; i++) {
var texText = mathElements[i].firstChild;
if (mathElements[i].tagName == "SPAN") {
katex.render(texText.data, mathElements[i], {
displayMode: mathElements[i].classList.contains('display'),
throwOnError: false,
macros: macros,
fleqn: false
});
}}});
</script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css" /></code>
</pre>
<p>So that explains why they are not doing the escaped brackets anymore, because they are calling katex.render manually now.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T07:53:32Z2020-09-09T07:53:36ZMarked2 refresh breaks with pandoc/katex<div><p>Ok, what I do now is: I use the</p>
<ul>
<li><code>--mathjax</code> flag for pandoc</li>
<li>switch on Mathjax in the Marked2 preferences</li>
</ul>
<p>Now it works flawlessly </p>
<p>I thought I tried that earlier, but probably it didn't work because of some combination of the changes I tried out with the wrong pandoc version and my custom pandoc template modifications.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-09T08:01:12Z2020-09-09T08:01:12ZMarked2 refresh breaks with pandoc/katex<div><p>Oh, I was too quick, now there are some other issues. I don't think it is worth for you to invest any more time into that, I guess it is just a can of worms. I will just use the built-in rendering without <code>--mathjax</code> or <code>--katex</code> flag which is good enough for prototyping and just check the result from time to time in the browser with <code>--katex</code> switched on.</p>
<p>Thank you so much for your help!</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T08:15:47Z2020-09-10T08:15:47ZMarked2 refresh breaks with pandoc/katex<div><p>I have a build that solves a lot of the issues, though there's still some weirdness with the way Pandoc strips out some characters when --katex is enabled. But give this a test and see if it helps overall:</p>
<p><a href="http://ckyp.us/yWakAx">Marked 2.6.2 (1008)</a></p>
<p>Thanks,<br>
Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T09:18:35Z2020-09-10T09:18:36ZMarked2 refresh breaks with pandoc/katex<div><p>I see you are not one to easily give up!</p>
<p>Here are my observations for the new version:</p>
<ul>
<li>
<p>when using <code>--katex</code> and Katex flag in Marked2 preferences on, the update now works, and scrolling to the right position also works. But it is garbled up, it seems that macros are lost, for example in <code>$(c_1, \ldots, c_n)$</code> the <code>c_1</code> and <code>c_2</code> render fine, but the <code>\ldots</code> is missing.</p>
</li>
<li>
<p>when using <code>--mathjax</code> and Mathjax flag on, update works, but it scrolls always to the top of the document. Also subindices do not work, as in <code>$S_1$</code>. There are some minor other problems as well.</p>
</li>
<li>
<p>when using no <code>--</code> option and neither Mathjax flag or Katex flag on, everything works great except for the fact that the built-in browser renderers are so atrocious, and display formulas appear inline and not display. When the Katex flag is on, it is all garbled up though, and when the Mathjax flag is on the subindices problem appears.</p>
</li>
</ul></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T13:20:16Z2020-09-10T13:20:16ZMarked2 refresh breaks with pandoc/katex<div><p>Ok, please try the same tests with this version. I think we're making progress! (I hope.)</p>
<p><a href="http://ckyp.us/OzzS61+">Marked 2.6.2 (1009)</a></p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T13:37:22Z2020-09-10T13:37:22ZMarked2 refresh breaks with pandoc/katex<div><p>On your second point, the subindices were breaking because I was<br>
manually escaping underscores, which apparently I don't need to do? I<br>
know there was a reason for it at some point, but I've lost track. As<br>
far as the scrolling goes when MathJax is enabled, I am seeing it scroll<br>
up a bit on refresh, but if "Scroll to first Edit" is enabled in<br>
Preferences->Preview, it restores its position properly. I'm not certain<br>
why MathJax is causing that and KaTeX isn't, will look into it.</p>
<p>Let me know how the latest test build goes.</p>
<p>-Brett</p></div>Bretttag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T14:26:13Z2020-09-10T14:26:14ZMarked2 refresh breaks with pandoc/katex<div><p>It works much much better now! As far as I can see, the only problem left is the scrolling to the top, which for me happens when the "Scroll to first Edit" is enabled. The edit-marker is also displayed at the very top.</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T14:32:00Z2020-09-10T14:32:02ZMarked2 refresh breaks with pandoc/katex<div><p>I had provided pandoc with a custom template with <code>--template custom.html</code>, and when I disable this, then "Scroll to first Edit" works properly! That's fine for me, as I don't need the custom template for editing, just later for publishing.</p>
<p>Thank you for digging into this and making it work!</p></div>Steven Obuatag:support.markedapp.com,2011-09-11:Comment/486120612020-09-10T15:09:20Z2020-09-10T15:09:20ZMarked2 refresh breaks with pandoc/katex<div><p>Awesome, glad to hear it’s improved. I’ll get a release out soon, but you’re welcome to keep using the test build for now.</p>
<ul>
<li>Brett</li>
</ul></div>Brett