Home → Problems →

Scrivener with german Umlauts in text

• Edit

Markus

30 May, 2023 08:53 AM

Hi Brett

I'm using Scrivener 3.3.1 (15588) together with Marked 2.6.22 (1040). For me Marked fails to preview the content of a text file of Scrivener if it contains german Umlauts. This does not concern the titles.

I attached a Scrivener file as a reference.

Cheers
Markus

// Configuration (do not edit)
Marked 2 1040 (Direct)
macOS Version: 10.15.7
---
additionalMathJaxConfig:
allowPageBreakInCode: YES
bookTxtIsLeanpub: NO
summaryMdIsGitBook: NO
codeIsPoetry: NO
codeCanWrap: YES
collapsibleHeadlines: NO
convertGithubCheckboxes: YES
convertGithubEmoji: YES
convertYAMLToMMD: NO
defaultMathJaxConfig: TeX-AMS-MML_HTMLorMML
defaultPreviewStyle: 101
defaultProcessor: MultiMarkdown
enableMiniMap: YES
excludeBlockQuotesFromCounts: NO
excludeFootnotesFromCounts: NO
excludeVerbatimFromCounts: NO
excludeCaptionsFromCounts: NO
folderPreviewExtensionsArray: md, markdown, txt, mdown, html, opml
githubNewlines: YES
h1IsPageBreak: NO
h2IsPageBreak: NO
hrIsPageBreak: NO
includeMathJax: NO
includeKatex: NO
limitTextWidth: NO
mathNumberEquations: NO
mathNumberEquationsSide: 0
mathNumberEquationsAMSOnly: NO
minimalJavaScript: NO
outlineModeExtensions:
permissionGranted: NO
printHeaderBottomLeft:
printHeaderBottomRight:
printHeaderFirstPage: NO
printFooterFirstPage: NO
printHeaderTopLeft:
printHeaderTopRight:
processHTMLFiles: YES
removeMetadataForPreview: NO
removeYAMLHeaders: NO
scrollToEdit: YES
shouldDetectCritic: YES
shouldHighlightMarkupErrors: NO
shouldLoadCritic: NO
syntaxHighlight: YES
syntaxHighlightOnlyLang: NO
useCustomMarkdownProcessor: NO
customMarkdownProcessor:
useCustomPreprocessor: NO
customPreprocessor:
useFirstH1AsPrintTitle: YES

• Umlaute.scriv.zip 37.3 KB

1. Support Staff 1 Posted by Brett on 30 May, 2023 12:07 PM

   

   Your test file didn't contain any umlauts so I added my own and was able
   to view it in Marked. Just for the sake of proper testing, can you
   either send a new file that does contain umlauts, or maybe just reply to
   this with the line that should have been there?

   Thanks,
   Brett

   On 30 May 2023, at 3:55, Markus wrote:

   • Edit

2. 2 Posted by Markus on 30 May, 2023 12:12 PM

   

   Sorry, I had deleted the line with the Umlauts for testing and forgot to put it in again. The attached file has them included.

   The line is as follows:

   Für das Geschenk möchte ich mich gebührlich bedanken.

   Cheers
   Markus

   • Umlaute-new.scriv.zip 35 KB
   • Edit

3. Support Staff 3 Posted by Brett on 30 May, 2023 12:41 PM

   

   This file renders fine for me, see attached image. I'm wondering if it has anything to do with the fact that you're on macOS 10.15 and I'm testing on macOS 13.4, as the Ruby setup has changed since then (Scrivener compilation uses a Ruby script to process).

   We can test this by opening Terminal and pasting the following command, modifying the path to the Scrivener file as needed (maybe need to modify /Applications too, if it's different on a German system):

   /Applications/Marked\ 2.app/Contents/Resources/ruby/scrivcompile.rb ~/Downloads/Umlaute-new.scriv
   

   Let me know if that makes sense, and if it does, let me know what output you get from it.

   • CleanShot_2023-05-30_at_07.37.17_2x.png 154 KB
   • Edit

4. 4 Posted by Markus on 30 May, 2023 12:46 PM

   

   In the terminal I get:

   Traceback (most recent call last):
     File "/Applications/Marked 2.app/Contents/Resources/ruby/html2text.py", line 971, in <module>
       main()
     File "/Applications/Marked 2.app/Contents/Resources/ruby/html2text.py", line 967, in main
       wrapwrite(h.handle(data),h.scrivener_doc)
     File "/Applications/Marked 2.app/Contents/Resources/ruby/html2text.py", line 261, in handle
       return self.optwrap(self.close())
     File "/Applications/Marked 2.app/Contents/Resources/ruby/html2text.py", line 273, in close
       self.outtext = self.outtext.join(self.outtextlist)
   UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

   • Edit

5. Support Staff 5 Posted by Brett on 30 May, 2023 12:49 PM

   

   Oh interesting, I’ve never seen the Python script portion of it have trouble with extended ascii. I’ll need some time to look into how to work around that, but will try to solve it this week. Thanks for the debugging help, I’ll post a test version of Marked for you as soon as I think I’ve solved it.

   Thanks,
   Brett
   On May 30, 2023 at 7:45 AM -0500, Markus <[email blocked]>, wrote:

   • Edit

6. 6 Posted by Markus on 30 May, 2023 12:51 PM

   

   That's great :)

   Thank you
   Markus

   • Edit

7. Support Staff 7 Posted by Brett on 02 Jun, 2023 02:37 PM

   

   I think I have the problem solved. Please download below and confirm for me.

   http://asssets.markedapp.com.s3.amazonaws.com/Marked2.6.241043.dmg

   • Edit

8. 8 Posted by Markus on 02 Jun, 2023 04:09 PM

   

   Unfortunately I still get an empty window. The terminal says:

   Traceback (most recent call last):
   5: from /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:376:in `<main>'
   4: from /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:376:in `each_with_index'
   3: from /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:376:in `each'
   2: from /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:390:in `block in <main>'
   1: from /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:167:in `markdownify'
   /Applications/Marked 2 2.app/Contents/Resources/ruby/scrivcompile.rb:221:in `get_python_version': undefined method `empty?' for nil:NilClass (NoMethodError)

   • Edit

9. Support Staff 9 Posted by Brett on 02 Jun, 2023 04:15 PM

   

   Shoot, I can fix that. Will have a new build in an hour.

   Thanks,
   Brett

   • Edit

10. Support Staff 10 Posted by Brett on 02 Jun, 2023 04:34 PM

    

    Try this build, please:

    https://stuff.brettterpstra.com/Marked2.6.241044.dmg

    (sorry for the different download URL, something broke with my uploader
    to Marked assets)

    -Brett

    On 2 Jun 2023, at 11:09, Markus wrote:

    • Edit

11. 11 Posted by Markus on 02 Jun, 2023 04:58 PM

    

    This version works :) I also tried a more complex Scrivener document and it shows up fine.

    Thank you very much for your quick support!

    Cheers
    Markus

    • Edit

12. Support Staff 12 Posted by Brett on 02 Jun, 2023 05:19 PM

    

    Excellent, thanks for helping me debug!

    -Brett

    On 2 Jun 2023, at 11:57, Markus wrote:

    • Edit