Home → Questions →

Include HTML directive

• Edit

Carl

17 Dec, 2015 05:49 PM

Hi,

I have an HTML document that pulls its data from a JSON file. I need to render multiple copies of this into my document but specify a different dataset/json file for each. Is there any way to achieve this ?

1. Support Staff 1 Posted by Brett on 17 Dec, 2015 06:09 PM

   

   I'm not certain I understand. Is the original document HTML or Markdown?
   Is the HTML populated via a GET call from JavaScript, or processed prior
   to rendering the output? Are you trying to use <<{} includes but with a
   query string?

   -Brett

   • Edit

2. 2 Posted by Carl on 17 Dec, 2015 06:17 PM

   

   The original document is markdown which looks a bit like this:

   Problem Statement

   We need to do something

   4Ps v. Competition

   Section1

   <<{document1.html}

   <<{document2.html}

   Section2

   <<{document3.html}

   <<{document4.html}

   Section3

   <<{document5.html}

   <<{document6.html}

   Section4

   <<{document7.html}

   <<{document8.html}

   The HTML documents contain charts, each of which has a different data set. Rather than me having to have 8 duplicated HTML documents, the only thing different in each is the data for the charts. I was hoping to have 1 single document and somehow use the <<{} directive to pass a variable that tells the HTML doc which JSON file to use for rendering the graph.

   • Edit

3. 3 Posted by Carl Bourne on 17 Dec, 2015 07:03 PM

   

   Make sense now ?

   • Edit

4. 4 Posted by Carl Bourne on 17 Dec, 2015 08:06 PM

   

   So this is a sample of the data that I'm trying to pass into the HTML page that renders the chart.

   data = {
   "title": 'Title 1', "categories": [ "Category1", "Category2", "Category3", "Category4", "Category5" ], "competition": { "Item1": [5, 8, 1, 2, 5 ], "Item2": [5, 8, 1, 2, 0 ], "Item3": [5, 8, 1, 2, 0 ], "Item4": [5, 8, 1, 2, 0 ], "Item5": [5, 8, 1, 2, 5 ], "Item6": [9, 9, 1, 2, 5 ], "Item7": [8, 5, 1, 2, 8 ], "Item8": [8, 5, 1, 2, 8 ], "Item9": [2, 5, 5, 5, 10 ], "Item10": [5, 5, 3, 3, 10 ] }, "misc": [2, 7, 9, 9, 10] };

   • Edit

5. Support Staff 5 Posted by Brett on 17 Dec, 2015 08:10 PM

   

   Not something Marked is going to do on its own, but you could pretty
   easily create a pre-processor script to do it. You'd set up a syntax
   like "%%json-file%%" and then parse for the %%tags%% in the
   preprocessor, replacing with the necessary markup and injecting the JSON
   file into the text returned to Marked for processing.

   I can help a bit with this if you have a foundation in a shell scripting
   language. Let me know.

   Side note: If you're on the App Store version, you'll need to set the
   preprocessor to the interpreter for the script (/bin/bash,
   /usr/bin/ruby, etc.) and the script as the argument. On the direct
   version you can just make the script executable and use it as the
   processor.

   • Edit

6. 6 Posted by Carl on 17 Dec, 2015 09:12 PM

   

   Looks good but maybe a little ambitious as my skills are more aligned to Ruby, Python and Golang. I've not done that much with shell scripting.

   Actually the HTML file is pretty minimal its mainly JS so I guess I could also inject that at the same time as the data ?

   Here's the whole thing:
   • Edit

7. 7 Posted by Carl on 17 Dec, 2015 09:15 PM

   

   Heres The file.

   • test.html 2.71 KB
   • Edit

8. 8 Posted by Carl on 18 Dec, 2015 01:04 PM

   

   Hi,

   Could you please give me an idea as to how I can started with creating a pre-processor script. I'll give this a go and see where it takes me. Can this be done in Ruby or Python or does it need to be shell script ?

   It would be great to see this kind of functionality ultimatly make it into the marked itself.

   Thanks

   • Edit

9. 9 Posted by Carl on 18 Dec, 2015 01:30 PM

   

   OK I should have read the docs. I just found the section that covers the pre-processor so I think I understand how that works now.

   So I get the part for creating a syntax like %%json-file%%. So my pre-processor would then find the instances of %%json-file%% and replace it with my HTML markup and JSON data which I guess would result in a markup document that now contains raw HTML and custom scripts including the related data.

   Is my understanding correct here ?

   From the docs, it does look like this processor can be created using any language ?

   • Edit

10. Support Staff 10 Posted by Brett on 18 Dec, 2015 01:42 PM

    

    That's correct. You're best off limiting yourself to the languages
    provided with OS X by default (bash, python, ruby) as Marked can only
    load a basic shell when it executes, and any environment variables or
    custom installs may not be able to run.

    And yes, if you use the html file as a template with its a placeholder
    for the data or link, your custom script can take the filename from the
    token and use it to concatenate the html shell and the data file for
    each token found.

    Let me know how it goes.

    -Brett

    • Edit

11. 11 Posted by Carl on 18 Dec, 2015 01:53 PM

    

    Thanks Brett.

    So I thinking of using Python and the Jinja2 templating system.

    So something like this:
    -----Test.md--------------- #Heading This is body text. graph will display next......
    {{ json-file}}

    This is more body.

    from jinja2 import Template

    template = Template(test.md) template.render(json='custom code here')

    Except that I'm guessing the test.md will be passed in via stdin and not a file.

    Does this look feasible ?

    Carl

    • Edit

12. 12 Posted by Carl on 18 Dec, 2015 02:04 PM

    

    Brett,

    So something more like this, where arg1 is the MD including the {{json-file}} and arg2 is the HTML/JSON/JS data that gets injected.

    from jinja2 import Template
    import sys
    input = sys.argv[1]
    template = Template(input)
    output = template.render(name=sys.argv[2])
    print output

    Carl

    • Edit

13. Support Staff 13 Posted by Brett on 18 Dec, 2015 02:27 PM

    

    That would require the argument to be variable, and the processor can't
    handle that. What you want is for the token to contain the filename you
    need, and the script to parse the STDIN input for matching tokens via
    regex (e.g. /%%(.*?)%%/) and then handle the templating based on that.

    -Brett

    • Edit

14. 14 Posted by Carl on 18 Dec, 2015 03:30 PM

    

    Arh - make sense. I'll try that.

    • Edit

15. 15 Posted by Carl on 18 Dec, 2015 05:31 PM

    

    Hi Brett,

    Are there any examples of a simple pre-processor preferably in python ? Just struggling a little bit understanding the input output process. Also, I assume if the pre-processor can only access the system default python its not going to pick up any non standard libraries.

    Carl

    • Edit

16. Support Staff 16 Posted by Brett on 18 Dec, 2015 05:48 PM

    

    The libraries generally work if installed with sudo easy_install, but pip and pyenv can cause issues that require a wrapper to load an environment.

    I have examples in Ruby, might be able to find some in Python, I'll get back to you when I'm back at my desk. Basically you'll read STDIN, do whatever you like with the text, then print it to STDOUT as Markdown for processing. If you want to set up a processor (as opposed to preprocessor) the script is responsible for returning html instead of Markdown.

    -Brett

    • Edit

17. 17 Posted by Carl on 18 Dec, 2015 06:01 PM

    

    Thanks Brett,

    Anything you've got will be much appreciated.

    Carl

    • Edit

18. 18 Posted by Carl on 18 Dec, 2015 09:09 PM

    

    Hi Brett,

    If you can't find any Python examples, Ruby will be fine. I'm sure I can work it out.

    Carl

    • Edit

19. Support Staff 19 Posted by Brett on 18 Dec, 2015 09:39 PM

    

    I'm not finding any of the Python ones that have been shown to me right now, I need to collect those in a GitHub repo for people. At its most basic, here's a ruby example:

     #!/usr/bin/ruby
    
     content = STDIN.read
     content += "\n\n<!-- marked style: BrettTerpstra.com -->\n"
     puts content
    

    Read it in, modify it, print it out. Yours will need to do a regex search and replace, which in Ruby would be easily accomplished with a gsub block. I'm not well versed enough with Python to do it easily, but the CriticMarkup preprocessor is a good start.

    • Edit

20. 20 Posted by Carl on 05 Jan, 2016 01:40 PM

    

    OK, So I built a custom pre-processor thats pretty much working except that it appears that some of the data ranges are being overwritten.

    I've attached an example of my markdown and the pre-processor.

    I think the the problem appears to be with the way the HTML is being injected into the document since the debug python log (logdata.log) shows that the data is being correctly interpreted and passed into my HTML template.

    I purposely created separate named data structures based on the value of the ID contained within the YAML data.

    I'm thinking I'm probably doing something really stupid but would like a second oppinion if thats OK.

    Carl

    • Edit

21. 21 Posted by Carl on 05 Jan, 2016 01:42 PM

    

    Doesn't seem to have uploaded my sample. Trying again.

    • test.md 1.26 KB
    • pre-processor.py 3.12 KB
    • Edit

22. Support Staff 22 Posted by Brett on 05 Jan, 2016 03:33 PM

    

    Ok, so I got this working with a little bit of hacking. First, the charts aren't showing up because Marked is moving the script includes to the bottom of the body element, but the jQuery calls to highcharts are happening before that load. I did two things to correct this:

    First, instead of using the XHTML Header metadata, I had the script write out the script tag immediately before returning the response.

    # print h
    sys.stdout.write('<script src="https://code.jquery.com/jquery-2.1.4.min.js"></script><script src="https://code.highcharts.com/highcharts.js"></script><script src="https://code.highcharts.com/modules/exporting.js"></script>')
    sys.stdout.write(h)
    

    Then I altered the anonymous function wrappers to ensure the remote jQuery had time to load before it attempted to call anything with the $ method:

    (function($){ [code] })(jQuery);
    

    As opposed to what you had:

    $(function () { [code] });
    

    Lastly, it was throwing a fatal error because console.log was being called without quotes on a variable that wasn't explicitly defined at that time, so changing the python script template on line 38 to this fixes that issue:

    console.log('{{ id }}')
    

    Hope that helps.

    -Brett

    • Edit

23. 23 Posted by Carl on 05 Jan, 2016 06:51 PM

    

    Hi Brett,

    Thanks for this. So I made the changes (see attached), however whilst it is rendering the charts its throwing the following error.

    TypeError: $('#c1').highcharts is not a function. (In '$('#c1').highcharts', '$('#c1').highcharts' is undefined)
    Error: Highcharts error #16: www.highcharts.com/errors/16

    • Edit

24. 24 Posted by Carl on 05 Jan, 2016 06:51 PM

    

    Upload failed

    • pre-processor.py 3.35 KB
    • test.md 1.06 KB
    • Edit

25. 25 Posted by Carl on 05 Jan, 2016 10:27 PM

    

    Hi Brett,

    The other thing I've noticed is that the JQuery and Highcharts libs are being loaded twice as shown in the screenshot. Also, with this approach none of the interactive chart elements in the charts seem to work. Do they work for you ?

    Carl

    • Screen_Shot_2016-01-05_at_21.45.23.png 190 KB
    • Edit

26. Support Staff 26 Posted by Brett on 05 Jan, 2016 11:04 PM

    

    Did you remove the xhtml header meta from the md file?

    - Brett

    • Edit

27. 27 Posted by Carl on 05 Jan, 2016 11:06 PM

    

    Yes, I certainly did. Double checked that. By the way, If I export to HTML everything looks and works OK.

    • Edit

28. Support Staff 28 Posted by Brett on 06 Jan, 2016 04:31 PM

    

    Ok, I'm not getting that result at all. I'm attaching my version of the pre-processor, as well as the test document just so we can make sure we're on the same page.

    If you could, let me know what version of Marked you're currently running, and please attach a screenshot of the source view (⌘U) of the result you're getting.

    Thanks,
    Brett

    • test.md 1.05 KB
    • pre-processor.py 3.35 KB
    • Edit