From 7fcb56260eeb232ea300578b2f338feea5d2af7a Mon Sep 17 00:00:00 2001 From: mdipierro Date: Sat, 14 Jul 2012 23:05:41 -0500 Subject: [PATCH] rewrite of MARKMIN, thanks Vladyslav --- VERSION | 2 +- gluon/contrib/markmin/markmin.html | 102 ++-- gluon/contrib/markmin/markmin2html.py | 801 +++++++++++++++++++++----- 3 files changed, 713 insertions(+), 192 deletions(-) diff --git a/VERSION b/VERSION index f7868640..c5887c13 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -Version 2.00.0 (2012-07-14 22:45:25) dev +Version 2.00.0 (2012-07-14 23:05:38) dev diff --git a/gluon/contrib/markmin/markmin.html b/gluon/contrib/markmin/markmin.html index db637503..aa556392 100644 --- a/gluon/contrib/markmin/markmin.html +++ b/gluon/contrib/markmin/markmin.html @@ -1,52 +1,82 @@ -

Markmin markup language

About

This is a new markup language that we call markmin designed to produce high quality scientific papers and books and also put them online. We provide serializers for html, latex and pdf. It is implemented in the markmin2html function in the markmin2html.py.

Example of usage:

m = "Hello **world** [[link http://web2py.com]]"
+
+                 
+              
Markmin markup language
About
This is a new markup language that we call markmin designed to produce high quality scientific papers and books and also put them online. We provide serializers for html, latex and pdf. It is implemented in the markmin2html function in the markmin2html.py.
Example of usage:
m = "Hello **world** [[link http://web2py.com]]"
 from markmin2html import markmin2html
 print markmin2html(m)
 from markmin2latex import markmin2latex
 print markmin2latex(m)
 from markmin2pdf import markmin2pdf # requires pdflatex
-print markmin2pdf(m)
Why?
We wanted a markup language with the following requirements:
  • less than 200 lines of functional code
  • easy to read
  • secure
  • support table, ul, ol, code
  • support html5 video and audio elements (html serialization only)
  • can align images and resize them
  • can specify class for tables and code elements
  • can add anchors
  • does not use _ for markup (since it creates odd behavior)
  • automatically links urls
  • fast
  • easy to extend
  • supports latex and pdf including references
  • allows to describe the markup in the markup (this document is generated from markmin syntax)
(results depend on text but in average for text ~100K markmin is 30% faster than markdown, for text ~10K it is 10x faster)
The web2py book published by lulu, for example, was entirely generated with markmin2pdf from the online web2py wiki
Download
markmin2html.py and markmin2latex.py are single files and have no web2py dependence. Their license is BSD.
Examples
Bold, italic, code and links
SOURCE                                   OUTPUT
# title                                  title
## section                               section
### subsection                           subsection
**bold**                                 bold
''italic''                               italic
~~strikeout~~                            strikeout
``verbatim``                         verbatim
``color with **bold**``:red          color with bold
``many colors``:color[blue:#ffff00]  many colors
http://google.com                        http://google.com
[[**click** me #myanchor]]               click me
[[click me [extra info] #myanchor popup]]click me
More on links
The format is always [[title link]] or [[title [extra] link]]. Notice you can nest bold, italic, strikeout and code inside the link title.
Anchors 
You can place an anchor anywhere in the text using the syntax [[name]] where name is the name of the anchor.
-You can then link the anchor with link, i.e. [[link #myanchor]] or link with an extra info, i.e.
-[[link with an extra info [extra info] #myanchor]].
Images
alt-string for the image
-This paragraph has an image aligned to the right with a width of 200px. Its is placed using the code
[[alt-string for the image [the image title] http://www.web2py.com/examples/static/web2py_logo.png right 200px]].
Unordered Lists
- Dog
+print markmin2pdf(m)
 ====================
This is a test block with new features:
This is a blockquote with a list with tables in it:
This is a paragraph before list. You can continue paragraph on the next lines.
This is an ordered list with tables:
  1. Item 1
  2. Item 2
  3. aabbcc
    112233
  4. Item 4 
    T1T2t3
    aaabbbccc
    dddfffggg
    12305.0
This this a new paragraph with a table. Table has header and footer:
Title 1Title 2Title 3
data 1data 22.00
data 4data5(long)23.00
data 833.50
Total:3 items58.50
Multilevel lists
Now lists can be multilevel:
  1. Ordered item 1 on level 1. You can continue item text on next strings
    1. Ordered item 1 of sublevel 2 with a paragraph (paragraph can start with point after plus or minus characters, e.g. ++. or --.)
    2. This is another item. But with 3 paragraphs, blockquote and sublists:
      This is the second paragraph in the item. You can add paragraphs to an item, using point notation, where first characters in the string are sequence of points with space between them and another string. For example, this paragraph (in sublevel 2) starts with two points:
      .. This is the second paragraph...
      this is a blockquote in a list
      You can use blockquote with headers, paragraphs, tables and lists in it:
      Tables can have or have not header and footer. This table is defined without any header and footer in it:
      redfox0
      bluedolphin1000
      greenleaf10000
      This is yet another paragraph in the item.
      • This is an item of unordered list (sublevel 3)
      • This is the second item of the unordered list (sublevel 3)
            1. This is a single item of ordered list in sublevel 6
          and this is a paragraph in sublevel 4
      • This is a new item with paragraph in sublevel 3.
        1. Start ordered list in sublevel 4 with code block: 
          line 1
+  line 2
+     line 3
        2. Yet another item with code block:
            line 1
+line 2
+  line 3
           This item finishes with this paragraph.
        Item in sublevel 3 can be continued with paragraphs.
          this is another
+code block
+    in the
+  sublevel 3 item
      1. The last item in sublevel 3
      This is a continuous paragraph for item 2 in sublevel 2. You can use such structure to create difficult structured documents.
    3. item 3 in sublevel 2
    • item 1 in sublevel 2 (new unordered list)
    • item 2 in sublevel 2
    • item 3 in sublevel 2
    1. item 1 in sublevel 2 (new ordered list)
    2. item 2 in sublevel 2
    3. item 3 in sublevle 2
  2. item 2 in level 1
  3. item 3 in level 1
  • new unordered list (item 1 in level 1)
  • level 2 in level 1
  • level 3 in level 1
  • level 4 in level 1
This is the last section of the test
Single paragraph with '----' in it will be turned into separator:
And this is the last paragraph in the test. Be happy!
====================
Why?
We wanted a markup language with the following requirements:
  • less than 300 lines of functional code
  • easy to read
  • secure
  • support table, ul, ol, code
  • support html5 video and audio elements (html serialization only)
  • can align images and resize them
  • can specify class for tables and code elements
  • can add anchors
  • does not use _ for markup (since it creates odd behavior)
  • automatically links urls
  • fast
  • easy to extend
  • supports latex and pdf including references
  • allows to describe the markup in the markup (this document is generated from markmin syntax)
(results depend on text but in average for text ~100K markmin is 30% faster than markdown, for text ~10K it is 10x faster)
The web2py book published by lulu, for example, was entirely generated with markmin2pdf from the online web2py wiki
Download
markmin2html.py and markmin2latex.py are single files and have no web2py dependence. Their license is BSD.
Examples
Bold, italic, code and links
SOURCEOUTPUT
# titletitle
## sectionsection
### subsectionsubsection
**bold**bold
''italic''italic
~~strikeout~~strikeout
``verbatim``verbatim
``color with **bold**``:redcolor with bold
``many colors``:color[blue:#ffff00]many colors
http://google.comhttp://google.com
[[**click** me #myanchor]]click me
[[click me [extra info] #myanchor popup]]click me
More on links
The format is always [[title link]] or [[title [extra] link]]. Notice you can nest bold, italic, strikeout and code inside the link title.
Anchors 
You can place an anchor anywhere in the text using the syntax [[name]] where name is the name of the anchor. You can then link the anchor with link, i.e. [[link #myanchor]] or link with an extra info, i.e. [[link with an extra info [extra info] #myanchor]].
Images
alt-string for the image This paragraph has an image aligned to the right with a width of 200px. Its is placed using the code
[[alt-string for the image [the image title] http://www.web2py.com/examples/static/web2py_logo.png right 200px]].
Unordered Lists
- Dog
 - Cat
-- Mouse
is rendered as
  • Dog
  • Cat
  • Mouse
Two new lines between items break the list in two lists.
Ordered Lists
+ Dog
+- Mouse
is rendered as
  • Dog
  • Cat
  • Mouse
Two new lines between items break the list in two lists.
Ordered Lists
+ Dog
 + Cat
-+ Mouse
is rendered as
  1. Dog
  2. Cat
  3. Mouse
Tables
Something like this
-
---------
-**A** | **B** | **C**
-0 | 0 | X
-0 | X | 0
-X | 0 | 0
------:abc

-is a table and is rendered as
-
ABC
00X
0X0
X00

-Four or more dashes delimit the table and | separates the columns.
-The :abc at the end sets the class for the table and it is optional.
Blockquote
A table with a single cell is rendered as a blockquote:
Hello world
Code, <code>, escaping and extra stuff
def test():
-    return "this is Python code"
Optionally a ` inside a ``...`` block can be inserted escaped with !`!.
NOTE: You can escape markmin constructions ('',``,**,~~,[,{,]},$,@) with '\' character:
-so \`\` can replace !`!`! escape string
The :python after the markup is also optional. If present, by default, it is used to set the class of the <code> block.
-The behavior can be overridden by passing an argument extra to the render function. For example:
markmin2html("``aaa``:custom",
-             extra=dict(custom=lambda text: 'x'+text+'x'))
generates
'xaaax'
(the ``...``:custom block is rendered by the custom=lambda function passed to render).
Html5 support
Markmin also supports the <video> and <audio> html5 tags using the notation:
-
[[message link video]]
++ Mouse
is rendered as
  1. Dog
  2. Cat
  3. Mouse
Multilevel Lists
+ Dogs
+ -- red
+ -- brown
+ -- black
++ Cats
+ -- fluffy
+ -- smooth
+ -- bald
++ Mice
+ -- small
+ -- big
+ -- huge
is rendered as
  1. Dogs
    • red
    • brown
    • black
  2. Cats
    • fluffy
    • smooth
    • bald
  3. Mice
    • small
    • big
    • huge
Tables (with optional header and/or footer)
Something like this
-----------------
+**A**|**B**|**C**
+=================
+  0  |  0  |  X
+  0  |  X  |  0
+  X  |  0  |  0
+=================
+**D**|**F**|**G**
+-----------------:abc[id]
 is a table and is rendered as
ABC
00X
0X0
X00
DFG
 Four or more dashes delimit the table and | separates the columns. The :abc, :id[abc_1] or :abc[abc_1] at the end sets the class and/or id for the table and it is optional.
Blockquote
A table with a single cell is rendered as a blockquote:
Hello world
Blockquote can contain headers, paragraphs, lists and tables:
-----
+  This is a paragraph in a blockquote
+
+  + item 1
+  + item 2
+  -- item 2.1
+  -- item 2.2
+  + item 3
+
+  ---------
+  0 | 0 | X
+  0 | X | 0
+  X | 0 | 0
+  ---------:tableclass1
+-----
is rendered as:
This is a paragraph in a blockquote
  1. item 1
  2. item 2
    • item 2.1
    • item 2.2
  3. item 3
00X
0X0
X00
Code, <code>, escaping and extra stuff
def test():
+    return "this is Python code"
Optionally a ` inside a ``...`` block can be inserted escaped with !`!.
NOTE: You can escape markmin constructions ('',``,**,~~,[,{,]},$,@) with '\' character: so \`\` can replace !`!`! escape string
The :python after the markup is also optional. If present, by default, it is used to set the class of the <code> block. The behavior can be overridden by passing an argument extra to the render function. For example:
markmin2html("``aaa``:custom",
+             extra=dict(custom=lambda text: 'x'+text+'x'))
generates
'xaaax'
(the ``...``:custom block is rendered by the custom=lambda function passed to render).
Html5 support
Markmin also supports the <video> and <audio> html5 tags using the notation:
[[message link video]]
 [[message link audio]]
 
 [[message [title] link video]]
-[[message [title] link audio]]

-where message will be shown in brousers without HTML5 video/audio tags support.
Latex and other extensions
Formulas can be embedded into HTML with $$formula$$.
-You can use Google charts to render the formula:
LATEX = '<img src="http://chart.apis.google.com/chart?cht=tx&chl=%s" />'
-markmin2html(text,{'latex':lambda code: LATEX % code.replace('"','\"')})
Code with syntax highlighting
This requires a syntax highlighting tool, such as the web2py CODE helper.
extra={'code_cpp':lambda text: CODE(text,language='cpp').xml(),
+[[message [title] link audio]]
 where message will be shown in brousers without HTML5 video/audio tags support.
Latex and other extensions
Formulas can be embedded into HTML with $$formula$$. You can use Google charts to render the formula:
LATEX = '<img src="http://chart.apis.google.com/chart?cht=tx&chl=%s" />'
+markmin2html(text,{'latex':lambda code: LATEX % code.replace('"','\"')})
Code with syntax highlighting
This requires a syntax highlighting tool, such as the web2py CODE helper.
extra={'code_cpp':lambda text: CODE(text,language='cpp').xml(),
        'code_java':lambda text: CODE(text,language='java').xml(),
        'code_python':lambda text: CODE(text,language='python').xml(),
-       'code_html':lambda text: CODE(text,language='html').xml()}

-or simple:
-
extra={'code':lambda text,lang='': CODE(text,language=lang).xml()}

-
markmin2html(text,extra=extra)
Code can now be marked up as in this example:
-
``
+       'code_html':lambda text: CODE(text,language='html').xml()}
 or simple:
extra={'code':lambda text,lang='python': CODE(text,language=lang).xml()}
markmin2html(text,extra=extra)
Code can now be marked up as in this example:
``
 <html><body>example</body></html>
-``:code_html

-OR
-
``
+``:code_html
 OR
``
 <html><body>example</body></html>
-``:code[html]
Citations and References
Citations are treated as internal links in html and proper citations in latex if there is a final section called "References". Items like
- [[key]] value
in the References will be translated into Latex
\bibitem{key} value
Here is an example of usage:
As shown in Ref.``mdipierro``:cite
+``:code[html]
Citations and References
Citations are treated as internal links in html and proper citations in latex if there is a final section called "References". Items like
- [[key]] value
in the References will be translated into Latex
\bibitem{key} value
Here is an example of usage:
As shown in Ref.``mdipierro``:cite
 
 ## References
-- [[mdipierro]] web2py Manual, 3rd Edition, lulu.com
Caveats
<ul/>, <ol/>, <code/>, <table/>, <blockquote/>, <h1/>, ..., <h6/> do not have <p>...</p> around them.

+
+- [[mdipierro]] web2py Manual, 3rd Edition, lulu.com

Caveats

<ul/>, <ol/>, <code/>, <table/>, <blockquote/>, <h1/>, ..., <h6/> do not have <p>...</p> around them.

diff --git a/gluon/contrib/markmin/markmin2html.py b/gluon/contrib/markmin/markmin2html.py index 7ae2fd4c..1d761ae9 100755 --- a/gluon/contrib/markmin/markmin2html.py +++ b/gluon/contrib/markmin/markmin2html.py @@ -1,9 +1,11 @@ #!/usr/bin/env python +# -*- coding: utf-8 -*- # created by Massimo Di Pierro -# improved by Vladyslav Kozlovskyy +# recreated by Vladyslav Kozlovskyy # license MIT/BSD/GPL import re -import cgi +from cgi import escape +from string import maketrans """ TODO: next version should use MathJax @@ -41,11 +43,152 @@ print markmin2latex(m) from markmin2pdf import markmin2pdf # requires pdflatex print markmin2pdf(m) `` +==================== +# This is a test block with new features: + +This is a blockquote with +a list with tables in it: +----------- + This is a paragraph before list. + You can continue paragraph on the + next lines. + + This is an ordered list with tables: + + Item 1 + + Item 2 + + -------- + aa|bb|cc + 11|22|33 + --------:tableclass1[tableid1] + + Item 4 + ----------- + T1| T2| t3 + =========== + aaa|bbb|ccc + ddd|fff|ggg + 123|0 |5.0 + -----------:tableclass1 +-----------:blockquoteclass[blockquoteid] + +This this a new paragraph +with a table. Table has header and footer: +------------------------------- +**Title 1**|**Title 2**|**Title 3** +============================== +data 1 | data 2 | 2.00 +data 4 |data5(long)| 23.00 + |data 8 | 33.50 +============================== +Total: | 3 items | 58.50 +------------------------------:tableclass1[tableid2] + +## Multilevel + lists + +Now lists can be multilevel: + ++ Ordered item 1 on level 1. + You can continue item text on + next strings + +++. Ordered item 1 of sublevel 2 with + a paragraph (paragraph can start + with point after plus or minus + characters, e.g. **++.** or **--.**) + +++. This is another item. But with 3 paragraphs, + blockquote and sublists: + +.. This is the second paragraph in the item. You + can add paragraphs to an item, using point + notation, where first characters in the string + are sequence of points with space between + them and another string. For example, this + paragraph (in sublevel 2) starts with two points: + ``.. This is the second paragraph...`` + +.. ---------- + ### this is a blockquote in a list + + You can use blockquote with headers, paragraphs, + tables and lists in it: + + Tables can have or have not header and footer. + This table is defined without any header + and footer in it: + --------------------- + red |fox | 0 + blue |dolphin | 1000 + green|leaf | 10000 + --------------------- + ---------- + +.. This is yet another paragraph in the item. + +--- This is an item of unordered list **(sublevel 3)** +--- This is the second item of the unordered list ''(sublevel 3)'' + +++++++ This is a single item of ordered list in sublevel 6 +.... and this is a paragraph in sublevel 4 +---. This is a new item with paragraph in sublevel 3. +++++ Start ordered list in sublevel 4 with code block: `` +line 1 + line 2 + line 3 +`` +++++. Yet another item with code block: +`` + line 1 +line 2 + line 3 +`` +This item finishes with this paragraph. + +... Item in sublevel 3 can be continued with paragraphs. + +... `` + this is another +code block + in the + sublevel 3 item +`` + ++++ The last item in sublevel 3 +.. This is a continuous paragraph for item 2 in sublevel 2. + You can use such structure to create difficult structured + documents. + +++ item 3 in sublevel 2 +-- item 1 in sublevel 2 (new unordered list) +-- item 2 in sublevel 2 +-- item 3 in sublevel 2 + +++ item 1 in sublevel 2 (new ordered list) +++ item 2 in sublevel 2 +++ item 3 in sublevle 2 + ++ item 2 in level 1 ++ item 3 in level 1 +- new unordered list (item 1 in level 1) +- level 2 in level 1 + +- level 3 in level 1 +- level 4 in level 1 +## This is the last section of the test + +Single paragraph with '----' in it will be turned into separator: + +----------- + +And this is the last paragraph in +the test. Be happy! + +==================== ## Why? We wanted a markup language with the following requirements: -- less than 200 lines of functional code +- less than 300 lines of functional code - easy to read - secure - support table, ul, ol, code @@ -78,6 +221,7 @@ markmin2html.py and markmin2latex.py are single files and have no web2py depende ------------------------------------------------------------------------------ **SOURCE** | **OUTPUT** +============================================================================== ``# title`` | **title** ``## section`` | **section** ``### subsection`` | **subsection** @@ -138,26 +282,64 @@ is rendered as + Mouse -### Tables +### Multilevel Lists + +`` ++ Dogs + -- red + -- brown + -- black ++ Cats + -- fluffy + -- smooth + -- bald ++ Mice + -- small + -- big + -- huge +`` + +is rendered as ++ Dogs + -- red + -- brown + -- black ++ Cats + -- fluffy + -- smooth + -- bald ++ Mice + -- small + -- big + -- huge + + +### Tables (with optional header and/or footer) Something like this `` ---------- -**A** | **B** | **C** -0 | 0 | X -0 | X | 0 -X | 0 | 0 ------:abc +----------------- +**A**|**B**|**C** +================= + 0 | 0 | X + 0 | X | 0 + X | 0 | 0 +================= +**D**|**F**|**G** +-----------------:abc[id] `` is a table and is rendered as ---------- -**A** | **B** | **C** +----------------- +**A**|**B**|**C** +================= 0 | 0 | X 0 | X | 0 X | 0 | 0 ------:abc +================= +**D**|**F**|**G** +-----------------:abc[id] Four or more dashes delimit the table and | separates the columns. -The ``:abc`` at the end sets the class for the table and it is optional. +The ``:abc``, ``:id[abc_1]`` or ``:abc[abc_1]`` at the end sets the class and/or id for the table and it is optional. ### Blockquote @@ -167,6 +349,44 @@ A table with a single cell is rendered as a blockquote: Hello world ----- +Blockquote can contain headers, paragraphs, lists and tables: + +`` +----- + This is a paragraph in a blockquote + + + item 1 + + item 2 + -- item 2.1 + -- item 2.2 + + item 3 + + --------- + 0 | 0 | X + 0 | X | 0 + X | 0 | 0 + ---------:tableclass1 +----- +`` + +is rendered as: +----- + This is a paragraph in a blockquote + + + item 1 + + item 2 + -- item 2.1 + -- item 2.2 + + item 3 + + --------- + 0 | 0 | X + 0 | X | 0 + X | 0 | 0 + ---------:tableclass1 +----- + + ### Code, ````, escaping and extra stuff `` @@ -227,7 +447,7 @@ extra={'code_cpp':lambda text: CODE(text,language='cpp').xml(), `` or simple: `` -extra={'code':lambda text,lang='': CODE(text,language=lang).xml()} +extra={'code':lambda text,lang='python': CODE(text,language=lang).xml()} `` `` markmin2html(text,extra=extra) @@ -266,10 +486,12 @@ Here is an example of usage: As shown in Ref.!`!`mdipierro`!`!:cite ## References + - [[mdipierro]] web2py Manual, 3rd Edition, lulu.com `` ### Caveats + ``