+
+
+# Overview
+
+Formula recognition presents significant challenges due to the complicated structure and varied notation of mathematical expressions. Despite continuous advancements in formula recognition models, the evaluation metrics employed by these models, such as BLEU and Edit Distance, still exhibit notable limitations. They overlook the fact that the same formula has diverse representations and is highly sensitive to the distribution of training data, thereby causing the unfairness in formula recognition evaluation. To this end, we propose a Character Detection Matching (CDM) metric, ensuring the evaluation objectivity by designing a image-level rather than LaTex-level metric score. Specifically, CDM renders both the model-predicted LaTeX and the ground-truth LaTeX formulas into image-formatted formulas, then employs visual feature extraction and localization techniques for precise character-level matching, incorporating spatial position information. Such a spatially-aware and character-matching method offers a more accurate and equitable evaluation compared with previous BLEU and Edit Distance metrics that rely solely on text-based character matching.
+
+Comparison between CDM and BLEU, Edit Distance metrics:
+
+
+
+
+
+The algorithm flow of CDM is as follows:
+
+
+
+
+
+
+CDM's character matching method based on rendered images provides more intuitive results and is not affected by the diversity of formula representations.
+
+
+
+# Usage
+
+## Try Online Demo
+
+Try CDM on our online demo: [(Hugging Face)🤗](https://huggingface.co/spaces/opendatalab/CDM-Demo)
+
+## Install CMD Locally
+
+Given CDM's complex environment dependencies, we recommend trying it on Linux systems.
+
+## prepare environment
+
+Nodejs, imagemagic, pdflatex are requried packages when render pdf files and convert them to images, here are installation guides.
+
+### step.1 install nodejs
+
+```
+wget https://registry.npmmirror.com/-/binary/node/latest-v16.x/node-v16.13.1-linux-x64.tar.gz
+
+tar -xvf node-v16.13.1-linux-x64.tar.gz
+
+mv node-v16.13.1-linux-x64/* /usr/local/nodejs/
+
+ln -s /usr/local/nodejs/bin/node /usr/local/bin
+
+ln -s /usr/local/nodejs/bin/npm /usr/local/bin
+
+node -v
+```
+
+### step.2 install imagemagic
+
+the version of imagemagic installed by `apt-get` usually be 6.x, so we also install it from source code.
+
+(Before compiling, ensure that libpng-dev is installed on the system; otherwise, the compiled magick will not support CDM usage.)
+```
+git clone https://github.com/ImageMagick/ImageMagick.git ImageMagick-7.1.1
+
+cd ImageMagick-7.1.1
+
+./configure
+
+make
+
+sudo make install
+
+sudo ldconfig /usr/local/lib
+
+convert --version
+```
+
+### step.3 install latexpdf
+
+Rendering Chinese formulas requires a Chinese font, Source Han Sans SC is currently used .
+
+```
+apt-get update
+
+sudo apt-get install texlive-full
+```
+
+### step.4 install python requriements
+
+```
+pip install -r requirements.txt
+```
+
+
+## Use CDM Locally
+
+Should the installation goes well, you may now use CDM to evaluate your formula recognition results.
+
+### 1. batch evaluation
+
+- prepare input json
+
+evaluate on UniMERNet results, use this convert script to get json file:
+
+```
+python convert2cdm_format.py -i {UniMERNet predictions} -o {save path}
+```
+
+otherwise, prepare a json file follow this format:
+
+```
+[
+ {
+ "img_id": "case_1", # optional key
+ "gt": "y = 2z + 3x",
+ "pred": "y = 2x + 3z"
+ },
+ {
+ "img_id": "case_2",
+ "gt": "y = x^2 + 1",
+ "pred": "y = x^2 + 1"
+ },
+ ...
+]
+```
+
+`Note that in json files, some special characters such as "\" need escaped character, for example "\begin" should be written as "\\begin".`
+
+- evaluate:
+
+```
+python evaluation.py -i {path_to_your_input_json}
+```
+
+
+### 2. launch a gradio demo
+
+```
+python app.py
+```
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/app.py b/vlmeval/dataset/MDPBench/metrics/cdm/app.py
new file mode 100644
index 000000000..cafe3291d
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/app.py
@@ -0,0 +1,391 @@
+import json
+import os
+import shutil
+import time
+from datetime import datetime
+
+import gradio as gr
+import numpy as np
+from modules.latex2bbox_color import latex2bbox_color
+from modules.visual_matcher import HungarianMatcher, SimpleAffineTransform
+from PIL import Image, ImageDraw
+from skimage.measure import ransac
+
+DATA_ROOT = "output"
+
+
+def gen_color_list(num=10, gap=15):
+ num += 1
+ single_num = 255 // gap + 1
+ max_num = single_num ** 3
+ num = min(num, max_num)
+ color_list = []
+ for idx in range(num):
+ R = idx // single_num**2
+ GB = idx % single_num**2
+ G = GB // single_num
+ B = GB % single_num
+
+ color_list.append((R * gap, G * gap, B * gap))
+ return color_list[1:]
+
+
+def process_latex(groundtruths, predictions, user_id="test"):
+ data_root = DATA_ROOT
+ temp_dir = os.path.join(data_root, "temp_dir")
+
+ data_root = os.path.join(data_root, user_id)
+ output_dir_info = {}
+ for subset, latex_list in zip(['gt', 'pred'], [groundtruths, predictions]):
+ sub_temp_dir = os.path.join(temp_dir, f"{user_id}_{subset}")
+ os.makedirs(sub_temp_dir, exist_ok=True)
+
+ output_path = os.path.join(data_root, subset)
+ output_dir_info[output_path] = []
+
+ os.makedirs(os.path.join(output_path, 'bbox'), exist_ok=True)
+ os.makedirs(os.path.join(output_path, 'vis'), exist_ok=True)
+
+ total_color_list = gen_color_list(num=5800)
+
+ for idx, latex in enumerate(latex_list):
+ basename = f"sample_{idx}"
+ input_arg = latex, basename, output_path, sub_temp_dir, total_color_list
+ time.time()
+ latex2bbox_color(input_arg)
+ time.time()
+
+ for subset in ['gt', 'pred']:
+ shutil.rmtree(os.path.join(temp_dir, f"{user_id}_{subset}"))
+
+
+def update_inliers(ori_inliers, sub_inliers):
+ inliers = np.copy(ori_inliers)
+ sub_idx = -1
+ for idx in range(len(ori_inliers)):
+ if ori_inliers[idx] == False:
+ sub_idx += 1
+ if sub_inliers[sub_idx] == True:
+ inliers[idx] = True
+ return inliers
+
+
+def reshape_inliers(ori_inliers, sub_inliers):
+ inliers = np.copy(ori_inliers)
+ sub_idx = -1
+ for idx in range(len(ori_inliers)):
+ if ori_inliers[idx] == False:
+ sub_idx += 1
+ if sub_inliers[sub_idx] == True:
+ inliers[idx] = True
+ else:
+ inliers[idx] = False
+ return inliers
+
+
+def evaluation(user_id="test"):
+ data_root = DATA_ROOT
+ data_root = os.path.join(data_root, user_id)
+ gt_box_dir = os.path.join(data_root, "gt")
+ pred_box_dir = os.path.join(data_root, "pred")
+ match_vis_dir = os.path.join(data_root, "vis_match")
+ os.makedirs(match_vis_dir, exist_ok=True)
+
+ max_iter = 3
+ min_samples = 3
+ residual_threshold = 25
+ max_trials = 50
+
+ metrics_per_img = {}
+ gt_basename_list = [item.split(".")[0]
+ for item in os.listdir(os.path.join(gt_box_dir, 'bbox'))]
+ for basename in gt_basename_list:
+ gt_valid, pred_valid = True, True
+ if not os.path.exists(os.path.join(gt_box_dir, 'bbox', basename + ".jsonl")):
+ gt_valid = False
+ else:
+ with open(os.path.join(gt_box_dir, 'bbox', basename + ".jsonl"), 'r') as f:
+ box_gt = []
+ for line in f:
+ info = json.loads(line)
+ if info['bbox']:
+ box_gt.append(info)
+ if not box_gt:
+ gt_valid = False
+ if not gt_valid:
+ continue
+
+ if not os.path.exists(os.path.join(pred_box_dir, 'bbox', basename + ".jsonl")):
+ pred_valid = False
+ else:
+ with open(os.path.join(pred_box_dir, 'bbox', basename + ".jsonl"), 'r') as f:
+ box_pred = []
+ for line in f:
+ info = json.loads(line)
+ if info['bbox']:
+ box_pred.append(info)
+ if not box_pred:
+ pred_valid = False
+ if not pred_valid:
+ metrics_per_img[basename] = {
+ "recall": 0,
+ "precision": 0,
+ "F1_score": 0,
+ }
+ continue
+ gt_img_path = os.path.join(gt_box_dir, 'vis', basename + "_base.png")
+ pred_img_path = os.path.join(pred_box_dir, 'vis', basename + "_base.png")
+
+ img_gt = Image.open(gt_img_path)
+ img_pred = Image.open(pred_img_path)
+
+ matcher = HungarianMatcher()
+ matched_idxes = matcher(box_gt, box_pred, img_gt.size, img_pred.size)
+ src = []
+ dst = []
+ for (idx1, idx2) in matched_idxes:
+ x1min, y1min, x1max, y1max = box_gt[idx1]['bbox']
+ x2min, y2min, x2max, y2max = box_pred[idx2]['bbox']
+ x1_c, y1_c = float((x1min + x1max) / 2), float((y1min + y1max) / 2)
+ x2_c, y2_c = float((x2min + x2max) / 2), float((y2min + y2max) / 2)
+ src.append([y1_c, x1_c])
+ dst.append([y2_c, x2_c])
+
+ src = np.array(src)
+ dst = np.array(dst)
+ if src.shape[0] <= min_samples:
+ inliers = np.array([True for _ in matched_idxes])
+ else:
+ inliers = np.array([False for _ in matched_idxes])
+ for i in range(max_iter):
+ if src[inliers == False].shape[0] <= min_samples:
+ break
+ model, inliers_1 = ransac((src[inliers == False], dst[inliers == False]), SimpleAffineTransform,
+ min_samples=min_samples, residual_threshold=residual_threshold, max_trials=max_trials)
+ if inliers_1 is not None and inliers_1.any():
+ inliers = update_inliers(inliers, inliers_1)
+ else:
+ break
+ if len(inliers[inliers == True]) >= len(matched_idxes):
+ break
+
+ for idx, (a, b) in enumerate(matched_idxes):
+ if inliers[idx] == True and matcher.cost['token'][a, b] == 1:
+ inliers[idx] = False
+
+ final_match_num = len(inliers[inliers == True])
+ recall = round(final_match_num / (len(box_gt)), 3)
+ precision = round(final_match_num / (len(box_pred)), 3)
+ F1_score = round(2 * final_match_num / (len(box_gt) + len(box_pred)), 3)
+ metrics_per_img[basename] = {
+ "recall": recall,
+ "precision": precision,
+ "F1_score": F1_score,
+ }
+
+ if True:
+ gap = 5
+ W1, H1 = img_gt.size
+ W2, H2 = img_pred.size
+ H = H1 + H2 + gap
+ W = max(W1, W2)
+
+ vis_img = Image.new('RGB', (W, H), (255, 255, 255))
+ vis_img.paste(img_gt, (0, 0))
+ vis_img.paste(Image.new('RGB', (W, gap), (0, 150, 200)), (0, H1))
+ vis_img.paste(img_pred, (0, H1 + gap))
+
+ match_img = vis_img.copy()
+ match_draw = ImageDraw.Draw(match_img)
+
+ gt_matched_idx = {
+ a: flag
+ for (a, b), flag in
+ zip(matched_idxes, inliers)
+ }
+ pred_matched_idx = {
+ b: flag
+ for (a, b), flag in
+ zip(matched_idxes, inliers)
+ }
+
+ for idx, box in enumerate(box_gt):
+ if idx in gt_matched_idx and gt_matched_idx[idx] == True:
+ color = "green"
+ else:
+ color = "red"
+ x_min, y_min, x_max, y_max = box['bbox']
+ match_draw.rectangle([x_min - 1, y_min - 1, x_max + 1, y_max + 1],
+ fill=None, outline=color, width=2)
+
+ for idx, box in enumerate(box_pred):
+ if idx in pred_matched_idx and pred_matched_idx[idx] == True:
+ color = "green"
+ else:
+ color = "red"
+ x_min, y_min, x_max, y_max = box['bbox']
+ match_draw.rectangle([x_min - 1,
+ y_min - 1 + H1 + gap,
+ x_max + 1,
+ y_max + 1 + H1 + gap],
+ fill=None,
+ outline=color,
+ width=2)
+
+ vis_img.save(os.path.join(match_vis_dir, basename + "_base.png"))
+ if W < 500:
+ padding = (500 - W) // 2 + 1
+ reshape_match_img = Image.new('RGB', (500, H), (255, 255, 255))
+ reshape_match_img.paste(match_img, (padding, 0))
+ reshape_match_img.paste(Image.new('RGB', (500, gap), (0, 150, 200)), (0, H1))
+ reshape_match_img.save(os.path.join(match_vis_dir, basename + ".png"))
+ else:
+ match_img.save(os.path.join(match_vis_dir, basename + ".png"))
+
+ acc_list = [val['F1_score'] for _, val in metrics_per_img.items()]
+ metrics_res = {
+ "mean_score": round(np.mean(acc_list), 3),
+ "details": metrics_per_img
+ }
+ metric_res_path = os.path.join(data_root, "metrics_res.json")
+ with open(metric_res_path, "w") as f:
+ f.write(json.dumps(metrics_res, indent=2))
+ return metrics_res, metric_res_path, match_vis_dir
+
+
+def calculate_metric_single(groundtruth, prediction):
+ user_id = datetime.now().strftime('%Y%m%d-%H%M%S')
+ process_latex([groundtruth], [prediction], user_id)
+ metrics_res, metric_res_path, match_vis_dir = evaluation(user_id)
+ basename = "sample_0"
+ image_path = os.path.join(match_vis_dir, basename + ".png")
+ sample = metrics_res["details"][basename]
+ score = sample['F1_score']
+ recall = sample['recall']
+ precision = sample['precision']
+ return score, recall, precision, gr.Image(image_path)
+
+
+def calculate_metric_batch(json_input):
+ user_id = datetime.now().strftime('%Y%m%d-%H%M%S')
+ with open(json_input.name, "r") as f:
+ input_data = json.load(f)
+ groundtruths = []
+ predictions = []
+ for item in input_data:
+ groundtruths.append(item['gt'])
+ predictions.append(item['pred'])
+ process_latex(groundtruths, predictions, user_id)
+ metrics_res, metric_res_path, match_vis_dir = evaluation(user_id)
+ return metric_res_path
+
+
+def gradio_reset_single():
+ return gr.update(value=None, placeholder='type gt latex code here'), gr.update(value=None, placeholder='type pred latex code here'), \
+ gr.update(value=None), gr.update(value=None), gr.update(value=None), gr.update(value=None)
+
+
+def gradio_reset_batch():
+ return gr.update(value=None), gr.update(value=None)
+
+
+def select_example1():
+ gt = "y = 2x + 3z"
+ pred = "y = 2z + 3x"
+ return gr.update(value=gt, placeholder='type gt latex code here'), gr.update(
+ value=pred, placeholder='type pred latex code here')
+
+
+def select_example2():
+ gt = "r = \\frac { \\alpha } { \\beta } \\vert \\sin \\beta \\left( \\sigma _ { 1 } \\pm \\sigma _ { 2 } \\right) \\vert"
+ pred = "r={\\frac{\\alpha}{\\beta}}|\\sin\\beta\\left(\\sigma_{2}+\\sigma_{1}\\right)|"
+ return gr.update(value=gt, placeholder='type gt latex code here'), gr.update(
+ value=pred, placeholder='type pred latex code here')
+
+
+def select_example3():
+ gt = "\\begin{array} { r l r } & { } & { \\mathbf { J } _ { L } = \\left( \\begin{array} { c c } { 0 } & { 0 } \\\\ { v _ { n } } & { 0 } \\end{array} \\right) , ~ \\mathbf { J } _ { R } = \\left( \\begin{array} { c c } { u _ { n - 1 } } & { 0 } \\\\ { 0 } & { 0 } \\end{array} \\right) , ~ } \\\\ & { } & {\\mathbf { K } = \\left( \\begin{array} { c c } { V _ { n - 1 } } & { u _ { n } } \\\\ { v _ { n - 1 } } & { V _ { n } } \\end{array} \\right) , } \\end{array}"
+ pred = "\\mathbf{J}_{U}={\\left(\\begin{array}{l l}{0}&{0}\\\\ {v_{n}}&{0}\\end{array}\\right)}\\,,\\ \\mathbf{J}_{R}={\\left(\\begin{array}{l l}{u_{n-1}}&{0}\\\\ {0}&{0}\\end{array}\\right)}\\,,\\mathbf{K}={\\left(\\begin{array}{l l}{V_{n-1}}&{u_{n}}\\\\ {v_{n-1}}&{V_{n}}\\end{array}\\right)}\\,,"
+ return gr.update(value=gt, placeholder='type gt latex code here'), gr.update(
+ value=pred, placeholder='type pred latex code here')
+
+
+if __name__ == "__main__":
+ title = """
Character Detection Matching (CDM)
"""
+
+ with gr.Blocks() as demo:
+ gr.Markdown(title)
+
+ gr.Button(
+ value="Quick Try: type latex code of gt and pred, get metrics and visualization.",
+ interactive=False,
+ variant="primary")
+
+ with gr.Row():
+ with gr.Column():
+ gt_input = gr.Textbox(
+ label='gt',
+ placeholder='type gt latex code here',
+ interactive=True)
+ pred_input = gr.Textbox(
+ label='pred',
+ placeholder='type pred latex code here',
+ interactive=True)
+ with gr.Row():
+ clear_single = gr.Button("Clear")
+ submit_single = gr.Button(value="Submit", interactive=True, variant="primary")
+ with gr.Accordion("Examples:"):
+ with gr.Row():
+ example1 = gr.Button("Example A(short)")
+ example2 = gr.Button("Example B(middle)")
+ example3 = gr.Button("Example C(long)")
+ with gr.Column():
+ with gr.Row():
+ score_output = gr.Number(label="F1 Score", interactive=False)
+ recall_output = gr.Number(label="Recall", interactive=False)
+ recision_output = gr.Number(label="Precision", interactive=False)
+ gr.Button(
+ value="Visualization (green bbox means correcttlly matched, red bbox means missed or wrong.)",
+ interactive=False)
+ vis_output = gr.Image(label=" ", interactive=False)
+
+ example1.click(select_example1, inputs=None, outputs=[gt_input, pred_input])
+ example2.click(select_example2, inputs=None, outputs=[gt_input, pred_input])
+ example3.click(select_example3, inputs=None, outputs=[gt_input, pred_input])
+
+ clear_single.click(
+ gradio_reset_single,
+ inputs=None,
+ outputs=[
+ gt_input,
+ pred_input,
+ score_output,
+ recall_output,
+ recision_output,
+ vis_output])
+ submit_single.click(
+ calculate_metric_single, inputs=[
+ gt_input, pred_input], outputs=[
+ score_output, recall_output, recision_output, vis_output])
+
+ gr.Button(
+ value="Batch Run: upload a json file and batch processing, this may take some times according to your latex amount and length.",
+ interactive=False,
+ variant="primary")
+
+ with gr.Row():
+ with gr.Column():
+ json_input = gr.File(label="Input Json", file_types=[".json"])
+ json_example = gr.File(
+ label="Input Example",
+ value="assets/example/input_example.json")
+ with gr.Row():
+ clear_batch = gr.Button("Clear")
+ submit_batch = gr.Button(value="Submit", interactive=True, variant="primary")
+
+ metric_output = gr.File(label="Output Metrics")
+
+ clear_batch.click(gradio_reset_batch, inputs=None, outputs=[json_input, metric_output])
+ submit_batch.click(calculate_metric_batch, inputs=[json_input], outputs=[metric_output])
+
+ demo.launch(share=True, server_name="0.0.0.0", server_port=10005, debug=True)
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_demo.png b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_demo.png
new file mode 100644
index 000000000..1a886e2e5
Binary files /dev/null and b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_demo.png differ
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework.png b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework.png
new file mode 100644
index 000000000..0e5cb3b21
Binary files /dev/null and b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework.png differ
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework_new.png b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework_new.png
new file mode 100644
index 000000000..ec89903d4
Binary files /dev/null and b/vlmeval/dataset/MDPBench/metrics/cdm/assets/demo/cdm_framework_new.png differ
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/convert2cdm_format.py b/vlmeval/dataset/MDPBench/metrics/cdm/convert2cdm_format.py
new file mode 100644
index 000000000..9d5d6077c
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/convert2cdm_format.py
@@ -0,0 +1,37 @@
+import argparse
+import json
+import os
+
+from tqdm import tqdm
+
+
+def change_data_format(input_json, output_json):
+ with open(input_json, 'r') as f:
+ all_datas = json.load(f)
+
+ data_list = []
+
+ for key in all_datas.keys():
+ subset = key[-4:-1].lower()
+ for data in tqdm(all_datas[key]['text']):
+ im_id = os.path.basename(data['image_path'])[0:-4]
+ basename = f"{subset}_{im_id}"
+ new_item = {
+ "img_id": basename,
+ "gt": data["reference"],
+ "pred": data["prediction"]
+ }
+ data_list.append(new_item)
+
+ with open(output_json, "w") as f:
+ f.write(json.dumps(data_list, indent=2))
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--input', '-i', type=str)
+ parser.add_argument('--output', '-o', type=str)
+ args = parser.parse_args()
+ print(args)
+
+ change_data_format(args.input, args.output)
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/evaluation.py b/vlmeval/dataset/MDPBench/metrics/cdm/evaluation.py
new file mode 100644
index 000000000..60983e454
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/evaluation.py
@@ -0,0 +1,305 @@
+import argparse
+import copy
+import json
+import os
+import shutil
+import time
+from datetime import datetime
+from multiprocessing import Pool
+
+import numpy as np
+from modules.latex2bbox_color import latex2bbox_color
+from modules.visual_matcher import HungarianMatcher, SimpleAffineTransform
+from PIL import Image, ImageDraw
+from skimage.measure import ransac
+from tqdm import tqdm
+
+
+def gen_color_list(num=10, gap=15):
+ num += 1
+ single_num = 255 // gap + 1
+ max_num = single_num ** 3
+ num = min(num, max_num)
+ color_list = []
+ for idx in range(num):
+ R = idx // single_num**2
+ GB = idx % single_num**2
+ G = GB // single_num
+ B = GB % single_num
+
+ color_list.append((R * gap, G * gap, B * gap))
+ return color_list[1:]
+
+
+def update_inliers(ori_inliers, sub_inliers):
+ inliers = np.copy(ori_inliers)
+ sub_idx = -1
+ for idx in range(len(ori_inliers)):
+ if ori_inliers[idx] == False:
+ sub_idx += 1
+ if sub_inliers[sub_idx] == True:
+ inliers[idx] = True
+ return inliers
+
+
+def reshape_inliers(ori_inliers, sub_inliers):
+ inliers = np.copy(ori_inliers)
+ sub_idx = -1
+ for idx in range(len(ori_inliers)):
+ if ori_inliers[idx] == False:
+ sub_idx += 1
+ if sub_inliers[sub_idx] == True:
+ inliers[idx] = True
+ else:
+ inliers[idx] = False
+ return inliers
+
+
+def gen_token_order(box_list):
+ new_box_list = copy.deepcopy(box_list)
+ for idx, box in enumerate(new_box_list):
+ new_box_list[idx]['order'] = idx / len(new_box_list)
+ return new_box_list
+
+
+def evaluation(data_root, user_id="test"):
+ data_root = os.path.join(data_root, user_id)
+ gt_box_dir = os.path.join(data_root, "gt")
+ pred_box_dir = os.path.join(data_root, "pred")
+ match_vis_dir = os.path.join(data_root, "vis_match")
+ os.makedirs(match_vis_dir, exist_ok=True)
+
+ max_iter = 3
+ min_samples = 3
+ residual_threshold = 25
+ max_trials = 50
+
+ metrics_per_img = {}
+ gt_basename_list = [item.split(".")[0]
+ for item in os.listdir(os.path.join(gt_box_dir, 'bbox'))]
+ for basename in tqdm(gt_basename_list):
+ gt_valid, pred_valid = True, True
+ if not os.path.exists(os.path.join(gt_box_dir, 'bbox', basename + ".jsonl")):
+ gt_valid = False
+ else:
+ with open(os.path.join(gt_box_dir, 'bbox', basename + ".jsonl"), 'r') as f:
+ box_gt = []
+ for line in f:
+ info = json.loads(line)
+ if info['bbox']:
+ box_gt.append(info)
+ if not box_gt:
+ gt_valid = False
+ if not gt_valid:
+ continue
+
+ if not os.path.exists(os.path.join(pred_box_dir, 'bbox', basename + ".jsonl")):
+ pred_valid = False
+ else:
+ with open(os.path.join(pred_box_dir, 'bbox', basename + ".jsonl"), 'r') as f:
+ box_pred = []
+ for line in f:
+ info = json.loads(line)
+ if info['bbox']:
+ box_pred.append(info)
+ if not box_pred:
+ pred_valid = False
+ if not pred_valid:
+ metrics_per_img[basename] = {
+ "recall": 0,
+ "precision": 0,
+ "F1_score": 0,
+ }
+ continue
+ gt_img_path = os.path.join(gt_box_dir, 'vis', basename + "_base.png")
+ pred_img_path = os.path.join(pred_box_dir, 'vis', basename + "_base.png")
+
+ img_gt = Image.open(gt_img_path)
+ img_pred = Image.open(pred_img_path)
+
+ matcher = HungarianMatcher()
+ matched_idxes = matcher(box_gt, box_pred, img_gt.size, img_pred.size)
+ src = []
+ dst = []
+ for (idx1, idx2) in matched_idxes:
+ x1min, y1min, x1max, y1max = box_gt[idx1]['bbox']
+ x2min, y2min, x2max, y2max = box_pred[idx2]['bbox']
+ x1_c, y1_c = float((x1min + x1max) / 2), float((y1min + y1max) / 2)
+ x2_c, y2_c = float((x2min + x2max) / 2), float((y2min + y2max) / 2)
+ src.append([y1_c, x1_c])
+ dst.append([y2_c, x2_c])
+
+ src = np.array(src)
+ dst = np.array(dst)
+ if src.shape[0] <= min_samples:
+ inliers = np.array([True for _ in matched_idxes])
+ else:
+ inliers = np.array([False for _ in matched_idxes])
+ for i in range(max_iter):
+ if src[inliers == False].shape[0] <= min_samples:
+ break
+ model, inliers_1 = ransac((src[inliers == False], dst[inliers == False]), SimpleAffineTransform,
+ min_samples=min_samples, residual_threshold=residual_threshold, max_trials=max_trials)
+ if inliers_1 is not None and inliers_1.any():
+ inliers = update_inliers(inliers, inliers_1)
+ else:
+ break
+ if len(inliers[inliers == True]) >= len(matched_idxes):
+ break
+
+ for idx, (a, b) in enumerate(matched_idxes):
+ if inliers[idx] == True and matcher.cost['token'][a, b] == 1:
+ inliers[idx] = False
+
+ final_match_num = len(inliers[inliers == True])
+ recall = round(final_match_num / (len(box_gt)), 3)
+ precision = round(final_match_num / (len(box_pred)), 3)
+ F1_score = round(2 * final_match_num / (len(box_gt) + len(box_pred)), 3)
+ metrics_per_img[basename] = {
+ "recall": recall,
+ "precision": precision,
+ "F1_score": F1_score,
+ }
+
+ if True:
+ gap = 5
+ W1, H1 = img_gt.size
+ W2, H2 = img_pred.size
+ H = H1 + H2 + gap
+ W = max(W1, W2)
+
+ vis_img = Image.new('RGB', (W, H), (255, 255, 255))
+ vis_img.paste(img_gt, (0, 0))
+ vis_img.paste(Image.new('RGB', (W, gap), (120, 120, 120)), (0, H1))
+ vis_img.paste(img_pred, (0, H1 + gap))
+
+ match_img = vis_img.copy()
+ match_draw = ImageDraw.Draw(match_img)
+
+ gt_matched_idx = {
+ a: flag
+ for (a, b), flag in
+ zip(matched_idxes, inliers)
+ }
+ pred_matched_idx = {
+ b: flag
+ for (a, b), flag in
+ zip(matched_idxes, inliers)
+ }
+
+ for idx, box in enumerate(box_gt):
+ if idx in gt_matched_idx and gt_matched_idx[idx] == True:
+ color = "green"
+ else:
+ color = "red"
+ x_min, y_min, x_max, y_max = box['bbox']
+ match_draw.rectangle([x_min - 1, y_min - 1, x_max + 1, y_max + 1],
+ fill=None, outline=color, width=2)
+
+ for idx, box in enumerate(box_pred):
+ if idx in pred_matched_idx and pred_matched_idx[idx] == True:
+ color = "green"
+ else:
+ color = "red"
+ x_min, y_min, x_max, y_max = box['bbox']
+ match_draw.rectangle([x_min - 1,
+ y_min - 1 + H1 + gap,
+ x_max + 1,
+ y_max + 1 + H1 + gap],
+ fill=None,
+ outline=color,
+ width=2)
+
+ vis_img.save(os.path.join(match_vis_dir, basename + "_base.png"))
+ match_img.save(os.path.join(match_vis_dir, basename + ".png"))
+
+ score_list = [val['F1_score'] for _, val in metrics_per_img.items()]
+ exp_list = [1 if score == 1 else 0 for score in score_list]
+ metrics_res = {
+ "mean_score": round(np.mean(score_list), 3),
+ "exp_rate": round(np.mean(exp_list), 3),
+ "details": metrics_per_img
+ }
+ metric_res_path = os.path.join(data_root, "metrics_res.json")
+ with open(metric_res_path, "w") as f:
+ f.write(json.dumps(metrics_res, indent=2))
+ return metrics_res, metric_res_path, match_vis_dir
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--input', '-i', type=str, default="assets/example/input_example.json")
+ parser.add_argument('--output', '-o', type=str, default="output")
+ parser.add_argument('--pools', '-p', type=int, default=240)
+ args = parser.parse_args()
+ print(args)
+
+ json_input, data_root, pool_num = args.input, args.output, args.pools
+ temp_dir = os.path.join(data_root, "temp_dir")
+ exp_name = os.path.basename(json_input).split('.')[0]
+ with open(json_input, "r") as f:
+ input_data = json.load(f)
+ img_ids = []
+ groundtruths = []
+ predictions = []
+ for idx, item in enumerate(input_data):
+ if "img_id" in item:
+ img_ids.append(item["img_id"])
+ else:
+ img_ids.append(f"sample_{idx}")
+ groundtruths.append(item['gt'])
+ predictions.append(item['pred'])
+
+ a = time.time()
+ user_id = exp_name
+
+ total_color_list = gen_color_list(num=5800)
+
+ data_root = os.path.join(data_root, user_id)
+ output_dir_info = {}
+ input_args = []
+ for subset, latex_list in zip(['gt', 'pred'], [groundtruths, predictions]):
+ sub_temp_dir = os.path.join(temp_dir, f"{exp_name}_{subset}")
+ os.makedirs(sub_temp_dir, exist_ok=True)
+ output_path = os.path.join(data_root, subset)
+ output_dir_info[output_path] = []
+
+ os.makedirs(os.path.join(output_path, 'bbox'), exist_ok=True)
+ os.makedirs(os.path.join(output_path, 'vis'), exist_ok=True)
+
+ for idx, latex in tqdm(enumerate(latex_list), desc=f"collect {subset} latex ..."):
+ basename = img_ids[idx]
+ input_arg = latex, basename, output_path, sub_temp_dir, total_color_list
+ input_args.append(input_arg)
+
+ if pool_num > 1:
+ print(
+ datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+ "using processpool, pool num:",
+ pool_num,
+ ", job num:",
+ len(input_args))
+ myP = Pool(args.pools)
+ for input_arg in input_args:
+ myP.apply_async(latex2bbox_color, args=(input_arg,))
+ myP.close()
+ myP.join()
+ else:
+ for input_arg in input_args:
+ latex2bbox_color(input_arg)
+ b = time.time()
+ print(datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+ "extract bbox done, time cost:", round(b - a, 3), "s")
+
+ for subset in ['gt', 'pred']:
+ shutil.rmtree(os.path.join(temp_dir, f"{exp_name}_{subset}"))
+
+ c = time.time()
+ metrics_res, metric_res_path, match_vis_dir = evaluation(args.output, exp_name)
+ d = time.time()
+ print(datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+ "calculate metrics done, time cost:", round(d - c, 3), "s")
+
+ print(f"=> process done, mean f1 score: {metrics_res['mean_score']}.")
+ print(f"=> more details of metrics are saved in `{metric_res_path}`")
+ print(f"=> visulization images are saved under `{match_vis_dir}`")
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex2bbox_color.py b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex2bbox_color.py
new file mode 100644
index 000000000..0e2863f16
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex2bbox_color.py
@@ -0,0 +1,245 @@
+import json
+import logging
+import os
+import subprocess
+from threading import Timer
+
+import numpy as np
+from PIL import Image, ImageDraw
+
+from .latex_processor import normalize_latex, token_add_color_RGB
+from .tokenize_latex.tokenize_latex import tokenize_latex
+
+tabular_template = r"""
+\documentclass[12pt]{article}
+\usepackage[landscape]{geometry}
+\usepackage{geometry}
+\geometry{apaper,scale=0.98}
+\pagestyle{empty}
+\usepackage{booktabs}
+\usepackage{multirow}
+\usepackage{amssymb}
+\usepackage{upgreek}
+\usepackage{amsmath}
+\usepackage{xcolor}
+\begin{document}
+\makeatletter
+\renewcommand*{\@textcolor}[3]{%%
+ \protect\leavevmode
+ \begingroup
+ \color#1{#2}#3%%
+ \endgroup
+}
+\makeatother
+\begin{displaymath}
+%s
+\end{displaymath}
+\end{document}
+"""
+
+# 需要配置Source Han Sans SC或其他中文字体
+formular_template = r"""
+\documentclass[12pt]{article}
+\usepackage[landscape]{geometry}
+\usepackage{geometry}
+\geometry{apaper,scale=0.98}
+\pagestyle{empty}
+\usepackage{amsmath}
+\usepackage{upgreek}
+\usepackage{amssymb}
+\usepackage{xcolor}
+\usepackage{xeCJK}
+\setCJKmainfont{Source Han Sans SC}
+\setCJKsansfont{Source Han Sans SC}
+\setCJKmonofont{Source Han Sans SC}
+\xeCJKsetup{CJKmath=true}
+\begin{document}
+\makeatletter
+\renewcommand*{\@textcolor}[3]{%%
+ \protect\leavevmode
+ \begingroup
+ \color#1{#2}#3%%
+ \endgroup
+}
+\makeatother
+\begin{displaymath}
+%s
+\end{displaymath}
+\end{document}
+"""
+
+
+def run_cmd(cmd, timeout_sec=30, temp_dir=None):
+ # 设置进程独立的环境变量
+ env = os.environ.copy()
+ if temp_dir:
+ env['TMPDIR'] = temp_dir
+ env['TMP'] = temp_dir
+ env['TEMP'] = temp_dir
+ env['MAGICK_TMPDIR'] = temp_dir
+ env['TEXMFCACHE'] = temp_dir
+ env['TEXMFVAR'] = temp_dir
+
+ proc = subprocess.Popen(cmd, shell=True, env=env)
+ def kill_proc(p): return p.kill()
+ timer = Timer(timeout_sec, kill_proc, [proc])
+ try:
+ timer.start()
+ stdout, stderr = proc.communicate()
+ finally:
+ timer.cancel()
+
+
+def convert_pdf2img(pdf_filename, png_filename, temp_dir=None):
+ cmd = "magick -density 200 -quality 100 \"%s\" \"%s\"" % (pdf_filename, png_filename)
+ run_cmd(cmd, temp_dir=temp_dir)
+
+
+def crop_image(image_path, pad=8):
+ img = Image.open(image_path).convert("L")
+ img_data = np.asarray(img, dtype=np.uint8)
+ nnz_inds = np.where(img_data != 255)
+ if len(nnz_inds[0]) == 0:
+ y_min = 0
+ y_max = 10
+ x_min = 0
+ x_max = 10
+ else:
+ y_min = np.min(nnz_inds[0])
+ y_max = np.max(nnz_inds[0])
+ x_min = np.min(nnz_inds[1])
+ x_max = np.max(nnz_inds[1])
+
+ img = Image.open(image_path).convert("RGB").crop(
+ (x_min - pad, y_min - pad, x_max + pad, y_max + pad))
+ img.save(image_path)
+
+
+def extrac_bbox_from_color_image(image_path, color_list):
+ img = Image.open(image_path).convert("RGB")
+ W, H = img.size
+ pixels = list(img.getdata())
+
+ bbox_list = []
+ for target_color in color_list:
+ target_pixels = [i for i, pixel in enumerate(pixels)if pixel == target_color]
+ x_list = []
+ y_list = []
+ for idx in target_pixels:
+ x_list.append(idx % W)
+ y_list.append(idx // W)
+ try:
+ y_min, y_max, x_min, x_max = min(y_list), max(y_list), min(x_list), max(x_list)
+ bbox_list.append([x_min - 1, y_min - 1, x_max + 1, y_max + 1])
+
+ except BaseException:
+ bbox_list.append([])
+ continue
+
+ img = img.convert("L")
+ img_bw = img.point(lambda x: 255 if x == 255 else 0, '1')
+ img_bw.convert("RGB").save(image_path)
+ return bbox_list
+
+
+def latex2bbox_color(input_arg):
+ latex, basename, output_path, temp_dir, total_color_list = input_arg
+ tabular_template if "tabular" in latex else formular_template
+ basename = basename.replace('.jpg', '') # *****
+ output_bbox_path = os.path.join(output_path, 'bbox', basename + '.jsonl')
+ output_vis_path = os.path.join(output_path, 'vis', basename + '.png')
+ output_base_path = os.path.join(output_path, 'vis', basename + '_base.png')
+
+ if os.path.exists(output_bbox_path) and os.path.exists(
+ output_vis_path) and os.path.exists(output_base_path):
+ return
+
+ try:
+ latex = latex.replace("\n", " ")
+ latex = latex.replace("\\%", "")
+ ret, new_latex = tokenize_latex(
+ latex, middle_file=os.path.join(
+ temp_dir, basename + '.txt'))
+ if not (ret and new_latex):
+ log = f"ERROR, Tokenize latex failed: {basename}."
+ logging.info(log)
+ new_latex = latex
+ new_latex = new_latex.replace("< P E R C E N T A G E T O K E N >", "\\%")
+ latex = normalize_latex(new_latex)
+ token_list = []
+ l_split = latex.strip().split(' ')
+ color_list = total_color_list[0:len(l_split)]
+ idx = 0
+ while idx < len(l_split):
+ l_split, idx, token_list = token_add_color_RGB(l_split, idx, token_list)
+
+ rgb_latex = " ".join(l_split)
+ for idx, color in enumerate(color_list):
+ R, G, B = color
+ rgb_latex = rgb_latex.replace(f"", f"{R},{G},{B}")
+
+ if len(token_list) > 1300:
+ paper_size = 3
+ elif len(token_list) > 600:
+ paper_size = 4
+ else:
+ paper_size = 5
+ final_latex = formular_template.replace("", str(paper_size)) % rgb_latex
+
+ except Exception as e:
+ log = f"ERROR, Preprocess latex failed: {basename}; {e}."
+ logging.info(log)
+ return
+
+ pre_name = output_path.replace('/', '_').replace('.', '_') + '_' + basename
+ tex_filename = os.path.join(temp_dir, pre_name + '.tex')
+ log_filename = os.path.join(temp_dir, pre_name + '.log')
+ aux_filename = os.path.join(temp_dir, pre_name + '.aux')
+
+ # print(os.path.exists(tex_filename), tex_filename)
+ with open(tex_filename, "w") as w:
+ # print(final_latex, file=w)
+ w.write(final_latex)
+ # print(os.path.exists(tex_filename), tex_filename)
+ # run_cmd(f"pdflatex -interaction=nonstopmode -output-directory={temp_dir} {tex_filename} >/dev/null")
+ run_cmd(
+ f"xelatex -interaction=nonstopmode -output-directory={temp_dir} \"{tex_filename}\" >/dev/null",
+ temp_dir=temp_dir)
+ try:
+ os.remove(tex_filename)
+ os.remove(log_filename)
+ os.remove(aux_filename)
+ except BaseException:
+ pass
+ pdf_filename = tex_filename[:-4] + '.pdf'
+ if not os.path.exists(pdf_filename):
+ log = f"ERROR, Compile pdf failed: {pdf_filename}"
+ logging.info(log)
+ else:
+ convert_pdf2img(pdf_filename, output_base_path)
+ os.remove(pdf_filename)
+
+ crop_image(output_base_path)
+ bbox_list = extrac_bbox_from_color_image(output_base_path, color_list)
+ vis = Image.open(output_base_path)
+ draw = ImageDraw.Draw(vis)
+
+ with open(output_bbox_path, 'w', encoding='utf-8') as f:
+ for token, box in zip(token_list, bbox_list):
+ item = {
+ "bbox": box,
+ "token": token
+ }
+ f.write(json.dumps(item, ensure_ascii=False) + '\n')
+
+ if not box:
+ continue
+ x_min, y_min, x_max, y_max = box
+ draw.rectangle([x_min, y_min, x_max, y_max],
+ fill=None, outline=(0, 250, 0), width=1)
+ try:
+ draw.text((x_min, y_min), token, (250, 0, 0))
+ except BaseException:
+ pass
+
+ vis.save(output_vis_path)
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_processor.py b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_processor.py
new file mode 100644
index 000000000..d6fe47446
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_processor.py
@@ -0,0 +1,565 @@
+import re
+
+SKIP_PATTERNS = [r'\{', r'\}', r'[\[\]]', r'\\begin\{.*?\}', r'\\end\{.*?\}',
+ r'\^', r'\_', r'\\.*rule.*', r'\\.*line.*', r'\[[\-.0-9]+[epm][xtm]\]']
+SKIP_Tokens = ['\\', '\\\\', '\\index', '\\a', '&', '$', '\\multirow', '\\def', '\\raggedright', '\\url', '\\cr', '\\ensuremath', '\\left', '\\right',
+ '\\mathchoice', '\\scriptstyle', '\\displaystyle', '\\qquad', '\\quad', '\\,', '\\!', '~', '\\boldmath']
+PHANTOM_Tokens = ['\\fontfamily', '\\vphantom', '\\phantom', '\\rowcolor', '\\ref']
+TWO_Tail_Tokens = ['\\frac', '\\binom']
+AB_Tail_Tokens = ['\\xrightarrow', '\\xleftarrow', '\\sqrt'] # special token \xxx [] {}
+TWO_Tail_Invisb_Tokens = ['\\overset', '\\underset', '\\stackrel']
+ONE_Tail_Tokens = ['\\widetilde', '\\overline', '\\hat', '\\widehat', '\\tilde', '\\Tilde', '\\dot', '\\bar', '\\vec', '\\underline', '\\underbrace', '\\check',
+ '\\breve', '\\Bar', '\\Vec', '\\mathring', '\\ddot']
+ONE_Tail_Invisb_Tokens = ['\\boldsymbol', '\\pmb', '\\textbf', '\\mathrm', '\\mathbf', '\\mathbb', '\\mathcal', '\\textmd', '\\texttt', '\\textnormal',
+ '\\text', '\\textit', '\\textup', '\\mathop', '\\mathbin', '\\smash', '\\operatorname', '\\textrm', '\\mathfrak', '\\emph',
+ '\\textsf', '\\textsc']
+
+
+def flatten_multiline(latex):
+ brace_map = {
+ "\\left(": "\\right)",
+ "\\left[": "\\right]",
+ "\\left{": "\\right}",
+ }
+ l_split = latex.split(' ')
+ if l_split[0] == "\\begin{array}":
+ if l_split[-1] == "\\end{array}":
+ l_split = l_split[2:-1]
+ else:
+ l_split = l_split[2:]
+
+ idx = 0
+ while idx < len(l_split):
+ token = l_split[idx]
+ if token.startswith("\\left") and token in brace_map.keys():
+ end_idx = find_matching_brace(l_split, idx, brace=[token, brace_map[token]])
+ if end_idx != -1:
+ idx = end_idx
+ elif token in ["\\\\", "~", "\\qquad"]:
+ l_split = l_split[0:idx] + l_split[idx + 1:]
+ idx -= 1
+ idx += 1
+ latex = ' '.join(l_split)
+ return "$ " + latex + " $"
+
+
+def clean_latex(text):
+ # TODO 让GPT写的去空格函数, 初步测了是没问题的, 不确定是否完全没有bug
+ cleaned_text = re.sub(r'(?<=[^\\])\s+(?=[^\\])', '', text)
+ # TODO 有一些不能去掉的空格给补充回来
+ for item in ["\\hline", "\\midrule", "\\times", "\\bf", "\\footnotesize", "\\cr", '\\log']:
+ cleaned_text = cleaned_text.replace(item, item + " ")
+ cleaned_text = cleaned_text.replace(" \\mathcolor{black}", "\\mathcolor{black}")
+ return cleaned_text
+
+
+def remove_trailing_latex(formula):
+ pattern = r'(\\(hspace\*?\{[^{}]*?\}|vspace\*?\{[^{}]*?\}|smallskip|medskip|quad|qquad|bigskip|[;,])|\~|\.)*$'
+ # Replace the matched pattern with an empty string
+ cleaned_formula = re.sub(pattern, '', formula, count=1)
+ return cleaned_formula
+
+
+def find_matching_brace(sequence, start_index, brace=['{', '}']):
+ # Finds the index of the matching brace for the one at start_index
+ left_brace, right_brace = brace
+ depth = 0
+ for i, char in enumerate(sequence[start_index:], start=start_index):
+ if char == left_brace:
+ depth += 1
+ elif char == right_brace:
+ depth -= 1
+ if depth == 0:
+ return i
+ if depth > 0:
+ error_info = "Warning! found no matching brace in sequence !"
+ raise ValueError(error_info)
+ return -1
+
+
+def normalize_latex(l, rm_trail=False):
+ if "tabular" in l:
+ latex_type = "tabular"
+ else:
+ latex_type = "formula"
+
+ if rm_trail:
+ l = remove_trailing_latex(l)
+ l = l.strip().replace(r'\pmatrix', r'\mypmatrix').replace(r'\matrix', r'\mymatrix')
+
+ # TODO \raggedright \arraybackslash, these align method, difficult to handle, remove it.
+ for item in ['\\raggedright', '\\arraybackslash']:
+ l = l.replace(item, "")
+
+ for item in ['\\lowercase', '\\uppercase']:
+ l = l.replace(item, "")
+
+ # TODO \hspace {1 . 5 cm}, for formula, change to \hspace{1.5cm}, for table, remove it.
+ pattern = r'\\[hv]space { [.0-9a-z ]+ }'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ if latex_type == "tabular":
+ new_token = ["" for item in old_token]
+ else:
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # TODO take \begin {tabular} {} as one token
+ # TODO there are \begin{array} in table too，so the process should run in
+ # both formula and table.
+ if latex_type == "tabular":
+ l = l.replace("\\begin {tabular}", "\\begin{tabular}")
+ l = l.replace("\\end {tabular}", "\\end{tabular}")
+ l = l.replace("\\begin {array}", "\\begin{array}")
+ l = l.replace("\\end {array}", "\\end{array}")
+ l_split = l.split(' ')
+ idx = 0
+ while idx < len(l_split):
+ token = l_split[idx]
+ if token == "\\begin{tabular}":
+ sub_idx = idx + 1
+ end_idx = find_matching_brace(l_split, sub_idx)
+ new_token = "".join(l_split[idx: end_idx + 1])
+ l_split = l_split[0:idx] + [new_token] + l_split[end_idx + 1:]
+ break
+ idx += 1
+ l = ' '.join(l_split)
+
+ # TODO some complex format, hart to deal with re.match, so using brace
+ # match, such as：\cmidrule ( l { 3 p t } r { 3 p t } ) { 1 - 1 }
+ l_split = l.split(' ')
+ idx = 0
+ while idx < len(l_split):
+ token = l_split[idx]
+ if token in ["\\cmidrule", "\\cline"]:
+ sub_idx = idx + 1
+ if l_split[sub_idx] == "(":
+ mid_end = find_matching_brace(l_split, sub_idx, brace=['(', ')'])
+ end_idx = find_matching_brace(l_split, mid_end + 1)
+ else:
+ end_idx = find_matching_brace(l_split, sub_idx)
+ new_token = "".join(l_split[idx: end_idx + 1])
+ l_split = l_split[0:idx] + [new_token] + l_split[end_idx + 1:]
+ idx += 1
+ l = ' '.join(l_split)
+
+ pattern = r'\\begin{array} { [lrc ]+ }'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [
+ item.replace(
+ "\\begin{array} ",
+ "").replace(
+ " ",
+ "").replace(
+ "",
+ "\\begin{array} ") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # TODO token such \not= should be one token
+ pattern = r'\\not [<>+=\-]'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # TODO tokens such as \dots \exp \sinh, split them to parts, so the bbox match will be easier.
+
+ l = " " + l + " "
+ l = l.replace(" \\ldots ", " . . . ")
+ l = l.replace(" \\cdots ", " . . . ")
+ l = l.replace(" \\dots ", " . . . ")
+ l = l.replace(" \\dotsb ", " . . . ")
+ l = l.replace(" \\log ", " \\mathrm { l o g } ")
+ l = l.replace(" \\exp ", " \\mathrm { e x p } ")
+ l = l.replace(" \\sin ", " \\mathrm { s i n } ")
+ l = l.replace(" \\cos ", " \\mathrm { c o s } ")
+ l = l.replace(" \\tan ", " \\mathrm { t a n } ")
+ l = l.replace(" \\tanh ", " \\mathrm { t a n h } ")
+ l = l.replace(" \\cosh ", " \\mathrm { c o s h } ")
+ l = l.replace(" \\sinh ", " \\mathrm { s i n h } ")
+
+ # ** token such as \big( should be one token
+ pattern = r'\\[Bb]ig[g]?[glrm]? [(){}|\[\]] '
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft + " ")
+
+ pattern = r'\\[Bb]ig[g]?[glrm]? \\.*? '
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft + " ")
+
+ # TODO when \operatorname * meets mathcolor it comes error, yet the * is
+ # useless, so we simply remove it bynow.
+ pattern = r'\\operatorname \*'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = ["\\operatorname" for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # TODO \lefteqn will lead to letter overlap, it's harmfull for render, so simply remove it.
+ l = l.replace("\\lefteqn", "")
+
+ # TODO \footnote can not seem as ONE_Tail_Invisb_Tokens(usually this type
+ # token add color by \mathrm {\color(x)}, yet \footnode should be
+ # \color{\footnote{x}}), so we simple change it to "^".
+ l = l.replace("\\footnote ", "^ ")
+
+ # TODO \' can not be rendered separately(cause to different visulize
+ # performence), so we take these tokens as one token such as \' e -> \'e,
+ # on the other hand, if { after \' then render them separately.
+ pattern = r'\\\' [^{] '
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft + " ")
+
+ # TODO [ -1.5ex ] [ 1.5pt ] [ 3 mm ] some layout adjustment, no need to
+ # render. combine them as one token.
+ if latex_type == "tabular":
+ pattern = r'\[ [\-.0-9 ]+[exptcm ]+ \]'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # ** \parbox { 3cm } {} shoudle be combined as one token
+ pattern = r'\\parbox {[^{]+}'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # ** \raisebox{}[][] {} shoudle be combined as one token, \raisebox{-1.5ex}[0pt]
+ pattern = r'\\raisebox {[^{]+} [\[\]0-9 exptcm]+{'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft[0:-1] + " {")
+
+ # ** \char shoudle be combined as one token
+ pattern = r'{ \\char[0-9\' ]+}'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, "{ " + aft[1:-1] + " }")
+
+ # ** \not xx shoudle be combined as one token
+ pattern = r'\\not [\\=\<\>][^ ]+ '
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft + " ")
+
+ # ** \specialrule{1pt}{2pt}{2pt}, special lines, shoudle be combined as one token
+ pattern = r'\\specialrule {[ .0-9a-z]+} {[ .0-9a-z]+} {[ .0-9a-z]+}'
+ old_token = re.findall(pattern, l, re.DOTALL)
+ new_token = [item.replace(" ", "") for item in old_token]
+ for bef, aft in zip(old_token, new_token):
+ l = l.replace(bef, aft)
+
+ # ** for easier add color, the original color should be removed, there are two type of color for now: \color[rgb]{0, 1, 0} and \color{red}
+ pattern = r'\\colorbox[ \[\]RGBrgb]+{ [A-Za-z 0-9,!]+ } |\\color[ \[\]RGBrgb]+{ [A-Za-z 0-9,!]+ } |\\textcolor[ \[\]RGBrgb]+{ [A-Za-z 0-9,!]+ } |\\cellcolor[ \[\]RGBrgb]+{ [A-Za-z 0-9,!]+ } '
+ old_token = re.findall(pattern, l, re.DOTALL)
+ for bef in old_token:
+ l = l.replace(bef, "")
+
+ # ** filling the missing brace [] and {} according to token.
+ l_split = l.split(' ')
+ idx = 0
+ while idx < len(l_split):
+ token = l_split[idx]
+ if token in ONE_Tail_Tokens + ONE_Tail_Invisb_Tokens:
+ # ** normalize tokens such as \hat, fill missing the {}, such as \hat \lambda -> \hat {\lambda}
+ sub_idx = idx + 1
+ while sub_idx < len(
+ l_split) and l_split[sub_idx] in ONE_Tail_Tokens + ONE_Tail_Invisb_Tokens:
+ sub_idx += 1
+ new_split = l_split[0:idx]
+ for ii in range(idx, sub_idx):
+ new_split = new_split + [l_split[ii], "{"]
+ if l_split[sub_idx] != "{":
+ new_split = new_split + [l_split[sub_idx]] + ["}"] * (sub_idx - idx)
+ l_split = new_split + l_split[sub_idx + 1:]
+ else:
+ end_idx = find_matching_brace(l_split, sub_idx)
+ new_split = new_split + l_split[sub_idx + 1:end_idx] + ["}"] * (sub_idx - idx)
+ l_split = new_split + l_split[end_idx + 1:]
+ elif token in AB_Tail_Tokens:
+ # ** normalize special tokens such as \sqrt, fill the missing [] {} in \sqrt [] {}, yet the [] is optional, for example: \sqrt A B -> \sqrt {A} B and \sqrt [A] B -> \sqrt [A] {B}
+ if l_split[idx + 1] != "[" and l_split[idx + 1] != "{":
+ l_split = l_split[0:idx + 1] + \
+ ["{"] + [l_split[idx + 1]] + ["}"] + l_split[idx + 2:]
+ else:
+ if l_split[idx + 1] == "[":
+ end1 = find_matching_brace(l_split, idx + 1, brace=['[', ']'])
+ else:
+ end1 = idx
+ if l_split[end1 + 1] != "{":
+ l_split = l_split[0:end1 + 1] + \
+ ["{"] + [l_split[end1 + 1]] + ["}"] + l_split[end1 + 2:]
+ elif token in TWO_Tail_Tokens + TWO_Tail_Invisb_Tokens:
+ # ** normalize special tokens such as \frac, add missing brace in \frac {A} {B} for example: \frac {\lambda} 2 -> \frac {\lambda} {2}
+ if l_split[idx + 1] != "{":
+ l_split = l_split[0:idx + 1] + \
+ ["{"] + [l_split[idx + 1]] + ["}"] + l_split[idx + 2:]
+ end1 = find_matching_brace(l_split, idx + 1)
+ if l_split[end1 + 1] != "{":
+ l_split = l_split[0:end1 + 1] + \
+ ["{"] + [l_split[end1 + 1]] + ["}"] + l_split[end1 + 2:]
+
+ idx += 1
+ l = ' '.join(l_split)
+
+ return l
+
+
+def token_add_color(l_split, idx, render_dict):
+ token = l_split[idx]
+ if token in PHANTOM_Tokens:
+ # ** special tokens that do not need render, skip it
+ if l_split[idx + 1] == '{':
+ brace_end = find_matching_brace(l_split, idx + 1)
+ else:
+ brace_end = idx + 1
+ next_idx = brace_end + 1
+ elif token in TWO_Tail_Tokens:
+ # ** tokens such as \frac A B, and the token needs render too.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ den_start = num_end + 1
+ den_end = find_matching_brace(l_split, den_start)
+ l_split_copy = l_split[:idx] + [r'\mathcolor{black}{' + token + '{'] + \
+ [r'\mathcolor{gray}{'] + l_split[num_start + 1:num_end] + \
+ ['}'] + [r'}{'] + [r'\mathcolor{gray}{'] + l_split[den_start + 1:den_end] + \
+ ['}'] + ['}'] + ['}'] + l_split[den_end + 1:]
+
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ next_idx = idx + 1
+ elif token in ONE_Tail_Tokens:
+ # ** tokens such as \hat A, and the token needs render too.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ l_split_copy = l_split[:idx] + [r'\mathcolor{black}{'] + l_split[idx: num_start + 1] + \
+ [r'\mathcolor{gray}{'] + l_split[num_start + 1: num_end] + \
+ ['}'] + l_split[num_end: num_end + 1] + ['}'] + l_split[num_end + 1:]
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ next_idx = idx + 1
+ elif token in ONE_Tail_Invisb_Tokens:
+ # ** tokens such as \text A B, and the token does not need render.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ sub_idx = num_start + 1
+ if num_end - num_start == 2:
+ l_split_copy = l_split.copy()
+ l_split_copy[sub_idx] = r'{\mathcolor{black}{' + l_split_copy[sub_idx] + '}}'
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, l_split[sub_idx]
+ next_idx = num_end
+ else:
+ while sub_idx < num_end:
+ l_split, sub_idx, render_dict = token_add_color(l_split, sub_idx, render_dict)
+ next_idx = num_end + 1
+ elif token in AB_Tail_Tokens:
+ # ** special token \xrightarrow, could be \xrightarrow [] {} or \xrightarrow {}, process method are different.
+ if l_split[idx + 1] == '{':
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ l_split_copy = l_split[:idx] + [r'\mathcolor{black}{'] + l_split[idx: idx + 2] \
+ + [r'\mathcolor{gray}{'] + l_split[num_start
+ + 1: num_end] + ['}}'] + l_split[num_end:]
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ sub_idx = num_start + 1
+ while sub_idx < num_end:
+ l_split, sub_idx, render_dict = token_add_color(l_split, sub_idx, render_dict)
+ next_idx = num_end + 1
+ elif l_split[idx + 1] == '[':
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start, brace=['[', ']'])
+ den_start = num_end + 1
+ den_end = find_matching_brace(l_split, den_start)
+ l_split_copy = l_split[:idx] + [r'{\mathcolor{black}{'] + l_split[idx: idx + 2] \
+ + [r'\mathcolor{gray}{'] + l_split[idx + 2: num_end] + ['}'] + l_split[num_end:den_start + 1] \
+ + [r'\mathcolor{gray}{'] + l_split[den_start + 1: den_end] + ['}'] + l_split[den_end: den_end + 1] \
+ + ['}}'] + l_split[den_end + 1:]
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ sub_idx = num_start + 1
+ while sub_idx < num_end:
+ l_split, sub_idx, render_dict = token_add_color(l_split, sub_idx, render_dict)
+ sub_idx = den_start + 1
+ while sub_idx < den_end:
+ l_split, sub_idx, render_dict = token_add_color(l_split, sub_idx, render_dict)
+ next_idx = den_end + 1
+ elif token in ["\\multicolumn", "\\multirow"]:
+ # ** tokens with three {}, such as \multicolumn {} {} {}, the text in third {} need be rendered.
+ first_start = idx + 1
+ first_end = find_matching_brace(l_split, first_start)
+ second_start = first_end + 1
+ second_end = find_matching_brace(l_split, second_start)
+ third_start = second_end + 1
+ third_end = find_matching_brace(l_split, third_start)
+
+ sub_idx = third_start + 1
+ while sub_idx < third_end:
+ l_split, sub_idx, render_dict = token_add_color(l_split, sub_idx, render_dict)
+ next_idx = third_end + 1
+ elif token in SKIP_Tokens + TWO_Tail_Invisb_Tokens or any(re.match(pattern, token) for pattern in SKIP_PATTERNS):
+ # ** tokens no need render, just skip
+ # print('skip', idx, token)
+ # TODO special case :[], could be single, or in \sqrt[]{}.
+ if (token == "[" and l_split[idx - 1] != "\\sqrt") or (token
+ == "]" and idx >= 3 and l_split[idx - 3] != "\\sqrt"):
+ l_split_copy = l_split.copy()
+ l_split_copy[idx] = r'\mathcolor{black}{ ' + l_split_copy[idx] + ' }'
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ next_idx = idx + 1
+ else:
+ next_idx = idx + 1
+ else:
+ # ** nomal token
+ l_split_copy = l_split.copy()
+ # TODO sometimes there is translation after add color, the exp prove that
+ # \mathcolor{black}{ A } is better than \mathcolor{black}{A}
+ l_split_copy[idx] = r'\mathcolor{black}{ ' + l_split_copy[idx] + ' }'
+
+ l_new = ' '.join(l_split_copy)
+ l_new = r'\mathcolor{gray}{ ' + l_new + ' }'
+ render_dict[str(idx)] = l_new, token
+ next_idx = idx + 1
+
+ return l_split, next_idx, render_dict
+
+
+def token_add_color_RGB(l_split, idx, token_list, brace_color=False):
+ """using \\mathcolor[RGB]{r,g,b} to render latex.
+ """
+ token = l_split[idx]
+ if not token:
+ next_idx = idx + 1
+ elif token in PHANTOM_Tokens:
+ # ** special tokens that do not need render, skip it
+ if l_split[idx + 1] == '{':
+ brace_end = find_matching_brace(l_split, idx + 1)
+ else:
+ brace_end = idx + 1
+ next_idx = brace_end + 1
+ elif token in TWO_Tail_Tokens:
+ # ** tokens such as \frac A B, and the token needs render too.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ den_start = num_end + 1
+ den_end = find_matching_brace(l_split, den_start)
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + [color_token + token] + \
+ l_split[idx + 1: den_end + 1] + ["}"] + l_split[den_end + 1:]
+ token_list.append(token)
+ next_idx = idx + 1
+ elif token in ONE_Tail_Tokens:
+ # ** tokens such as \hat A, and the token needs render too.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ if token != "\\underbrace" and num_end + 1 < len(l_split) and l_split[num_end + 1] == "_":
+ l_split = l_split[:idx] + ["{" + color_token + token] + \
+ l_split[idx + 1: num_end + 1] + ["}}"] + l_split[num_end + 1:]
+ else:
+ l_split = l_split[:idx] + [color_token + token] + \
+ l_split[idx + 1: num_end + 1] + ["}"] + l_split[num_end + 1:]
+ token_list.append(token)
+ next_idx = idx + 1
+ elif token in ONE_Tail_Invisb_Tokens:
+ # ** tokens such as \text A B, and the token does not need render.
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ sub_idx = num_start + 1
+ if num_end - num_start == 2:
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ token_list.append(l_split[num_start + 1])
+ l_split = l_split[:num_start + 1] + [color_token
+ + l_split[num_start + 1] + "}"] + l_split[num_end:]
+ else:
+ while sub_idx < num_end:
+ l_split, sub_idx, token_list = token_add_color_RGB(l_split, sub_idx, token_list)
+ next_idx = num_end + 1
+ elif token in AB_Tail_Tokens:
+ # ** special token \xrightarrow, could be \xrightarrow [] {} or \xrightarrow {}, process method are different.
+ if l_split[idx + 1] == '{':
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start)
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + [color_token + token] + \
+ l_split[idx + 1: num_end + 1] + ["}"] + l_split[num_end + 1:]
+ token_list.append(token)
+ sub_idx = num_start + 1
+ while sub_idx < num_end:
+ l_split, sub_idx, token_list = token_add_color_RGB(l_split, sub_idx, token_list)
+ next_idx = num_end + 1
+ elif l_split[idx + 1] == '[':
+ num_start = idx + 1
+ num_end = find_matching_brace(l_split, num_start, brace=['[', ']'])
+ den_start = num_end + 1
+ den_end = find_matching_brace(l_split, den_start)
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + [color_token + token] + \
+ l_split[idx + 1: den_end + 1] + ["}"] + l_split[den_end + 1:]
+ token_list.append(token)
+ sub_idx = num_start + 1
+ while sub_idx < num_end:
+ l_split, sub_idx, token_list = token_add_color_RGB(
+ l_split, sub_idx, token_list, brace_color=True)
+ sub_idx = den_start + 1
+ while sub_idx < den_end:
+ l_split, sub_idx, token_list = token_add_color_RGB(l_split, sub_idx, token_list)
+ next_idx = den_end + 1
+ elif token in ["\\multicolumn", "\\multirow"]:
+ # ** tokens with three {}, such as \multicolumn {} {} {}, the text in third {} need be rendered.
+ first_start = idx + 1
+ first_end = find_matching_brace(l_split, first_start)
+ second_start = first_end + 1
+ second_end = find_matching_brace(l_split, second_start)
+ third_start = second_end + 1
+ third_end = find_matching_brace(l_split, third_start)
+
+ sub_idx = third_start + 1
+ while sub_idx < third_end:
+ l_split, sub_idx, token_list = token_add_color_RGB(l_split, sub_idx, token_list)
+ next_idx = third_end + 1
+ elif token in SKIP_Tokens + TWO_Tail_Invisb_Tokens or any(re.match(pattern, token) for pattern in SKIP_PATTERNS):
+ # ** tokens no need render, just skip
+ # print('skip', idx, token)
+ # TODO special case :[], could be single, or in \sqrt[]{}.
+ if (token == "[" and l_split[idx - 1] != "\\sqrt") or (token
+ == "]" and idx >= 3 and l_split[idx - 3] != "\\sqrt"):
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + [color_token + l_split[idx] + "}"] + l_split[idx + 1:]
+ token_list.append(token)
+ next_idx = idx + 1
+ else:
+ next_idx = idx + 1
+ else:
+ # ** nomal token
+ if brace_color or (idx > 1 and l_split[idx - 1] == "_"):
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + ["{" + color_token + l_split[idx] + "}}"] + l_split[idx + 1:]
+ token_list.append(token)
+ next_idx = idx + 1
+ else:
+ color_token = "\\color[RGB]{>}{".replace("", str(len(token_list)))
+ l_split = l_split[:idx] + [color_token + l_split[idx] + "}"] + l_split[idx + 1:]
+ token_list.append(token)
+ next_idx = idx + 1
+ return l_split, next_idx, token_list
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_render_percentage.py b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_render_percentage.py
new file mode 100644
index 000000000..dfc4f2b88
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/latex_render_percentage.py
@@ -0,0 +1,114 @@
+import argparse
+import json
+import os
+import shutil
+import subprocess
+import time
+from multiprocessing import Pool
+
+formular_template = r"""
+\documentclass[12pt]{article}
+\usepackage[landscape]{geometry}
+\usepackage{geometry}
+\geometry{a5paper,scale=0.98}
+\pagestyle{empty}
+# \usepackage{booktabs}
+\usepackage{amsmath}
+\usepackage{amssymb}
+\usepackage{xcolor}
+\begin{document}
+\makeatletter
+\renewcommand*{\@textcolor}[3]{%%
+ \protect\leavevmode
+ \begingroup
+ \color#1{#2}#3%%
+ \endgroup
+}
+\makeatother
+\begin{displaymath}
+%s
+\end{displaymath}
+\end{document}
+"""
+
+
+def run_shell_cmd(cmd, max_time=15):
+ child = subprocess.Popen(cmd, shell=True)
+ for i in range(max_time):
+ if child.poll():
+ return True
+ if i == max_time - 1:
+ child.kill()
+ return False
+ time.sleep(1)
+ return False
+
+
+def render_latex(latex_code, basename, latex_dir, pdf_dir):
+ latex_path = os.path.join(latex_dir, basename + ".tex")
+ pdf_path = os.path.join(pdf_dir, basename + ".pdf")
+ with open(latex_path, "w") as f:
+ f.write(formular_template % latex_code)
+# # cmd = f"pdflatex -interaction=nonstopmode -output-directory={pdf_dir} -output-format=pdf {latex_path} >/dev/null"
+# run_cmd(
+# f"/mnt/hwfile/opendatalab/guzhuangcheng/programme/texlive/bin/x86_64-linux/xelatex -interaction=nonstopmode -output-directory={temp_dir} {tex_filename} >/dev/null")
+# run_shell_cmd(cmd)
+# return pdf_path
+#
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--input', '-i', type=str, default='data/pred_results/test.json')
+ parser.add_argument('--clean', action='store_true', default=False)
+ parser.add_argument('--gt', action='store_true', default=False)
+ args = parser.parse_args()
+
+ if args.gt:
+ output_path = os.path.join("output", 'gt.json')
+ load_key = 'gt'
+ else:
+ load_key = 'pred'
+ output_path = os.path.join("output", os.path.basename(args.input))
+
+ temp_dir = f"render_temp_dir"
+ try:
+ shutil.rmtree(temp_dir)
+ except BaseException:
+ pass
+ latex_dir = os.path.join(temp_dir, "texes")
+ pdf_dir = os.path.join(temp_dir, "pdfs")
+ os.makedirs(latex_dir, exist_ok=True)
+ os.makedirs(pdf_dir, exist_ok=True)
+
+ with open(args.input, "r") as f:
+ input_data = json.load(f)
+
+ myP = Pool(200)
+ for idx, item in enumerate(input_data):
+ basename = f"sample_{idx}"
+ myP.apply_async(render_latex, args=(item[load_key], basename, latex_dir, pdf_dir))
+ myP.close()
+ print("processing, may take some times.")
+ myP.join()
+
+ success_num = 0
+ total_num = 0
+ for idx, item in enumerate(input_data):
+ basename = f"sample_{idx}"
+ total_num += 1
+ pdf_path = os.path.join(pdf_dir, basename + ".pdf")
+ if os.path.exists(pdf_path):
+ success_num += 1
+ item['renderable'] = 1
+ else:
+ item['renderable'] = 0
+
+ print("total num:", total_num, "render success num:", success_num)
+ with open(output_path, "w") as f:
+ f.write(json.dumps(input_data, indent=2))
+ if args.clean:
+ try:
+ shutil.rmtree(temp_dir)
+ except BaseException:
+ pass
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_formula.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_formula.js
new file mode 100644
index 000000000..883e661a0
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_formula.js
@@ -0,0 +1,385 @@
+const path = require('path');
+var katex = require(path.join(__dirname,"third_party/katex/katex.js"))
+options = require(path.join(__dirname,"third_party/katex/src/Options.js"))
+var readline = require('readline');
+var rl = readline.createInterface({
+ input: process.stdin,
+ output: process.stdout,
+ terminal: false
+});
+
+
+rl.on('line', function(line){
+ a = line
+ if (line[0] == "%") {
+ line = line.substr(1, line.length - 1);
+ }
+ line = line.split('%')[0];
+
+ line = line.split('\\~').join(' ');
+
+ for (var i = 0; i < 300; i++) {
+ line = line.replace(/\\>/, " ");
+ line = line.replace('$', ' ');
+ line = line.replace(/\\label{.*?}/, "");
+ }
+
+ if (line.indexOf("matrix") == -1 && line.indexOf("cases")==-1 &&
+ line.indexOf("array")==-1 && line.indexOf("begin")==-1) {
+ for (var i = 0; i < 300; i++) {
+ line = line.replace(/\\\\/, "\\,");
+ }
+ }
+
+
+ line = line + " "
+ // global_str is tokenized version (build in parser.js)
+ // norm_str is normalized version build by renderer below.
+ try {
+
+
+ if (process.argv[2] == "tokenize") {
+ var tree = katex.__parse(line, {});
+ console.log(global_str.replace(/\\label { .*? }/, ""));
+ } else {
+ for (var i = 0; i < 300; ++i) {
+ line = line.replace(/{\\rm/, "\\mathrm{");
+ line = line.replace(/{ \\rm/, "\\mathrm{");
+ line = line.replace(/\\rm{/, "\\mathrm{");
+ }
+
+ var tree = katex.__parse(line, {});
+ buildExpression(tree, new options({}));
+ for (var i = 0; i < 300; ++i) {
+ norm_str = norm_str.replace('SSSSSS', '$');
+ norm_str = norm_str.replace(' S S S S S S', '$');
+ }
+ console.log(norm_str.replace(/\\label { .*? }/, ""));
+ }
+ } catch (e) {
+ console.error(line);
+ console.error(norm_str);
+ console.error(e);
+ console.log();
+ }
+ global_str = ""
+ norm_str = ""
+})
+
+
+
+// This is a LaTeX AST to LaTeX Renderer (modified version of KaTeX AST-> MathML).
+norm_str = ""
+
+var groupTypes = {};
+
+groupTypes.mathord = function(group, options) {
+ if (options.font == "mathrm"){
+ for (i = 0; i < group.value.length; ++i ) {
+ if (group.value[i] == " ") {
+ norm_str = norm_str + group.value[i] + "\; ";
+ } else {
+ norm_str = norm_str + group.value[i] + " ";
+ }
+ }
+ } else {
+ norm_str = norm_str + group.value + " ";
+ }
+};
+
+groupTypes.textord = function(group, options) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.bin = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.rel = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.open = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.close = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.inner = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.punct = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.ordgroup = function(group, options) {
+ norm_str = norm_str + "{ ";
+
+ buildExpression(group.value, options);
+
+ norm_str = norm_str + "} ";
+};
+
+groupTypes.text = function(group, options) {
+
+ norm_str = norm_str + "\\mathrm { ";
+
+ buildExpression(group.value.body, options);
+ norm_str = norm_str + "} ";
+};
+
+groupTypes.color = function(group, options) {
+ var inner = buildExpression(group.value.value, options);
+
+ var node = new mathMLTree.MathNode("mstyle", inner);
+
+ node.setAttribute("mathcolor", group.value.color);
+
+ return node;
+};
+
+groupTypes.supsub = function(group, options) {
+ buildGroup(group.value.base, options);
+
+ if (group.value.sub) {
+ norm_str = norm_str + "_ ";
+ if (group.value.sub.type != 'ordgroup') {
+ norm_str = norm_str + " { ";
+ buildGroup(group.value.sub, options);
+ norm_str = norm_str + "} ";
+ } else {
+ buildGroup(group.value.sub, options);
+ }
+
+ }
+
+ if (group.value.sup) {
+ norm_str = norm_str + "^ ";
+ if (group.value.sup.type != 'ordgroup') {
+ norm_str = norm_str + " { ";
+ buildGroup(group.value.sup, options);
+ norm_str = norm_str + "} ";
+ } else {
+ buildGroup(group.value.sup, options);
+ }
+ }
+
+};
+
+groupTypes.genfrac = function(group, options) {
+ if (!group.value.hasBarLine) {
+ norm_str = norm_str + "\\binom ";
+ } else {
+ norm_str = norm_str + "\\frac ";
+ }
+ buildGroup(group.value.numer, options);
+ buildGroup(group.value.denom, options);
+
+};
+
+groupTypes.array = function(group, options) {
+ norm_str = norm_str + "\\begin{array} { ";
+ if (group.value.cols) {
+ group.value.cols.map(function(start) {
+ if (start && start.align) {
+ norm_str = norm_str + start.align + " ";}});
+ } else {
+ group.value.body[0].map(function(start) {
+ norm_str = norm_str + "l ";
+ } );
+ }
+ norm_str = norm_str + "} ";
+ group.value.body.map(function(row) {
+ if (row.some(cell => cell.value.length > 0)) { // orginal code: if (row[0].value.length > 0)
+ out = row.map(function(cell) {
+ buildGroup(cell, options);
+ if (norm_str.length > 4
+ && norm_str.substring(norm_str.length-4, norm_str.length) == "{ } ") {
+ norm_str = norm_str.substring(0, norm_str.length-4) ;
+ }
+ norm_str = norm_str + "& ";
+ });
+ norm_str = norm_str.substring(0, norm_str.length-2) + "\\\\ ";
+ }
+ });
+ norm_str = norm_str + "\\end{array} ";
+};
+
+groupTypes.sqrt = function(group, options) {
+ var node;
+ if (group.value.index) {
+ norm_str = norm_str + "\\sqrt [ ";
+ buildExpression(group.value.index.value, options);
+ norm_str = norm_str + "] ";
+ buildGroup(group.value.body, options);
+ } else {
+ norm_str = norm_str + "\\sqrt ";
+ buildGroup(group.value.body, options);
+ }
+};
+
+groupTypes.leftright = function(group, options) {
+
+
+
+ norm_str = norm_str + "\\left" + group.value.left + " ";
+ buildExpression(group.value.body, options);
+ norm_str = norm_str + "\\right" + group.value.right + " ";
+};
+
+groupTypes.accent = function(group, options) {
+ if (group.value.base.type != 'ordgroup') {
+ norm_str = norm_str + group.value.accent + " { ";
+ buildGroup(group.value.base, options);
+ norm_str = norm_str + "} ";
+ } else {
+ norm_str = norm_str + group.value.accent + " ";
+ buildGroup(group.value.base, options);
+ }
+};
+
+groupTypes.spacing = function(group) {
+ var node;
+ if (group.value == " ") {
+ norm_str = norm_str + "~ ";
+ } else {
+ norm_str = norm_str + group.value + " ";
+ }
+ return node;
+};
+
+groupTypes.op = function(group) {
+ var node;
+
+ // TODO(emily): handle big operators using the `largeop` attribute
+
+
+ if (group.value.symbol) {
+ // This is a symbol. Just add the symbol.
+ norm_str = norm_str + group.value.body + " ";
+
+ } else {
+ if (group.value.limits == false) {
+ norm_str = norm_str + "\\\operatorname { ";
+ } else {
+ norm_str = norm_str + "\\\operatorname* { ";
+ }
+ for (i = 1; i < group.value.body.length; ++i ) {
+ norm_str = norm_str + group.value.body[i] + " ";
+ }
+ norm_str = norm_str + "} ";
+ }
+};
+
+groupTypes.katex = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mtext", [new mathMLTree.TextNode("KaTeX")]);
+
+ return node;
+};
+
+
+
+groupTypes.font = function(group, options) {
+ var font = group.value.font;
+ if (font == "mbox" || font == "hbox") {
+ font = "mathrm";
+ }
+ norm_str = norm_str + "\\" + font + " ";
+ buildGroup(group.value.body, options.withFont(font));
+};
+
+groupTypes.delimsizing = function(group) {
+ var children = [];
+ norm_str = norm_str + group.value.funcName + " " + group.value.value + " ";
+};
+
+groupTypes.styling = function(group, options) {
+ norm_str = norm_str + " " + group.value.original + " ";
+ buildExpression(group.value.value, options);
+
+};
+
+groupTypes.sizing = function(group, options) {
+
+ if (group.value.original == "\\rm") {
+ norm_str = norm_str + "\\mathrm { ";
+ buildExpression(group.value.value, options.withFont("mathrm"));
+ norm_str = norm_str + "} ";
+ } else {
+ norm_str = norm_str + " " + group.value.original + " ";
+ buildExpression(group.value.value, options);
+ }
+};
+
+groupTypes.overline = function(group, options) {
+ norm_str = norm_str + "\\overline { ";
+
+ buildGroup(group.value.body, options);
+ norm_str = norm_str + "} ";
+ norm_str = norm_str;
+
+};
+
+groupTypes.underline = function(group, options) {
+ norm_str = norm_str + "\\underline { ";
+ buildGroup(group.value.body, options);
+ norm_str = norm_str + "} ";
+
+ norm_str = norm_str;
+
+};
+
+groupTypes.rule = function(group) {
+ norm_str = norm_str + "\\rule { "+group.value.width.number+" "+group.value.width.unit+" } { "+group.value.height.number+" "+group.value.height.unit+ " } ";
+
+};
+
+groupTypes.llap = function(group, options) {
+ norm_str = norm_str + "\\llap ";
+ buildGroup(group.value.body, options);
+};
+
+groupTypes.rlap = function(group, options) {
+ norm_str = norm_str + "\\rlap ";
+ buildGroup(group.value.body, options);
+
+};
+
+groupTypes.phantom = function(group, options, prev) {
+ norm_str = norm_str + "\\phantom { ";
+ buildExpression(group.value.value, options);
+ norm_str = norm_str + "} ";
+
+};
+
+/**
+ * Takes a list of nodes, builds them, and returns a list of the generated
+ * MathML nodes. A little simpler than the HTML version because we don't do any
+ * previous-node handling.
+ */
+var buildExpression = function(expression, options) {
+ var groups = [];
+ for (var i = 0; i < expression.length; i++) {
+ var group = expression[i];
+ buildGroup(group, options);
+ }
+ // console.log(norm_str);
+ // return groups;
+};
+
+/**
+ * Takes a group from the parser and calls the appropriate groupTypes function
+ * on it to produce a MathML node.
+ */
+var buildGroup = function(group, options) {
+ if (groupTypes[group.type]) {
+ groupTypes[group.type](group, options);
+ } else {
+ throw new ParseError(
+ "Got group of unknown type: '" + group.type + "'");
+ }
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_tabular.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_tabular.js
new file mode 100644
index 000000000..40ac043bd
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/preprocess_tabular.js
@@ -0,0 +1,393 @@
+const path = require('path');
+var katex = require(path.join(__dirname,"third_party/katex/katex.js"))
+options = require(path.join(__dirname,"third_party/katex/src/Options.js"))
+var readline = require('readline');
+var rl = readline.createInterface({
+ input: process.stdin,
+ output: process.stdout,
+ terminal: false
+});
+
+
+rl.on('line', function(line){
+ a = line
+ if (line[0] == "%") {
+ line = line.substr(1, line.length - 1);
+ }
+ // line = line.split('%')[0];
+
+ line = line.split('\\~').join(' ');
+
+ for (var i = 0; i < 300; i++) {
+ line = line.replace(/\\>/, " ");
+ // line = line.replace('$', ' ');
+ line = line.replace(/\\label{.*?}/, "");
+ }
+
+ if (line.indexOf("matrix") == -1 && line.indexOf("cases")==-1 &&
+ line.indexOf("array")==-1 && line.indexOf("begin")==-1) {
+ for (var i = 0; i < 300; i++) {
+ line = line.replace(/\\\\/, "\\,");
+ }
+ }
+
+
+ line = line + " "
+ // global_str is tokenized version (build in parser.js)
+ // norm_str is normalized version build by renderer below.
+ try {
+
+
+ if (process.argv[2] == "tokenize") {
+ var tree = katex.__parse(line, {});
+ console.log(global_str.replace(/\\label { .*? }/, ""));
+ } else {
+ for (var i = 0; i < 300; ++i) {
+ line = line.replace(/{\\rm/, "\\mathrm{");
+ line = line.replace(/{ \\rm/, "\\mathrm{");
+ line = line.replace(/\\rm{/, "\\mathrm{");
+ }
+
+ var tree = katex.__parse(line, {});
+ buildExpression(tree, new options({}));
+ for (var i = 0; i < 300; ++i) {
+ norm_str = norm_str.replace('SSSSSS', '$');
+ norm_str = norm_str.replace(' S S S S S S', '$');
+ }
+ console.log(norm_str.replace(/\\label { .*? }/, ""));
+ }
+ } catch (e) {
+ console.error(line);
+ console.error(norm_str);
+ console.error(e);
+ console.log("");
+ }
+ global_str = ""
+ norm_str = ""
+})
+
+
+
+// This is a LaTeX AST to LaTeX Renderer (modified version of KaTeX AST-> MathML).
+norm_str = ""
+
+var groupTypes = {};
+
+groupTypes.mathord = function(group, options) {
+ if (options.font == "mathrm"){
+ for (i = 0; i < group.value.length; ++i ) {
+ if (group.value[i] == " ") {
+ norm_str = norm_str + group.value[i] + "\; ";
+ } else {
+ norm_str = norm_str + group.value[i] + " ";
+ }
+ }
+ } else {
+ norm_str = norm_str + group.value + " ";
+ }
+};
+
+groupTypes.textord = function(group, options) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.bin = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.rel = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.open = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.close = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.inner = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.punct = function(group) {
+ norm_str = norm_str + group.value + " ";
+};
+
+groupTypes.ordgroup = function(group, options) {
+ norm_str = norm_str + "{ ";
+
+ buildExpression(group.value, options);
+
+ norm_str = norm_str + "} ";
+};
+
+groupTypes.text = function(group, options) {
+
+ norm_str = norm_str + "\\mathrm { ";
+
+ buildExpression(group.value.body, options);
+ norm_str = norm_str + "} ";
+};
+
+groupTypes.color = function(group, options) {
+ var inner = buildExpression(group.value.value, options);
+
+ var node = new mathMLTree.MathNode("mstyle", inner);
+
+ node.setAttribute("mathcolor", group.value.color);
+
+ return node;
+};
+
+groupTypes.supsub = function(group, options) {
+ buildGroup(group.value.base, options);
+
+ if (group.value.sub) {
+ norm_str = norm_str + "_ ";
+ if (group.value.sub.type != 'ordgroup') {
+ norm_str = norm_str + " { ";
+ buildGroup(group.value.sub, options);
+ norm_str = norm_str + "} ";
+ } else {
+ buildGroup(group.value.sub, options);
+ }
+
+ }
+
+ if (group.value.sup) {
+ norm_str = norm_str + "^ ";
+ if (group.value.sup.type != 'ordgroup') {
+ norm_str = norm_str + " { ";
+ buildGroup(group.value.sup, options);
+ norm_str = norm_str + "} ";
+ } else {
+ buildGroup(group.value.sup, options);
+ }
+ }
+
+};
+
+groupTypes.genfrac = function(group, options) {
+ if (!group.value.hasBarLine) {
+ norm_str = norm_str + "\\binom ";
+ } else {
+ norm_str = norm_str + "\\frac ";
+ }
+ buildGroup(group.value.numer, options);
+ buildGroup(group.value.denom, options);
+
+};
+
+groupTypes.array = function(group, options) {
+ norm_str = norm_str + "\\begin{" + group.value.style + "} ";
+
+ if (group.value.style == "array" || group.value.style == "tabular" || group.value.style == "tabularx") {
+ norm_str = norm_str + "{ ";
+ if (group.value.cols) {
+ group.value.cols.map(function(start) {
+ if (start) {
+ if (start.type == "align") {
+ norm_str = norm_str + start.align + " ";
+ } else if (start.type == "separator") {
+ norm_str = norm_str + start.separator + " ";
+ }
+ }
+ });
+ } else {
+ group.value.body[0].map(function(start) {
+ norm_str = norm_str + "c ";
+ } );
+ }
+ norm_str = norm_str + "} ";
+ }
+ group.value.body.map(function(row) {
+ if (row.length > 1 || row[0].value.length > 0) {
+ if (row[0].value[0] && row[0].value[0].value == "\\hline") {
+ norm_str = norm_str + "\\hline ";
+ row[0].value = row[0].value.slice(1);
+ }
+ out = row.map(function(cell) {
+ buildGroup(cell, options);
+ norm_str = norm_str + "& ";
+ });
+ norm_str = norm_str.substring(0, norm_str.length-2) + "\\\\ ";
+ }
+ });
+ norm_str = norm_str + "\\end{" + group.value.style + "} ";
+};
+
+groupTypes.sqrt = function(group, options) {
+ var node;
+ if (group.value.index) {
+ norm_str = norm_str + "\\sqrt [ " + group.value.index + " ] ";
+ buildGroup(group.value.body, options);
+ } else {
+ norm_str = norm_str + "\\sqrt ";
+ buildGroup(group.value.body, options);
+ }
+};
+
+groupTypes.leftright = function(group, options) {
+
+
+
+ norm_str = norm_str + "\\left" + group.value.left + " ";
+ buildExpression(group.value.body, options);
+ norm_str = norm_str + "\\right" + group.value.right + " ";
+};
+
+groupTypes.accent = function(group, options) {
+ if (group.value.base.type != 'ordgroup') {
+ norm_str = norm_str + group.value.accent + " { ";
+ buildGroup(group.value.base, options);
+ norm_str = norm_str + "} ";
+ } else {
+ norm_str = norm_str + group.value.accent + " ";
+ buildGroup(group.value.base, options);
+ }
+};
+
+groupTypes.spacing = function(group) {
+ var node;
+ if (group.value == " ") {
+ norm_str = norm_str + "~ ";
+ } else {
+ norm_str = norm_str + group.value + " ";
+ }
+ return node;
+};
+
+groupTypes.op = function(group) {
+ var node;
+
+ // TODO(emily): handle big operators using the `largeop` attribute
+
+
+ if (group.value.symbol) {
+ // This is a symbol. Just add the symbol.
+ norm_str = norm_str + group.value.body + " ";
+
+ } else {
+ if (group.value.limits == false) {
+ norm_str = norm_str + "\\\operatorname { ";
+ } else {
+ norm_str = norm_str + "\\\operatorname* { ";
+ }
+ for (i = 1; i < group.value.body.length; ++i ) {
+ norm_str = norm_str + group.value.body[i] + " ";
+ }
+ norm_str = norm_str + "} ";
+ }
+};
+
+groupTypes.katex = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mtext", [new mathMLTree.TextNode("KaTeX")]);
+
+ return node;
+};
+
+
+
+groupTypes.font = function(group, options) {
+ var font = group.value.font;
+ if (font == "mbox" || font == "hbox") {
+ font = "mathrm";
+ }
+ norm_str = norm_str + "\\" + font + " ";
+ buildGroup(group.value.body, options.withFont(font));
+};
+
+groupTypes.delimsizing = function(group) {
+ var children = [];
+ norm_str = norm_str + group.value.funcName + " " + group.value.value + " ";
+};
+
+groupTypes.styling = function(group, options) {
+ norm_str = norm_str + " " + group.value.original + " ";
+ buildExpression(group.value.value, options);
+
+};
+
+groupTypes.sizing = function(group, options) {
+
+ if (group.value.original == "\\rm") {
+ norm_str = norm_str + "\\mathrm { ";
+ buildExpression(group.value.value, options.withFont("mathrm"));
+ norm_str = norm_str + "} ";
+ } else {
+ norm_str = norm_str + " " + group.value.original + " ";
+ buildExpression(group.value.value, options);
+ }
+};
+
+groupTypes.overline = function(group, options) {
+ norm_str = norm_str + "\\overline { ";
+
+ buildGroup(group.value.body, options);
+ norm_str = norm_str + "} ";
+ norm_str = norm_str;
+
+};
+
+groupTypes.underline = function(group, options) {
+ norm_str = norm_str + "\\underline { ";
+ buildGroup(group.value.body, options);
+ norm_str = norm_str + "} ";
+
+ norm_str = norm_str;
+
+};
+
+groupTypes.rule = function(group) {
+ norm_str = norm_str + "\\rule { "+group.value.width.number+" "+group.value.width.unit+" } { "+group.value.height.number+" "+group.value.height.unit+ " } ";
+
+};
+
+groupTypes.llap = function(group, options) {
+ norm_str = norm_str + "\\llap ";
+ buildGroup(group.value.body, options);
+};
+
+groupTypes.rlap = function(group, options) {
+ norm_str = norm_str + "\\rlap ";
+ buildGroup(group.value.body, options);
+
+};
+
+groupTypes.phantom = function(group, options, prev) {
+ norm_str = norm_str + "\\phantom { ";
+ buildExpression(group.value.value, options);
+ norm_str = norm_str + "} ";
+
+};
+
+/**
+ * Takes a list of nodes, builds them, and returns a list of the generated
+ * MathML nodes. A little simpler than the HTML version because we don't do any
+ * previous-node handling.
+ */
+var buildExpression = function(expression, options) {
+ var groups = [];
+ for (var i = 0; i < expression.length; i++) {
+ var group = expression[i];
+ buildGroup(group, options);
+ }
+ // console.log(norm_str);
+ // return groups;
+};
+
+/**
+ * Takes a group from the parser and calls the appropriate groupTypes function
+ * on it to produce a MathML node.
+ */
+var buildGroup = function(group, options) {
+ if (groupTypes[group.type]) {
+ groupTypes[group.type](group, options);
+ } else {
+ throw new ParseError(
+ "Got group of unknown type: '" + group.type + "'");
+ }
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/README.md b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/README.md
new file mode 100644
index 000000000..bc74abab4
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/README.md
@@ -0,0 +1 @@
+Directly taken from https://github.com/harvardnlp/im2markup
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/README.md b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/README.md
new file mode 100644
index 000000000..31cf658d8
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/README.md
@@ -0,0 +1,68 @@
+# [](https://khan.github.io/KaTeX/) [](https://travis-ci.org/Khan/KaTeX)
+
+[](https://gitter.im/Khan/KaTeX?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
+KaTeX is a fast, easy-to-use JavaScript library for TeX math rendering on the web.
+
+ * **Fast:** KaTeX renders its math synchronously and doesn't need to reflow the page. See how it compares to a competitor in [this speed test](http://jsperf.com/katex-vs-mathjax/).
+ * **Print quality:** KaTeX’s layout is based on Donald Knuth’s TeX, the gold standard for math typesetting.
+ * **Self contained:** KaTeX has no dependencies and can easily be bundled with your website resources.
+ * **Server side rendering:** KaTeX produces the same output regardless of browser or environment, so you can pre-render expressions using Node.js and send them as plain HTML.
+
+KaTeX supports all major browsers, including Chrome, Safari, Firefox, Opera, and IE 8 - IE 11. A list of supported commands can be on the [wiki](https://github.com/Khan/KaTeX/wiki/Function-Support-in-KaTeX).
+
+## Usage
+
+You can [download KaTeX](https://github.com/khan/katex/releases) and host it on your server or include the `katex.min.js` and `katex.min.css` files on your page directly from a CDN:
+
+```html
+
+
+```
+
+#### In-browser rendering
+
+Call `katex.render` with a TeX expression and a DOM element to render into:
+
+```js
+katex.render("c = \\pm\\sqrt{a^2 + b^2}", element);
+```
+
+If KaTeX can't parse the expression, it throws a `katex.ParseError` error.
+
+#### Server side rendering or rendering to a string
+
+To generate HTML on the server or to generate an HTML string of the rendered math, you can use `katex.renderToString`:
+
+```js
+var html = katex.renderToString("c = \\pm\\sqrt{a^2 + b^2}");
+// '...'
+```
+
+Make sure to include the CSS and font files, but there is no need to include the JavaScript. Like `render`, `renderToString` throws if it can't parse the expression.
+
+#### Rendering options
+
+You can provide an object of options as the last argument to `katex.render` and `katex.renderToString`. Available options are:
+
+- `displayMode`: `boolean`. If `true` the math will be rendered in display mode, which will put the math in display style (so `\int` and `\sum` are large, for example), and will center the math on the page on its own line. If `false` the math will be rendered in inline mode. (default: `false`)
+- `throwOnError`: `boolean`. If `true`, KaTeX will throw a `ParseError` when it encounters an unsupported command. If `false`, KaTeX will render the unsupported command as text in the color given by `errorColor`. (default: `true`)
+- `errorColor`: `string`. A color string given in the format `"#XXX"` or `"#XXXXXX"`. This option determines the color which unsupported commands are rendered in. (default: `#cc0000`)
+
+For example:
+
+```js
+katex.render("c = \\pm\\sqrt{a^2 + b^2}", element, { displayMode: true });
+```
+
+#### Automatic rendering of math on a page
+
+Math on the page can be automatically rendered using the auto-render extension. See [the Auto-render README](contrib/auto-render/README.md) for more information.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md)
+
+## License
+
+KaTeX is licensed under the [MIT License](http://opensource.org/licenses/MIT).
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js
new file mode 100644
index 000000000..b64de377c
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js
@@ -0,0 +1,32 @@
+#!/usr/bin/env node
+// Simple CLI for KaTeX.
+// Reads TeX from stdin, outputs HTML to stdout.
+/* eslint no-console:0 */
+
+var katex = require("./");
+var input = "";
+
+// Skip the first two args, which are just "node" and "cli.js"
+var args = process.argv.slice(2);
+
+if (args.indexOf("--help") !== -1) {
+ console.log(process.argv[0] + " " + process.argv[1] +
+ " [ --help ]" +
+ " [ --display-mode ]");
+
+ console.log("\n" +
+ "Options:");
+ console.log(" --help Display this help message");
+ console.log(" --display-mode Render in display mode (not inline mode)");
+ process.exit();
+}
+
+process.stdin.on("data", function(chunk) {
+ input += chunk.toString();
+});
+
+process.stdin.on("end", function() {
+ var options = { displayMode: args.indexOf("--display-mode") !== -1 };
+ var output = katex.renderToString(input, options);
+ console.log(output);
+});
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js
new file mode 100644
index 000000000..4d64606bf
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js
@@ -0,0 +1,74 @@
+/* eslint no-console:0 */
+/**
+ * This is the main entry point for KaTeX. Here, we expose functions for
+ * rendering expressions either to DOM nodes or to markup strings.
+ *
+ * We also expose the ParseError class to check if errors thrown from KaTeX are
+ * errors in the expression, or errors in javascript handling.
+ */
+
+var ParseError = require("./src/ParseError");
+var Settings = require("./src/Settings");
+
+var buildTree = require("./src/buildTree");
+var parseTree = require("./src/parseTree");
+var utils = require("./src/utils");
+
+/**
+ * Parse and build an expression, and place that expression in the DOM node
+ * given.
+ */
+var render = function(expression, baseNode, options) {
+ utils.clearNode(baseNode);
+
+ var settings = new Settings(options);
+
+ var tree = parseTree(expression, settings);
+ var node = buildTree(tree, expression, settings).toNode();
+
+ baseNode.appendChild(node);
+};
+
+// KaTeX's styles don't work properly in quirks mode. Print out an error, and
+// disable rendering.
+if (typeof document !== "undefined") {
+ if (document.compatMode !== "CSS1Compat") {
+ typeof console !== "undefined" && console.warn(
+ "Warning: KaTeX doesn't work in quirks mode. Make sure your " +
+ "website has a suitable doctype.");
+
+ render = function() {
+ throw new ParseError("KaTeX doesn't work in quirks mode.");
+ };
+ }
+}
+
+/**
+ * Parse and build an expression, and return the markup for that.
+ */
+var renderToString = function(expression, options) {
+ var settings = new Settings(options);
+
+ var tree = parseTree(expression, settings);
+ return buildTree(tree, expression, settings).toMarkup();
+};
+
+/**
+ * Parse an expression and return the parse tree.
+ */
+var generateParseTree = function(expression, options) {
+ var settings = new Settings(options);
+ return parseTree(expression, settings);
+};
+
+module.exports = {
+ render: render,
+ renderToString: renderToString,
+ /**
+ * NOTE: This method is not currently recommended for public use.
+ * The internal tree representation is unstable and is very likely
+ * to change. Use at your own risk.
+ */
+ __parse: generateParseTree,
+ ParseError: ParseError,
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js
new file mode 100644
index 000000000..5cb8d3d4e
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js
@@ -0,0 +1,162 @@
+/**
+ * The Lexer class handles tokenizing the input in various ways. Since our
+ * parser expects us to be able to backtrack, the lexer allows lexing from any
+ * given starting point.
+ *
+ * Its main exposed function is the `lex` function, which takes a position to
+ * lex from and a type of token to lex. It defers to the appropriate `_innerLex`
+ * function.
+ *
+ * The various `_innerLex` functions perform the actual lexing of different
+ * kinds.
+ */
+
+var matchAt = require("../../match-at");
+
+var ParseError = require("./ParseError");
+
+// The main lexer class
+function Lexer(input) {
+ this._input = input;
+}
+
+// The resulting token returned from `lex`.
+function Token(text, data, position) {
+ this.text = text;
+ this.data = data;
+ this.position = position;
+}
+
+/* The following tokenRegex
+ * - matches typical whitespace (but not NBSP etc.) using its first group
+ * - matches symbol combinations which result in a single output character
+ * - does not match any control character \x00-\x1f except whitespace
+ * - does not match a bare backslash
+ * - matches any ASCII character except those just mentioned
+ * - does not match the BMP private use area \uE000-\uF8FF
+ * - does not match bare surrogate code units
+ * - matches any BMP character except for those just described
+ * - matches any valid Unicode surrogate pair
+ * - matches a backslash followed by one or more letters
+ * - matches a backslash followed by any BMP character, including newline
+ * Just because the Lexer matches something doesn't mean it's valid input:
+ * If there is no matching function or symbol definition, the Parser will
+ * still reject the input.
+ */
+var tokenRegex = new RegExp(
+ "([ \r\n\t]+)|(" + // whitespace
+ "---?" + // special combinations
+ "|[!-\\[\\]-\u2027\u202A-\uD7FF\uF900-\uFFFF]" + // single codepoint
+ "|[\uD800-\uDBFF][\uDC00-\uDFFF]" + // surrogate pair
+ "|\\\\(?:[a-zA-Z]+|[^\uD800-\uDFFF])" + // function name
+ ")"
+);
+
+var whitespaceRegex = /\s*/;
+
+/**
+ * This function lexes a single normal token. It takes a position and
+ * whether it should completely ignore whitespace or not.
+ */
+Lexer.prototype._innerLex = function(pos, ignoreWhitespace) {
+ var input = this._input;
+ if (pos === input.length) {
+ return new Token("EOF", null, pos);
+ }
+ var match = matchAt(tokenRegex, input, pos);
+ if (match === null) {
+ throw new ParseError(
+ "Unexpected character: '" + input[pos] + "'",
+ this, pos);
+ } else if (match[2]) { // matched non-whitespace
+ return new Token(match[2], null, pos + match[2].length);
+ } else if (ignoreWhitespace) {
+ return this._innerLex(pos + match[1].length, true);
+ } else { // concatenate whitespace to a single space
+ return new Token(" ", null, pos + match[1].length);
+ }
+};
+
+// A regex to match a CSS color (like #ffffff or BlueViolet)
+var cssColor = /#[a-z0-9]+|[a-z]+/i;
+
+/**
+ * This function lexes a CSS color.
+ */
+Lexer.prototype._innerLexColor = function(pos) {
+ var input = this._input;
+
+ // Ignore whitespace
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ var match;
+ if ((match = matchAt(cssColor, input, pos))) {
+ // If we look like a color, return a color
+ return new Token(match[0], null, pos + match[0].length);
+ } else {
+ throw new ParseError("Invalid color", this, pos);
+ }
+};
+
+// A regex to match a dimension. Dimensions look like
+// "1.2em" or ".4pt" or "1 ex"
+var sizeRegex = /(-?)\s*(\d+(?:\.\d*)?|\.\d+)\s*([a-z]{2})/;
+
+/**
+ * This function lexes a dimension.
+ */
+Lexer.prototype._innerLexSize = function(pos) {
+ var input = this._input;
+
+ // Ignore whitespace
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ var match;
+ if ((match = matchAt(sizeRegex, input, pos))) {
+ var unit = match[3];
+ // We only currently handle "em" and "ex" units
+ // if (unit !== "em" && unit !== "ex") {
+ // throw new ParseError("Invalid unit: '" + unit + "'", this, pos);
+ // }
+ return new Token(match[0], {
+ number: +(match[1] + match[2]),
+ unit: unit,
+ }, pos + match[0].length);
+ }
+
+ throw new ParseError("Invalid size", this, pos);
+};
+
+/**
+ * This function lexes a string of whitespace.
+ */
+Lexer.prototype._innerLexWhitespace = function(pos) {
+ var input = this._input;
+
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ return new Token(whitespace[0], null, pos);
+};
+
+/**
+ * This function lexes a single token starting at `pos` and of the given mode.
+ * Based on the mode, we defer to one of the `_innerLex` functions.
+ */
+Lexer.prototype.lex = function(pos, mode) {
+ if (mode === "math") {
+ return this._innerLex(pos, true);
+ } else if (mode === "text") {
+ return this._innerLex(pos, false);
+ } else if (mode === "color") {
+ return this._innerLexColor(pos);
+ } else if (mode === "size") {
+ return this._innerLexSize(pos);
+ } else if (mode === "whitespace") {
+ return this._innerLexWhitespace(pos);
+ }
+};
+
+module.exports = Lexer;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js
new file mode 100644
index 000000000..39ff37bfc
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js
@@ -0,0 +1,189 @@
+/**
+ * This file contains information about the options that the Parser carries
+ * around with it while parsing. Data is held in an `Options` object, and when
+ * recursing, a new `Options` object can be created with the `.with*` and
+ * `.reset` functions.
+ */
+
+/**
+ * This is the main options class. It contains the style, size, color, and font
+ * of the current parse level. It also contains the style and size of the parent
+ * parse level, so size changes can be handled efficiently.
+ *
+ * Each of the `.with*` and `.reset` functions passes its current style and size
+ * as the parentStyle and parentSize of the new options class, so parent
+ * handling is taken care of automatically.
+ */
+function Options(data) {
+ this.style = data.style;
+ this.color = data.color;
+ this.size = data.size;
+ this.phantom = data.phantom;
+ this.font = data.font;
+
+ if (data.parentStyle === undefined) {
+ this.parentStyle = data.style;
+ } else {
+ this.parentStyle = data.parentStyle;
+ }
+
+ if (data.parentSize === undefined) {
+ this.parentSize = data.size;
+ } else {
+ this.parentSize = data.parentSize;
+ }
+}
+
+/**
+ * Returns a new options object with the same properties as "this". Properties
+ * from "extension" will be copied to the new options object.
+ */
+Options.prototype.extend = function(extension) {
+ var data = {
+ style: this.style,
+ size: this.size,
+ color: this.color,
+ parentStyle: this.style,
+ parentSize: this.size,
+ phantom: this.phantom,
+ font: this.font,
+ };
+
+ for (var key in extension) {
+ if (extension.hasOwnProperty(key)) {
+ data[key] = extension[key];
+ }
+ }
+
+ return new Options(data);
+};
+
+/**
+ * Create a new options object with the given style.
+ */
+Options.prototype.withStyle = function(style) {
+ return this.extend({
+ style: style,
+ });
+};
+
+/**
+ * Create a new options object with the given size.
+ */
+Options.prototype.withSize = function(size) {
+ return this.extend({
+ size: size,
+ });
+};
+
+/**
+ * Create a new options object with the given color.
+ */
+Options.prototype.withColor = function(color) {
+ return this.extend({
+ color: color,
+ });
+};
+
+/**
+ * Create a new options object with "phantom" set to true.
+ */
+Options.prototype.withPhantom = function() {
+ return this.extend({
+ phantom: true,
+ });
+};
+
+/**
+ * Create a new options objects with the give font.
+ */
+Options.prototype.withFont = function(font) {
+ return this.extend({
+ font: font,
+ });
+};
+
+/**
+ * Create a new options object with the same style, size, and color. This is
+ * used so that parent style and size changes are handled correctly.
+ */
+Options.prototype.reset = function() {
+ return this.extend({});
+};
+
+/**
+ * A map of color names to CSS colors.
+ * TODO(emily): Remove this when we have real macros
+ */
+var colorMap = {
+ "katex-blue": "#6495ed",
+ "katex-orange": "#ffa500",
+ "katex-pink": "#ff00af",
+ "katex-red": "#df0030",
+ "katex-green": "#28ae7b",
+ "katex-gray": "gray",
+ "katex-purple": "#9d38bd",
+ "katex-blueA": "#c7e9f1",
+ "katex-blueB": "#9cdceb",
+ "katex-blueC": "#58c4dd",
+ "katex-blueD": "#29abca",
+ "katex-blueE": "#1c758a",
+ "katex-tealA": "#acead7",
+ "katex-tealB": "#76ddc0",
+ "katex-tealC": "#5cd0b3",
+ "katex-tealD": "#55c1a7",
+ "katex-tealE": "#49a88f",
+ "katex-greenA": "#c9e2ae",
+ "katex-greenB": "#a6cf8c",
+ "katex-greenC": "#83c167",
+ "katex-greenD": "#77b05d",
+ "katex-greenE": "#699c52",
+ "katex-goldA": "#f7c797",
+ "katex-goldB": "#f9b775",
+ "katex-goldC": "#f0ac5f",
+ "katex-goldD": "#e1a158",
+ "katex-goldE": "#c78d46",
+ "katex-redA": "#f7a1a3",
+ "katex-redB": "#ff8080",
+ "katex-redC": "#fc6255",
+ "katex-redD": "#e65a4c",
+ "katex-redE": "#cf5044",
+ "katex-maroonA": "#ecabc1",
+ "katex-maroonB": "#ec92ab",
+ "katex-maroonC": "#c55f73",
+ "katex-maroonD": "#a24d61",
+ "katex-maroonE": "#94424f",
+ "katex-purpleA": "#caa3e8",
+ "katex-purpleB": "#b189c6",
+ "katex-purpleC": "#9a72ac",
+ "katex-purpleD": "#715582",
+ "katex-purpleE": "#644172",
+ "katex-mintA": "#f5f9e8",
+ "katex-mintB": "#edf2df",
+ "katex-mintC": "#e0e5cc",
+ "katex-grayA": "#fdfdfd",
+ "katex-grayB": "#f7f7f7",
+ "katex-grayC": "#eeeeee",
+ "katex-grayD": "#dddddd",
+ "katex-grayE": "#cccccc",
+ "katex-grayF": "#aaaaaa",
+ "katex-grayG": "#999999",
+ "katex-grayH": "#555555",
+ "katex-grayI": "#333333",
+ "katex-kaBlue": "#314453",
+ "katex-kaGreen": "#639b24",
+};
+
+/**
+ * Gets the CSS color of the current options object, accounting for the
+ * `colorMap`.
+ */
+Options.prototype.getColor = function() {
+ if (this.phantom) {
+ return "transparent";
+ } else {
+ return colorMap[this.color] || this.color;
+ }
+};
+
+module.exports = Options;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js
new file mode 100644
index 000000000..320f0bd69
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js
@@ -0,0 +1,40 @@
+/**
+ * This is the ParseError class, which is the main error thrown by KaTeX
+ * functions when something has gone wrong. This is used to distinguish internal
+ * errors from errors in the expression that the user provided.
+ */
+function ParseError(message, lexer, position) {
+ var error = "KaTeX parse error: " + message;
+
+ if (lexer !== undefined && position !== undefined) {
+ // If we have the input and a position, make the error a bit fancier
+
+ // Prepend some information
+ error += " at position " + position + ": ";
+
+ // Get the input
+ var input = lexer._input;
+ // Insert a combining underscore at the correct position
+ input = input.slice(0, position) + "\u0332" +
+ input.slice(position);
+
+ // Extract some context from the input and add it to the error
+ var begin = Math.max(0, position - 15);
+ var end = position + 15;
+ error += input.slice(begin, end);
+ }
+
+ // Some hackery to make ParseError a prototype of Error
+ // See http://stackoverflow.com/a/8460753
+ var self = new Error(error);
+ self.name = "ParseError";
+ self.__proto__ = ParseError.prototype;
+
+ self.position = position;
+ return self;
+}
+
+// More hackery
+ParseError.prototype.__proto__ = Error.prototype;
+
+module.exports = ParseError;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js
new file mode 100644
index 000000000..aca6cd291
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js
@@ -0,0 +1,798 @@
+/* eslint no-constant-condition:0 */
+var functions = require("./functions");
+var environments = require("./environments");
+var Lexer = require("./Lexer");
+var symbols = require("./symbols");
+var utils = require("./utils");
+
+var parseData = require("./parseData");
+var ParseError = require("./ParseError");
+
+global_str = ""
+
+/**
+ * This file contains the parser used to parse out a TeX expression from the
+ * input. Since TeX isn't context-free, standard parsers don't work particularly
+ * well.
+ *
+ * The strategy of this parser is as such:
+ *
+ * The main functions (the `.parse...` ones) take a position in the current
+ * parse string to parse tokens from. The lexer (found in Lexer.js, stored at
+ * this.lexer) also supports pulling out tokens at arbitrary places. When
+ * individual tokens are needed at a position, the lexer is called to pull out a
+ * token, which is then used.
+ *
+ * The parser has a property called "mode" indicating the mode that
+ * the parser is currently in. Currently it has to be one of "math" or
+ * "text", which denotes whether the current environment is a math-y
+ * one or a text-y one (e.g. inside \text). Currently, this serves to
+ * limit the functions which can be used in text mode.
+ *
+ * The main functions then return an object which contains the useful data that
+ * was parsed at its given point, and a new position at the end of the parsed
+ * data. The main functions can call each other and continue the parsing by
+ * using the returned position as a new starting point.
+ *
+ * There are also extra `.handle...` functions, which pull out some reused
+ * functionality into self-contained functions.
+ *
+ * The earlier functions return ParseNodes.
+ * The later functions (which are called deeper in the parse) sometimes return
+ * ParseFuncOrArgument, which contain a ParseNode as well as some data about
+ * whether the parsed object is a function which is missing some arguments, or a
+ * standalone object which can be used as an argument to another function.
+ */
+
+/**
+ * Main Parser class
+ */
+function Parser(input, settings) {
+ // Make a new lexer
+ this.lexer = new Lexer(input);
+ // Store the settings for use in parsing
+ this.settings = settings;
+}
+
+var ParseNode = parseData.ParseNode;
+
+/**
+ * An initial function (without its arguments), or an argument to a function.
+ * The `result` argument should be a ParseNode.
+ */
+function ParseFuncOrArgument(result, isFunction) {
+ this.result = result;
+ // Is this a function (i.e. is it something defined in functions.js)?
+ this.isFunction = isFunction;
+}
+
+/**
+ * Checks a result to make sure it has the right type, and throws an
+ * appropriate error otherwise.
+ *
+ * @param {boolean=} consume whether to consume the expected token,
+ * defaults to true
+ */
+Parser.prototype.expect = function(text, consume) {
+ if (this.nextToken.text !== text) {
+ throw new ParseError(
+ "Expected '" + text + "', got '" + this.nextToken.text + "'",
+ this.lexer, this.nextToken.position
+ );
+ }
+ if (consume !== false) {
+ this.consume();
+ }
+};
+
+/**
+ * Considers the current look ahead token as consumed,
+ * and fetches the one after that as the new look ahead.
+ */
+Parser.prototype.consume = function() {
+ this.pos = this.nextToken.position;
+
+ global_str = global_str + " " + this.nextToken.text
+ this.nextToken = this.lexer.lex(this.pos, this.mode);
+};
+
+/**
+ * Main parsing function, which parses an entire input.
+ *
+ * @return {?Array.}
+ */
+Parser.prototype.parse = function() {
+ // Try to parse the input
+ this.mode = "math";
+ this.pos = 0;
+ this.nextToken = this.lexer.lex(this.pos, this.mode);
+ var parse = this.parseInput();
+ return parse;
+};
+
+/**
+ * Parses an entire input tree.
+ */
+Parser.prototype.parseInput = function() {
+ // Parse an expression
+ var expression = this.parseExpression(false);
+ // If we succeeded, make sure there's an EOF at the end
+ this.expect("EOF", false);
+ return expression;
+};
+
+var endOfExpression = ["}", "\\end", "\\right", "&", "\\\\", "\\cr"];
+
+/**
+ * Parses an "expression", which is a list of atoms.
+ *
+ * @param {boolean} breakOnInfix Should the parsing stop when we hit infix
+ * nodes? This happens when functions have higher precendence
+ * than infix nodes in implicit parses.
+ *
+ * @param {?string} breakOnToken The token that the expression should end with,
+ * or `null` if something else should end the expression.
+ *
+ * @return {ParseNode}
+ */
+Parser.prototype.parseExpression = function(breakOnInfix, breakOnToken) {
+ var body = [];
+ // Keep adding atoms to the body until we can't parse any more atoms (either
+ // we reached the end, a }, or a \right)
+ while (true) {
+ var lex = this.nextToken;
+ var pos = this.pos;
+ if (endOfExpression.indexOf(lex.text) !== -1) {
+ break;
+ }
+ if (breakOnToken && lex.text === breakOnToken) {
+ break;
+ }
+ var atom = this.parseAtom();
+ if (!atom) {
+ if (!this.settings.throwOnError && lex.text[0] === "\\") {
+ var errorNode = this.handleUnsupportedCmd();
+ body.push(errorNode);
+
+ pos = lex.position;
+ continue;
+ }
+
+ break;
+ }
+ if (breakOnInfix && atom.type === "infix") {
+ // rewind so we can parse the infix atom again
+ this.pos = pos;
+ this.nextToken = lex;
+ break;
+ }
+ body.push(atom);
+ }
+ return this.handleInfixNodes(body);
+};
+
+/**
+ * Rewrites infix operators such as \over with corresponding commands such
+ * as \frac.
+ *
+ * There can only be one infix operator per group. If there's more than one
+ * then the expression is ambiguous. This can be resolved by adding {}.
+ *
+ * @returns {Array}
+ */
+Parser.prototype.handleInfixNodes = function(body) {
+ var overIndex = -1;
+ var funcName;
+
+ for (var i = 0; i < body.length; i++) {
+ var node = body[i];
+ if (node.type === "infix") {
+ if (overIndex !== -1) {
+ throw new ParseError("only one infix operator per group",
+ this.lexer, -1);
+ }
+ overIndex = i;
+ funcName = node.value.replaceWith;
+ }
+ }
+
+ if (overIndex !== -1) {
+ var numerNode;
+ var denomNode;
+
+ var numerBody = body.slice(0, overIndex);
+ var denomBody = body.slice(overIndex + 1);
+
+ if (numerBody.length === 1 && numerBody[0].type === "ordgroup") {
+ numerNode = numerBody[0];
+ } else {
+ numerNode = new ParseNode("ordgroup", numerBody, this.mode);
+ }
+
+ if (denomBody.length === 1 && denomBody[0].type === "ordgroup") {
+ denomNode = denomBody[0];
+ } else {
+ denomNode = new ParseNode("ordgroup", denomBody, this.mode);
+ }
+
+ var value = this.callFunction(
+ funcName, [numerNode, denomNode], null);
+ return [new ParseNode(value.type, value, this.mode)];
+ } else {
+ return body;
+ }
+};
+
+// The greediness of a superscript or subscript
+var SUPSUB_GREEDINESS = 1;
+
+/**
+ * Handle a subscript or superscript with nice errors.
+ */
+Parser.prototype.handleSupSubscript = function(name) {
+ var symbol = this.nextToken.text;
+ var symPos = this.pos;
+ this.consume();
+ var group = this.parseGroup();
+
+ if (!group) {
+ if (!this.settings.throwOnError && this.nextToken.text[0] === "\\") {
+ return this.handleUnsupportedCmd();
+ } else {
+ // throw new ParseError(
+ // "Expected group after '" + symbol + "'",
+ // this.lexer,
+ // symPos + 1
+ // );
+ }
+ } else if (group.isFunction) {
+ // ^ and _ have a greediness, so handle interactions with functions'
+ // greediness
+ var funcGreediness = functions[group.result].greediness;
+ if (funcGreediness > SUPSUB_GREEDINESS) {
+ return this.parseFunction(group);
+ } else {
+ throw new ParseError(
+ "Got function '" + group.result + "' with no arguments " +
+ "as " + name,
+ this.lexer, symPos + 1);
+ }
+ } else {
+ return group.result;
+ }
+};
+
+/**
+ * Converts the textual input of an unsupported command into a text node
+ * contained within a color node whose color is determined by errorColor
+ */
+Parser.prototype.handleUnsupportedCmd = function() {
+ var text = this.nextToken.text;
+ var textordArray = [];
+
+ for (var i = 0; i < text.length; i++) {
+ textordArray.push(new ParseNode("textord", text[i], "text"));
+ }
+
+ var textNode = new ParseNode(
+ "text",
+ {
+ body: textordArray,
+ type: "text",
+ },
+ this.mode);
+
+ var colorNode = new ParseNode(
+ "color",
+ {
+ color: this.settings.errorColor,
+ value: [textNode],
+ type: "color",
+ },
+ this.mode);
+
+ this.consume();
+ return colorNode;
+};
+
+/**
+ * Parses a group with optional super/subscripts.
+ *
+ * @return {?ParseNode}
+ */
+Parser.prototype.parseAtom = function() {
+ // The body of an atom is an implicit group, so that things like
+ // \left(x\right)^2 work correctly.
+ var base = this.parseImplicitGroup();
+
+ // In text mode, we don't have superscripts or subscripts
+ if (this.mode === "text") {
+ return base;
+ }
+
+ // Note that base may be empty (i.e. null) at this point.
+
+ var superscript;
+ var subscript;
+ while (true) {
+ // Lex the first token
+ var lex = this.nextToken;
+
+ if (lex.text === "\\limits" || lex.text === "\\nolimits") {
+ // We got a limit control
+ if (!base || base.type !== "op") {
+ throw new ParseError(
+ "Limit controls must follow a math operator",
+ this.lexer, this.pos);
+ } else {
+ var limits = lex.text === "\\limits";
+ base.value.limits = limits;
+ base.value.alwaysHandleSupSub = true;
+ }
+ this.consume();
+ } else if (lex.text === "^") {
+ // We got a superscript start
+ // if (superscript) {
+ // throw new ParseError(
+ // "Double superscript", this.lexer, this.pos);
+ // }
+ superscript = this.handleSupSubscript("superscript");
+ } else if (lex.text === "_") {
+ // We got a subscript start
+ // if (subscript) {
+ // throw new ParseError(
+ // "Double subscript", this.lexer, this.pos);
+ // }
+ subscript = this.handleSupSubscript("subscript");
+ } else if (lex.text === "'") {
+ // We got a prime
+ var prime = new ParseNode("textord", "\\prime", this.mode);
+
+ // Many primes can be grouped together, so we handle this here
+ var primes = [prime];
+ this.consume();
+ // Keep lexing tokens until we get something that's not a prime
+ while (this.nextToken.text === "'") {
+ // For each one, add another prime to the list
+ primes.push(prime);
+ this.consume();
+ }
+ // Put them into an ordgroup as the superscript
+ superscript = new ParseNode("ordgroup", primes, this.mode);
+ } else {
+ // If it wasn't ^, _, or ', stop parsing super/subscripts
+ break;
+ }
+ }
+
+ if (superscript || subscript) {
+ // If we got either a superscript or subscript, create a supsub
+ return new ParseNode("supsub", {
+ base: base,
+ sup: superscript,
+ sub: subscript,
+ }, this.mode);
+ } else {
+ // Otherwise return the original body
+ return base;
+ }
+};
+
+// A list of the size-changing functions, for use in parseImplicitGroup
+var sizeFuncs = [
+ "\\tiny", "\\scriptsize", "\\footnotesize", "\\small", "\\normalsize",
+ "\\large", "\\Large", "\\LARGE", "\\huge", "\\Huge", "\\textrm", "\\rm", "\\cal",
+ "\\bf", "\\siptstyle", "\\boldmath", "\\it"
+];
+
+// A list of the style-changing functions, for use in parseImplicitGroup
+var styleFuncs = [
+ "\\displaystyle", "\\textstyle", "\\scriptstyle", "\\scriptscriptstyle",
+];
+
+/**
+ * Parses an implicit group, which is a group that starts at the end of a
+ * specified, and ends right before a higher explicit group ends, or at EOL. It
+ * is used for functions that appear to affect the current style, like \Large or
+ * \textrm, where instead of keeping a style we just pretend that there is an
+ * implicit grouping after it until the end of the group. E.g.
+ * small text {\Large large text} small text again
+ * It is also used for \left and \right to get the correct grouping.
+ *
+ * @return {?ParseNode}
+ */
+Parser.prototype.parseImplicitGroup = function() {
+ var start = this.parseSymbol();
+
+ if (start == null) {
+ // If we didn't get anything we handle, fall back to parseFunction
+ return this.parseFunction();
+ }
+
+ var func = start.result;
+ var body;
+ if (func === "\\left") {
+ // If we see a left:
+ // Parse the entire left function (including the delimiter)
+ var left = this.parseFunction(start);
+ // Parse out the implicit body
+ body = this.parseExpression(false);
+ // Check the next token
+ this.expect("\\right", false);
+ var right = this.parseFunction();
+ return new ParseNode("leftright", {
+ body: body,
+ left: left.value.value,
+ right: right.value.value,
+ }, this.mode);
+ } else if (func === "\\begin") {
+ // begin...end is similar to left...right
+ var begin = this.parseFunction(start);
+ var envName = begin.value.name;
+ var name = (begin.value.name + "")
+
+ global_str = global_str.substring(0, global_str.length - (name.length * 2 + 2)) + name + "}"
+
+ if (!environments.hasOwnProperty(envName)) {
+ throw new ParseError(
+ "No such environment: " + envName,
+ this.lexer, begin.value.namepos);
+ }
+ // Build the environment object. Arguments and other information will
+ // be made available to the begin and end methods using properties.
+ var env = environments[envName];
+ var args = this.parseArguments("\\begin{" + envName + "}", env);
+ var context = {
+ mode: this.mode,
+ envName: envName,
+ parser: this,
+ lexer: this.lexer,
+ positions: args.pop(),
+ };
+ var result = env.handler(context, args);
+ this.expect("\\end", false);
+ var end = this.parseFunction();
+
+ var name = (begin.value.name + "")
+
+ global_str = global_str.substring(0, global_str.length - (name.length * 2 + 2)) + name + "}"
+ if (end.value.name !== envName) {
+ throw new ParseError(
+ "Mismatch: \\begin{" + envName + "} matched " +
+ "by \\end{" + end.value.name + "}",
+ this.lexer /* , end.value.namepos */);
+ // TODO: Add position to the above line and adjust test case,
+ // requires #385 to get merged first
+ }
+ result.position = end.position;
+
+ return result;
+
+ } else if (func.value == "\\matrix" || func.value == "\\pmatrix" || func.value == "\\cases") {
+ // if (!environments.hasOwnProperty(envName)) {
+ // throw new ParseError(
+ // "No such environment: " + envName,
+ // this.lexer, begin.value.namepos);
+ // }
+ // Build the environment object. Arguments and other information will
+ // be made available to the begin and end methods using properties.
+
+ envName = func.value.slice(1);
+ var env = environments[envName];
+ // var args = this.parseArguments("\\matrix{", env);
+ this.expect("{", true);
+ var context = {
+ mode: this.mode,
+ envName: envName,
+ parser: this,
+ lexer: this.lexer
+ };
+
+ var result = env.handler(context, {} );
+ // exit();
+ this.expect("}", true);
+ // var end = this.parseFunction();
+ var next = this.nextToken.text;
+ // exit();
+ // console.log(next);
+ // var name = ( + "")
+
+ // global_str = global_str.substring(0, global_str.length - (name.length * 2 + 2)) + name + "}"
+ // result.position = end.position;
+
+ return result;
+
+ } else if (utils.contains(sizeFuncs, func)) {
+ // If we see a sizing function, parse out the implict body
+ body = this.parseExpression(false);
+
+ return new ParseNode("sizing", {
+ // Figure out what size to use based on the list of functions above
+ original: func,
+ size: "size" + (utils.indexOf(sizeFuncs, func) + 1),
+ value: body,
+ }, this.mode);
+ } else if (utils.contains(styleFuncs, func)) {
+ // If we see a styling function, parse out the implict body
+ body = this.parseExpression(true);
+ return new ParseNode("styling", {
+ // Figure out what style to use by pulling out the style from
+ // the function name
+ original: func,
+ style: func.slice(1, func.length - 5),
+ value: body,
+ }, this.mode);
+ } else {
+ // Defer to parseFunction if it's not a function we handle
+ return this.parseFunction(start);
+ }
+};
+
+/**
+ * Parses an entire function, including its base and all of its arguments.
+ * The base might either have been parsed already, in which case
+ * it is provided as an argument, or it's the next group in the input.
+ *
+ * @param {ParseFuncOrArgument=} baseGroup optional as described above
+ * @return {?ParseNode}
+ */
+Parser.prototype.parseFunction = function(baseGroup) {
+ if (!baseGroup) {
+ baseGroup = this.parseGroup();
+ }
+
+ if (baseGroup) {
+ if (baseGroup.isFunction) {
+ var func = baseGroup.result;
+ var funcData = functions[func];
+ if (this.mode === "text" && !funcData.allowedInText) {
+ // throw new ParseError(
+ // "Can't use function '" + func + "' in text mode",
+ // this.lexer, baseGroup.position);
+ }
+
+ var args = this.parseArguments(func, funcData);
+ var result = this.callFunction(func, args, args.pop());
+ return new ParseNode(result.type, result, this.mode);
+ } else {
+ return baseGroup.result;
+ }
+ } else {
+ return null;
+ }
+};
+
+/**
+ * Call a function handler with a suitable context and arguments.
+ */
+Parser.prototype.callFunction = function(name, args, positions) {
+ var context = {
+ funcName: name,
+ parser: this,
+ lexer: this.lexer,
+ positions: positions,
+ };
+ return functions[name].handler(context, args);
+};
+
+/**
+ * Parses the arguments of a function or environment
+ *
+ * @param {string} func "\name" or "\begin{name}"
+ * @param {{numArgs:number,numOptionalArgs:number|undefined}} funcData
+ * @return the array of arguments, with the list of positions as last element
+ */
+Parser.prototype.parseArguments = function(func, funcData) {
+ var totalArgs = funcData.numArgs + funcData.numOptionalArgs;
+ if (totalArgs === 0) {
+ return [[this.pos]];
+ }
+
+ var baseGreediness = funcData.greediness;
+ var positions = [this.pos];
+ var args = [];
+
+ for (var i = 0; i < totalArgs; i++) {
+ var argType = funcData.argTypes && funcData.argTypes[i];
+ var arg;
+ if (i < funcData.numOptionalArgs) {
+ if (argType) {
+ arg = this.parseSpecialGroup(argType, true);
+ } else {
+ arg = this.parseOptionalGroup();
+ }
+ if (!arg) {
+ args.push(null);
+ positions.push(this.pos);
+ continue;
+ }
+ } else {
+ if (argType) {
+ arg = this.parseSpecialGroup(argType);
+ } else {
+ arg = this.parseGroup();
+ }
+ if (!arg) {
+ if (!this.settings.throwOnError &&
+ this.nextToken.text[0] === "\\") {
+ arg = new ParseFuncOrArgument(
+ this.handleUnsupportedCmd(this.nextToken.text),
+ false);
+ } else {
+ throw new ParseError(
+ "Expected group after '" + func + "'",
+ this.lexer, this.pos);
+ }
+ }
+ }
+ var argNode;
+ if (arg.isFunction) {
+ var argGreediness =
+ functions[arg.result].greediness;
+ if (argGreediness > baseGreediness) {
+ argNode = this.parseFunction(arg);
+ } else {
+ // throw new ParseError(
+ // "Got function '" + arg.result + "' as " +
+ // "argument to '" + func + "'",
+ // this.lexer, this.pos - 1);
+ }
+ } else {
+ argNode = arg.result;
+ }
+ args.push(argNode);
+ positions.push(this.pos);
+ }
+
+ args.push(positions);
+
+ return args;
+};
+
+
+/**
+ * Parses a group when the mode is changing. Takes a position, a new mode, and
+ * an outer mode that is used to parse the outside.
+ *
+ * @return {?ParseFuncOrArgument}
+ */
+Parser.prototype.parseSpecialGroup = function(innerMode, optional) {
+ var outerMode = this.mode;
+ // Handle `original` argTypes
+ if (innerMode === "original") {
+ innerMode = outerMode;
+ }
+
+ if (innerMode === "color" || innerMode === "size") {
+ // color and size modes are special because they should have braces and
+ // should only lex a single symbol inside
+ var openBrace = this.nextToken;
+ if (optional && openBrace.text !== "[") {
+ // optional arguments should return null if they don't exist
+ return null;
+ }
+ // The call to expect will lex the token after the '{' in inner mode
+ this.mode = innerMode;
+ this.expect(optional ? "[" : "{");
+ var inner = this.nextToken;
+ this.mode = outerMode;
+ var data;
+ if (innerMode === "color") {
+ data = inner.text;
+ } else {
+ data = inner.data;
+ }
+ this.consume(); // consume the token stored in inner
+ this.expect(optional ? "]" : "}");
+ return new ParseFuncOrArgument(
+ new ParseNode(innerMode, data, outerMode),
+ false);
+ } else if (innerMode === "text") {
+ // text mode is special because it should ignore the whitespace before
+ // it
+ var whitespace = this.lexer.lex(this.pos, "whitespace");
+ this.pos = whitespace.position;
+ }
+
+ // By the time we get here, innerMode is one of "text" or "math".
+ // We switch the mode of the parser, recurse, then restore the old mode.
+ this.mode = innerMode;
+ this.nextToken = this.lexer.lex(this.pos, innerMode);
+ var res;
+ if (optional) {
+ res = this.parseOptionalGroup();
+ } else {
+ res = this.parseGroup();
+ }
+ this.mode = outerMode;
+ this.nextToken = this.lexer.lex(this.pos, outerMode);
+ return res;
+};
+
+/**
+ * Parses a group, which is either a single nucleus (like "x") or an expression
+ * in braces (like "{x+y}")
+ *
+ * @return {?ParseFuncOrArgument}
+ */
+Parser.prototype.parseGroup = function() {
+ // Try to parse an open brace
+ if (this.nextToken.text === "{") {
+ // If we get a brace, parse an expression
+ this.consume();
+ var expression = this.parseExpression(false);
+ // Make sure we get a close brace
+ this.expect("}");
+ return new ParseFuncOrArgument(
+ new ParseNode("ordgroup", expression, this.mode),
+ false);
+ } else {
+ // Otherwise, just return a nucleus
+ return this.parseSymbol();
+ }
+};
+
+/**
+ * Parses a group, which is an expression in brackets (like "[x+y]")
+ *
+ * @return {?ParseFuncOrArgument}
+ */
+Parser.prototype.parseOptionalGroup = function() {
+ // Try to parse an open bracket
+ if (this.nextToken.text === "[") {
+ // If we get a brace, parse an expression
+ this.consume();
+ var expression = this.parseExpression(false, "]");
+ // Make sure we get a close bracket
+ this.expect("]");
+ return new ParseFuncOrArgument(
+ new ParseNode("ordgroup", expression, this.mode),
+ false);
+ } else {
+ // Otherwise, return null,
+ return null;
+ }
+};
+
+/**
+ * Parse a single symbol out of the string. Here, we handle both the functions
+ * we have defined, as well as the single character symbols
+ *
+ * @return {?ParseFuncOrArgument}
+ */
+Parser.prototype.parseSymbol = function() {
+ var nucleus = this.nextToken;
+
+ if (functions[nucleus.text]) {
+ this.consume();
+ // If there exists a function with this name, we return the function and
+ // say that it is a function.
+ return new ParseFuncOrArgument(
+ nucleus.text,
+ true);
+ } else if (symbols[this.mode][nucleus.text]) {
+ this.consume();
+ // Otherwise if this is a no-argument function, find the type it
+ // corresponds to in the symbols map
+ return new ParseFuncOrArgument(
+ new ParseNode(symbols[this.mode][nucleus.text].group,
+ nucleus.text, this.mode),
+ false);
+ } else if (nucleus.text == "EOF" || nucleus.text == "{") {
+ return null;
+
+ } else {
+ this.consume();
+ // console.error(nucleus);
+ return new ParseFuncOrArgument(
+ new ParseNode(symbols["math"]["\\sigma"].group,
+ nucleus.text, this.mode),
+ false);
+ // console.log(nucleus.text);
+ // return null;
+ }
+};
+
+Parser.prototype.ParseNode = ParseNode;
+
+module.exports = Parser;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Settings.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Settings.js
new file mode 100644
index 000000000..644014504
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Settings.js
@@ -0,0 +1,28 @@
+/**
+ * This is a module for storing settings passed into KaTeX. It correctly handles
+ * default settings.
+ */
+
+/**
+ * Helper function for getting a default value if the value is undefined
+ */
+function get(option, defaultValue) {
+ return option === undefined ? defaultValue : option;
+}
+
+/**
+ * The main Settings object
+ *
+ * The current options stored are:
+ * - displayMode: Whether the expression should be typeset by default in
+ * textstyle or displaystyle (default false)
+ */
+function Settings(options) {
+ // allow null options
+ options = options || {};
+ this.displayMode = get(options.displayMode, false);
+ this.throwOnError = get(options.throwOnError, true);
+ this.errorColor = get(options.errorColor, "#cc0000");
+}
+
+module.exports = Settings;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Style.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Style.js
new file mode 100644
index 000000000..10e5ef2cc
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Style.js
@@ -0,0 +1,126 @@
+/**
+ * This file contains information and classes for the various kinds of styles
+ * used in TeX. It provides a generic `Style` class, which holds information
+ * about a specific style. It then provides instances of all the different kinds
+ * of styles possible, and provides functions to move between them and get
+ * information about them.
+ */
+
+/**
+ * The main style class. Contains a unique id for the style, a size (which is
+ * the same for cramped and uncramped version of a style), a cramped flag, and a
+ * size multiplier, which gives the size difference between a style and
+ * textstyle.
+ */
+function Style(id, size, multiplier, cramped) {
+ this.id = id;
+ this.size = size;
+ this.cramped = cramped;
+ this.sizeMultiplier = multiplier;
+}
+
+/**
+ * Get the style of a superscript given a base in the current style.
+ */
+Style.prototype.sup = function() {
+ return styles[sup[this.id]];
+};
+
+/**
+ * Get the style of a subscript given a base in the current style.
+ */
+Style.prototype.sub = function() {
+ return styles[sub[this.id]];
+};
+
+/**
+ * Get the style of a fraction numerator given the fraction in the current
+ * style.
+ */
+Style.prototype.fracNum = function() {
+ return styles[fracNum[this.id]];
+};
+
+/**
+ * Get the style of a fraction denominator given the fraction in the current
+ * style.
+ */
+Style.prototype.fracDen = function() {
+ return styles[fracDen[this.id]];
+};
+
+/**
+ * Get the cramped version of a style (in particular, cramping a cramped style
+ * doesn't change the style).
+ */
+Style.prototype.cramp = function() {
+ return styles[cramp[this.id]];
+};
+
+/**
+ * HTML class name, like "displaystyle cramped"
+ */
+Style.prototype.cls = function() {
+ return sizeNames[this.size] + (this.cramped ? " cramped" : " uncramped");
+};
+
+/**
+ * HTML Reset class name, like "reset-textstyle"
+ */
+Style.prototype.reset = function() {
+ return resetNames[this.size];
+};
+
+// IDs of the different styles
+var D = 0;
+var Dc = 1;
+var T = 2;
+var Tc = 3;
+var S = 4;
+var Sc = 5;
+var SS = 6;
+var SSc = 7;
+
+// String names for the different sizes
+var sizeNames = [
+ "displaystyle textstyle",
+ "textstyle",
+ "scriptstyle",
+ "scriptscriptstyle",
+];
+
+// Reset names for the different sizes
+var resetNames = [
+ "reset-textstyle",
+ "reset-textstyle",
+ "reset-scriptstyle",
+ "reset-scriptscriptstyle",
+];
+
+// Instances of the different styles
+var styles = [
+ new Style(D, 0, 1.0, false),
+ new Style(Dc, 0, 1.0, true),
+ new Style(T, 1, 1.0, false),
+ new Style(Tc, 1, 1.0, true),
+ new Style(S, 2, 0.7, false),
+ new Style(Sc, 2, 0.7, true),
+ new Style(SS, 3, 0.5, false),
+ new Style(SSc, 3, 0.5, true),
+];
+
+// Lookup tables for switching from one style to another
+var sup = [S, Sc, S, Sc, SS, SSc, SS, SSc];
+var sub = [Sc, Sc, Sc, Sc, SSc, SSc, SSc, SSc];
+var fracNum = [T, Tc, S, Sc, SS, SSc, SS, SSc];
+var fracDen = [Tc, Tc, Sc, Sc, SSc, SSc, SSc, SSc];
+var cramp = [Dc, Dc, Tc, Tc, Sc, Sc, SSc, SSc];
+
+// We only export some of the styles. Also, we don't export the `Style` class so
+// no more styles can be generated.
+module.exports = {
+ DISPLAY: styles[D],
+ TEXT: styles[T],
+ SCRIPT: styles[S],
+ SCRIPTSCRIPT: styles[SS],
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildCommon.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildCommon.js
new file mode 100644
index 000000000..b60e1860a
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildCommon.js
@@ -0,0 +1,450 @@
+/* eslint no-console:0 */
+/**
+ * This module contains general functions that can be used for building
+ * different kinds of domTree nodes in a consistent manner.
+ */
+
+var domTree = require("./domTree");
+var fontMetrics = require("./fontMetrics");
+var symbols = require("./symbols");
+var utils = require("./utils");
+
+var greekCapitals = [
+ "\\Gamma",
+ "\\Delta",
+ "\\Theta",
+ "\\Lambda",
+ "\\Xi",
+ "\\Pi",
+ "\\Sigma",
+ "\\Upsilon",
+ "\\Phi",
+ "\\Psi",
+ "\\Omega",
+];
+
+var dotlessLetters = [
+ "\u0131", // dotless i, \imath
+ "\u0237", // dotless j, \jmath
+];
+
+/**
+ * Makes a symbolNode after translation via the list of symbols in symbols.js.
+ * Correctly pulls out metrics for the character, and optionally takes a list of
+ * classes to be attached to the node.
+ */
+var makeSymbol = function(value, style, mode, color, classes) {
+ // Replace the value with its replaced value from symbol.js
+ if (symbols[mode][value] && symbols[mode][value].replace) {
+ value = symbols[mode][value].replace;
+ }
+
+ var metrics = fontMetrics.getCharacterMetrics(value, style);
+
+ var symbolNode;
+ if (metrics) {
+ symbolNode = new domTree.symbolNode(
+ value, metrics.height, metrics.depth, metrics.italic, metrics.skew,
+ classes);
+ } else {
+ // TODO(emily): Figure out a good way to only print this in development
+ typeof console !== "undefined" && console.warn(
+ "No character metrics for '" + value + "' in style '" +
+ style + "'");
+ symbolNode = new domTree.symbolNode(value, 0, 0, 0, 0, classes);
+ }
+
+ if (color) {
+ symbolNode.style.color = color;
+ }
+
+ return symbolNode;
+};
+
+/**
+ * Makes a symbol in Main-Regular or AMS-Regular.
+ * Used for rel, bin, open, close, inner, and punct.
+ */
+var mathsym = function(value, mode, color, classes) {
+ // Decide what font to render the symbol in by its entry in the symbols
+ // table.
+ // Have a special case for when the value = \ because the \ is used as a
+ // textord in unsupported command errors but cannot be parsed as a regular
+ // text ordinal and is therefore not present as a symbol in the symbols
+ // table for text
+ if (value === "\\" || symbols[mode][value].font === "main") {
+ return makeSymbol(value, "Main-Regular", mode, color, classes);
+ } else {
+ return makeSymbol(
+ value, "AMS-Regular", mode, color, classes.concat(["amsrm"]));
+ }
+};
+
+/**
+ * Makes a symbol in the default font for mathords and textords.
+ */
+var mathDefault = function(value, mode, color, classes, type) {
+ if (type === "mathord") {
+ return mathit(value, mode, color, classes);
+ } else if (type === "textord") {
+ return makeSymbol(
+ value, "Main-Regular", mode, color, classes.concat(["mathrm"]));
+ } else {
+ throw new Error("unexpected type: " + type + " in mathDefault");
+ }
+};
+
+/**
+ * Makes a symbol in the italic math font.
+ */
+var mathit = function(value, mode, color, classes) {
+ if (/[0-9]/.test(value.charAt(0)) ||
+ // glyphs for \imath and \jmath do not exist in Math-Italic so we
+ // need to use Main-Italic instead
+ utils.contains(dotlessLetters, value) ||
+ utils.contains(greekCapitals, value)) {
+ return makeSymbol(
+ value, "Main-Italic", mode, color, classes.concat(["mainit"]));
+ } else {
+ return makeSymbol(
+ value, "Math-Italic", mode, color, classes.concat(["mathit"]));
+ }
+};
+
+/**
+ * Makes either a mathord or textord in the correct font and color.
+ */
+var makeOrd = function(group, options, type) {
+ var mode = group.mode;
+ var value = group.value;
+ if (symbols[mode][value] && symbols[mode][value].replace) {
+ value = symbols[mode][value].replace;
+ }
+
+ var classes = ["mord"];
+ var color = options.getColor();
+
+ var font = options.font;
+ if (font) {
+ if (font === "mathit" || utils.contains(dotlessLetters, value)) {
+ return mathit(value, mode, color, classes);
+ } else {
+ var fontName = fontMap[font].fontName;
+ if (fontMetrics.getCharacterMetrics(value, fontName)) {
+ return makeSymbol(
+ value, fontName, mode, color, classes.concat([font]));
+ } else {
+ return mathDefault(value, mode, color, classes, type);
+ }
+ }
+ } else {
+ return mathDefault(value, mode, color, classes, type);
+ }
+};
+
+/**
+ * Calculate the height, depth, and maxFontSize of an element based on its
+ * children.
+ */
+var sizeElementFromChildren = function(elem) {
+ var height = 0;
+ var depth = 0;
+ var maxFontSize = 0;
+
+ if (elem.children) {
+ for (var i = 0; i < elem.children.length; i++) {
+ if (elem.children[i].height > height) {
+ height = elem.children[i].height;
+ }
+ if (elem.children[i].depth > depth) {
+ depth = elem.children[i].depth;
+ }
+ if (elem.children[i].maxFontSize > maxFontSize) {
+ maxFontSize = elem.children[i].maxFontSize;
+ }
+ }
+ }
+
+ elem.height = height;
+ elem.depth = depth;
+ elem.maxFontSize = maxFontSize;
+};
+
+/**
+ * Makes a span with the given list of classes, list of children, and color.
+ */
+var makeSpan = function(classes, children, color) {
+ var span = new domTree.span(classes, children);
+
+ sizeElementFromChildren(span);
+
+ if (color) {
+ span.style.color = color;
+ }
+
+ return span;
+};
+
+/**
+ * Makes a document fragment with the given list of children.
+ */
+var makeFragment = function(children) {
+ var fragment = new domTree.documentFragment(children);
+
+ sizeElementFromChildren(fragment);
+
+ return fragment;
+};
+
+/**
+ * Makes an element placed in each of the vlist elements to ensure that each
+ * element has the same max font size. To do this, we create a zero-width space
+ * with the correct font size.
+ */
+var makeFontSizer = function(options, fontSize) {
+ var fontSizeInner = makeSpan([], [new domTree.symbolNode("\u200b")]);
+ fontSizeInner.style.fontSize =
+ (fontSize / options.style.sizeMultiplier) + "em";
+
+ var fontSizer = makeSpan(
+ ["fontsize-ensurer", "reset-" + options.size, "size5"],
+ [fontSizeInner]);
+
+ return fontSizer;
+};
+
+/**
+ * Makes a vertical list by stacking elements and kerns on top of each other.
+ * Allows for many different ways of specifying the positioning method.
+ *
+ * Arguments:
+ * - children: A list of child or kern nodes to be stacked on top of each other
+ * (i.e. the first element will be at the bottom, and the last at
+ * the top). Element nodes are specified as
+ * {type: "elem", elem: node}
+ * while kern nodes are specified as
+ * {type: "kern", size: size}
+ * - positionType: The method by which the vlist should be positioned. Valid
+ * values are:
+ * - "individualShift": The children list only contains elem
+ * nodes, and each node contains an extra
+ * "shift" value of how much it should be
+ * shifted (note that shifting is always
+ * moving downwards). positionData is
+ * ignored.
+ * - "top": The positionData specifies the topmost point of
+ * the vlist (note this is expected to be a height,
+ * so positive values move up)
+ * - "bottom": The positionData specifies the bottommost point
+ * of the vlist (note this is expected to be a
+ * depth, so positive values move down
+ * - "shift": The vlist will be positioned such that its
+ * baseline is positionData away from the baseline
+ * of the first child. Positive values move
+ * downwards.
+ * - "firstBaseline": The vlist will be positioned such that
+ * its baseline is aligned with the
+ * baseline of the first child.
+ * positionData is ignored. (this is
+ * equivalent to "shift" with
+ * positionData=0)
+ * - positionData: Data used in different ways depending on positionType
+ * - options: An Options object
+ *
+ */
+var makeVList = function(children, positionType, positionData, options) {
+ var depth;
+ var currPos;
+ var i;
+ if (positionType === "individualShift") {
+ var oldChildren = children;
+ children = [oldChildren[0]];
+
+ // Add in kerns to the list of children to get each element to be
+ // shifted to the correct specified shift
+ depth = -oldChildren[0].shift - oldChildren[0].elem.depth;
+ currPos = depth;
+ for (i = 1; i < oldChildren.length; i++) {
+ var diff = -oldChildren[i].shift - currPos -
+ oldChildren[i].elem.depth;
+ var size = diff -
+ (oldChildren[i - 1].elem.height +
+ oldChildren[i - 1].elem.depth);
+
+ currPos = currPos + diff;
+
+ children.push({type: "kern", size: size});
+ children.push(oldChildren[i]);
+ }
+ } else if (positionType === "top") {
+ // We always start at the bottom, so calculate the bottom by adding up
+ // all the sizes
+ var bottom = positionData;
+ for (i = 0; i < children.length; i++) {
+ if (children[i].type === "kern") {
+ bottom -= children[i].size;
+ } else {
+ bottom -= children[i].elem.height + children[i].elem.depth;
+ }
+ }
+ depth = bottom;
+ } else if (positionType === "bottom") {
+ depth = -positionData;
+ } else if (positionType === "shift") {
+ depth = -children[0].elem.depth - positionData;
+ } else if (positionType === "firstBaseline") {
+ depth = -children[0].elem.depth;
+ } else {
+ depth = 0;
+ }
+
+ // Make the fontSizer
+ var maxFontSize = 0;
+ for (i = 0; i < children.length; i++) {
+ if (children[i].type === "elem") {
+ maxFontSize = Math.max(maxFontSize, children[i].elem.maxFontSize);
+ }
+ }
+ var fontSizer = makeFontSizer(options, maxFontSize);
+
+ // Create a new list of actual children at the correct offsets
+ var realChildren = [];
+ currPos = depth;
+ for (i = 0; i < children.length; i++) {
+ if (children[i].type === "kern") {
+ currPos += children[i].size;
+ } else {
+ var child = children[i].elem;
+
+ var shift = -child.depth - currPos;
+ currPos += child.height + child.depth;
+
+ var childWrap = makeSpan([], [fontSizer, child]);
+ childWrap.height -= shift;
+ childWrap.depth += shift;
+ childWrap.style.top = shift + "em";
+
+ realChildren.push(childWrap);
+ }
+ }
+
+ // Add in an element at the end with no offset to fix the calculation of
+ // baselines in some browsers (namely IE, sometimes safari)
+ var baselineFix = makeSpan(
+ ["baseline-fix"], [fontSizer, new domTree.symbolNode("\u200b")]);
+ realChildren.push(baselineFix);
+
+ var vlist = makeSpan(["vlist"], realChildren);
+ // Fix the final height and depth, in case there were kerns at the ends
+ // since the makeSpan calculation won't take that in to account.
+ vlist.height = Math.max(currPos, vlist.height);
+ vlist.depth = Math.max(-depth, vlist.depth);
+ return vlist;
+};
+
+// A table of size -> font size for the different sizing functions
+var sizingMultiplier = {
+ size1: 0.5,
+ size2: 0.7,
+ size3: 0.8,
+ size4: 0.9,
+ size5: 1.0,
+ size6: 1.2,
+ size7: 1.44,
+ size8: 1.73,
+ size9: 2.07,
+ size10: 2.49,
+};
+
+// A map of spacing functions to their attributes, like size and corresponding
+// CSS class
+var spacingFunctions = {
+ "\\qquad": {
+ size: "2em",
+ className: "qquad",
+ },
+ "\\quad": {
+ size: "1em",
+ className: "quad",
+ },
+ "\\enspace": {
+ size: "0.5em",
+ className: "enspace",
+ },
+ "\\;": {
+ size: "0.277778em",
+ className: "thickspace",
+ },
+ "\\:": {
+ size: "0.22222em",
+ className: "mediumspace",
+ },
+ "\\,": {
+ size: "0.16667em",
+ className: "thinspace",
+ },
+ "\\!": {
+ size: "-0.16667em",
+ className: "negativethinspace",
+ },
+};
+
+/**
+ * Maps TeX font commands to objects containing:
+ * - variant: string used for "mathvariant" attribute in buildMathML.js
+ * - fontName: the "style" parameter to fontMetrics.getCharacterMetrics
+ */
+// A map between tex font commands an MathML mathvariant attribute values
+var fontMap = {
+ // styles
+ "mathbf": {
+ variant: "bold",
+ fontName: "Main-Bold",
+ },
+ "mathrm": {
+ variant: "normal",
+ fontName: "Main-Regular",
+ },
+
+ // "mathit" is missing because it requires the use of two fonts: Main-Italic
+ // and Math-Italic. This is handled by a special case in makeOrd which ends
+ // up calling mathit.
+
+ // families
+ "mathbb": {
+ variant: "double-struck",
+ fontName: "AMS-Regular",
+ },
+ "mathcal": {
+ variant: "script",
+ fontName: "Caligraphic-Regular",
+ },
+ "mathfrak": {
+ variant: "fraktur",
+ fontName: "Fraktur-Regular",
+ },
+ "mathscr": {
+ variant: "script",
+ fontName: "Script-Regular",
+ },
+ "mathsf": {
+ variant: "sans-serif",
+ fontName: "SansSerif-Regular",
+ },
+ "mathtt": {
+ variant: "monospace",
+ fontName: "Typewriter-Regular",
+ },
+};
+
+module.exports = {
+ fontMap: fontMap,
+ makeSymbol: makeSymbol,
+ mathsym: mathsym,
+ makeSpan: makeSpan,
+ makeFragment: makeFragment,
+ makeVList: makeVList,
+ makeOrd: makeOrd,
+ sizingMultiplier: sizingMultiplier,
+ spacingFunctions: spacingFunctions,
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildHTML.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildHTML.js
new file mode 100644
index 000000000..42c33a6c4
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildHTML.js
@@ -0,0 +1,1402 @@
+/* eslint no-console:0 */
+/**
+ * This file does the main work of building a domTree structure from a parse
+ * tree. The entry point is the `buildHTML` function, which takes a parse tree.
+ * Then, the buildExpression, buildGroup, and various groupTypes functions are
+ * called, to produce a final HTML tree.
+ */
+
+var ParseError = require("./ParseError");
+var Style = require("./Style");
+
+var buildCommon = require("./buildCommon");
+var delimiter = require("./delimiter");
+var domTree = require("./domTree");
+var fontMetrics = require("./fontMetrics");
+var utils = require("./utils");
+
+var makeSpan = buildCommon.makeSpan;
+
+/**
+ * Take a list of nodes, build them in order, and return a list of the built
+ * nodes. This function handles the `prev` node correctly, and passes the
+ * previous element from the list as the prev of the next element.
+ */
+var buildExpression = function(expression, options, prev) {
+ var groups = [];
+ for (var i = 0; i < expression.length; i++) {
+ var group = expression[i];
+ groups.push(buildGroup(group, options, prev));
+ prev = group;
+ }
+ return groups;
+};
+
+// List of types used by getTypeOfGroup,
+// see https://github.com/Khan/KaTeX/wiki/Examining-TeX#group-types
+var groupToType = {
+ mathord: "mord",
+ textord: "mord",
+ bin: "mbin",
+ rel: "mrel",
+ text: "mord",
+ open: "mopen",
+ close: "mclose",
+ inner: "minner",
+ genfrac: "mord",
+ array: "mord",
+ spacing: "mord",
+ punct: "mpunct",
+ ordgroup: "mord",
+ op: "mop",
+ katex: "mord",
+ overline: "mord",
+ underline: "mord",
+ rule: "mord",
+ leftright: "minner",
+ sqrt: "mord",
+ accent: "mord",
+};
+
+/**
+ * Gets the final math type of an expression, given its group type. This type is
+ * used to determine spacing between elements, and affects bin elements by
+ * causing them to change depending on what types are around them. This type
+ * must be attached to the outermost node of an element as a CSS class so that
+ * spacing with its surrounding elements works correctly.
+ *
+ * Some elements can be mapped one-to-one from group type to math type, and
+ * those are listed in the `groupToType` table.
+ *
+ * Others (usually elements that wrap around other elements) often have
+ * recursive definitions, and thus call `getTypeOfGroup` on their inner
+ * elements.
+ */
+var getTypeOfGroup = function(group) {
+ if (group == null) {
+ // Like when typesetting $^3$
+ return groupToType.mathord;
+ } else if (group.type === "supsub") {
+ return getTypeOfGroup(group.value.base);
+ } else if (group.type === "llap" || group.type === "rlap") {
+ return getTypeOfGroup(group.value);
+ } else if (group.type === "color") {
+ return getTypeOfGroup(group.value.value);
+ } else if (group.type === "sizing") {
+ return getTypeOfGroup(group.value.value);
+ } else if (group.type === "styling") {
+ return getTypeOfGroup(group.value.value);
+ } else if (group.type === "delimsizing") {
+ return groupToType[group.value.delimType];
+ } else {
+ return groupToType[group.type];
+ }
+};
+
+/**
+ * Sometimes, groups perform special rules when they have superscripts or
+ * subscripts attached to them. This function lets the `supsub` group know that
+ * its inner element should handle the superscripts and subscripts instead of
+ * handling them itself.
+ */
+var shouldHandleSupSub = function(group, options) {
+ if (!group) {
+ return false;
+ } else if (group.type === "op") {
+ // Operators handle supsubs differently when they have limits
+ // (e.g. `\displaystyle\sum_2^3`)
+ return group.value.limits &&
+ (options.style.size === Style.DISPLAY.size ||
+ group.value.alwaysHandleSupSub);
+ } else if (group.type === "accent") {
+ return isCharacterBox(group.value.base);
+ } else {
+ return null;
+ }
+};
+
+/**
+ * Sometimes we want to pull out the innermost element of a group. In most
+ * cases, this will just be the group itself, but when ordgroups and colors have
+ * a single element, we want to pull that out.
+ */
+var getBaseElem = function(group) {
+ if (!group) {
+ return false;
+ } else if (group.type === "ordgroup") {
+ if (group.value.length === 1) {
+ return getBaseElem(group.value[0]);
+ } else {
+ return group;
+ }
+ } else if (group.type === "color") {
+ if (group.value.value.length === 1) {
+ return getBaseElem(group.value.value[0]);
+ } else {
+ return group;
+ }
+ } else {
+ return group;
+ }
+};
+
+/**
+ * TeXbook algorithms often reference "character boxes", which are simply groups
+ * with a single character in them. To decide if something is a character box,
+ * we find its innermost group, and see if it is a single character.
+ */
+var isCharacterBox = function(group) {
+ var baseElem = getBaseElem(group);
+
+ // These are all they types of groups which hold single characters
+ return baseElem.type === "mathord" ||
+ baseElem.type === "textord" ||
+ baseElem.type === "bin" ||
+ baseElem.type === "rel" ||
+ baseElem.type === "inner" ||
+ baseElem.type === "open" ||
+ baseElem.type === "close" ||
+ baseElem.type === "punct";
+};
+
+var makeNullDelimiter = function(options) {
+ return makeSpan([
+ "sizing", "reset-" + options.size, "size5",
+ options.style.reset(), Style.TEXT.cls(),
+ "nulldelimiter",
+ ]);
+};
+
+/**
+ * This is a map of group types to the function used to handle that type.
+ * Simpler types come at the beginning, while complicated types come afterwards.
+ */
+var groupTypes = {};
+
+groupTypes.mathord = function(group, options, prev) {
+ return buildCommon.makeOrd(group, options, "mathord");
+};
+
+groupTypes.textord = function(group, options, prev) {
+ return buildCommon.makeOrd(group, options, "textord");
+};
+
+groupTypes.bin = function(group, options, prev) {
+ var className = "mbin";
+ // Pull out the most recent element. Do some special handling to find
+ // things at the end of a \color group. Note that we don't use the same
+ // logic for ordgroups (which count as ords).
+ var prevAtom = prev;
+ while (prevAtom && prevAtom.type === "color") {
+ var atoms = prevAtom.value.value;
+ prevAtom = atoms[atoms.length - 1];
+ }
+ // See TeXbook pg. 442-446, Rules 5 and 6, and the text before Rule 19.
+ // Here, we determine whether the bin should turn into an ord. We
+ // currently only apply Rule 5.
+ if (!prev || utils.contains(["mbin", "mopen", "mrel", "mop", "mpunct"],
+ getTypeOfGroup(prevAtom))) {
+ group.type = "textord";
+ className = "mord";
+ }
+
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), [className]);
+};
+
+groupTypes.rel = function(group, options, prev) {
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), ["mrel"]);
+};
+
+groupTypes.open = function(group, options, prev) {
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), ["mopen"]);
+};
+
+groupTypes.close = function(group, options, prev) {
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), ["mclose"]);
+};
+
+groupTypes.inner = function(group, options, prev) {
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), ["minner"]);
+};
+
+groupTypes.punct = function(group, options, prev) {
+ return buildCommon.mathsym(
+ group.value, group.mode, options.getColor(), ["mpunct"]);
+};
+
+groupTypes.ordgroup = function(group, options, prev) {
+ return makeSpan(
+ ["mord", options.style.cls()],
+ buildExpression(group.value, options.reset())
+ );
+};
+
+groupTypes.text = function(group, options, prev) {
+ return makeSpan(["text", "mord", options.style.cls()],
+ buildExpression(group.value.body, options.reset()));
+};
+
+groupTypes.color = function(group, options, prev) {
+ var elements = buildExpression(
+ group.value.value,
+ options.withColor(group.value.color),
+ prev
+ );
+
+ // \color isn't supposed to affect the type of the elements it contains.
+ // To accomplish this, we wrap the results in a fragment, so the inner
+ // elements will be able to directly interact with their neighbors. For
+ // example, `\color{red}{2 +} 3` has the same spacing as `2 + 3`
+ return new buildCommon.makeFragment(elements);
+};
+
+groupTypes.supsub = function(group, options, prev) {
+ // Superscript and subscripts are handled in the TeXbook on page
+ // 445-446, rules 18(a-f).
+
+ // Here is where we defer to the inner group if it should handle
+ // superscripts and subscripts itself.
+ if (shouldHandleSupSub(group.value.base, options)) {
+ return groupTypes[group.value.base.type](group, options, prev);
+ }
+
+ var base = buildGroup(group.value.base, options.reset());
+ var supmid;
+ var submid;
+ var sup;
+ var sub;
+
+ if (group.value.sup) {
+ sup = buildGroup(group.value.sup,
+ options.withStyle(options.style.sup()));
+ supmid = makeSpan(
+ [options.style.reset(), options.style.sup().cls()], [sup]);
+ }
+
+ if (group.value.sub) {
+ sub = buildGroup(group.value.sub,
+ options.withStyle(options.style.sub()));
+ submid = makeSpan(
+ [options.style.reset(), options.style.sub().cls()], [sub]);
+ }
+
+ // Rule 18a
+ var supShift;
+ var subShift;
+ if (isCharacterBox(group.value.base)) {
+ supShift = 0;
+ subShift = 0;
+ } else {
+ supShift = base.height - fontMetrics.metrics.supDrop;
+ subShift = base.depth + fontMetrics.metrics.subDrop;
+ }
+
+ // Rule 18c
+ var minSupShift;
+ if (options.style === Style.DISPLAY) {
+ minSupShift = fontMetrics.metrics.sup1;
+ } else if (options.style.cramped) {
+ minSupShift = fontMetrics.metrics.sup3;
+ } else {
+ minSupShift = fontMetrics.metrics.sup2;
+ }
+
+ // scriptspace is a font-size-independent size, so scale it
+ // appropriately
+ var multiplier = Style.TEXT.sizeMultiplier *
+ options.style.sizeMultiplier;
+ var scriptspace =
+ (0.5 / fontMetrics.metrics.ptPerEm) / multiplier + "em";
+
+ var supsub;
+ if (!group.value.sup) {
+ // Rule 18b
+ subShift = Math.max(
+ subShift, fontMetrics.metrics.sub1,
+ sub.height - 0.8 * fontMetrics.metrics.xHeight);
+
+ supsub = buildCommon.makeVList([
+ {type: "elem", elem: submid},
+ ], "shift", subShift, options);
+
+ supsub.children[0].style.marginRight = scriptspace;
+
+ // Subscripts shouldn't be shifted by the base's italic correction.
+ // Account for that by shifting the subscript back the appropriate
+ // amount. Note we only do this when the base is a single symbol.
+ if (base instanceof domTree.symbolNode) {
+ supsub.children[0].style.marginLeft = -base.italic + "em";
+ }
+ } else if (!group.value.sub) {
+ // Rule 18c, d
+ supShift = Math.max(supShift, minSupShift,
+ sup.depth + 0.25 * fontMetrics.metrics.xHeight);
+
+ supsub = buildCommon.makeVList([
+ {type: "elem", elem: supmid},
+ ], "shift", -supShift, options);
+
+ supsub.children[0].style.marginRight = scriptspace;
+ } else {
+ supShift = Math.max(
+ supShift, minSupShift,
+ sup.depth + 0.25 * fontMetrics.metrics.xHeight);
+ subShift = Math.max(subShift, fontMetrics.metrics.sub2);
+
+ var ruleWidth = fontMetrics.metrics.defaultRuleThickness;
+
+ // Rule 18e
+ if ((supShift - sup.depth) - (sub.height - subShift) <
+ 4 * ruleWidth) {
+ subShift = 4 * ruleWidth - (supShift - sup.depth) + sub.height;
+ var psi = 0.8 * fontMetrics.metrics.xHeight -
+ (supShift - sup.depth);
+ if (psi > 0) {
+ supShift += psi;
+ subShift -= psi;
+ }
+ }
+
+ supsub = buildCommon.makeVList([
+ {type: "elem", elem: submid, shift: subShift},
+ {type: "elem", elem: supmid, shift: -supShift},
+ ], "individualShift", null, options);
+
+ // See comment above about subscripts not being shifted
+ if (base instanceof domTree.symbolNode) {
+ supsub.children[0].style.marginLeft = -base.italic + "em";
+ }
+
+ supsub.children[0].style.marginRight = scriptspace;
+ supsub.children[1].style.marginRight = scriptspace;
+ }
+
+ return makeSpan([getTypeOfGroup(group.value.base)],
+ [base, supsub]);
+};
+
+groupTypes.genfrac = function(group, options, prev) {
+ // Fractions are handled in the TeXbook on pages 444-445, rules 15(a-e).
+ // Figure out what style this fraction should be in based on the
+ // function used
+ var fstyle = options.style;
+ if (group.value.size === "display") {
+ fstyle = Style.DISPLAY;
+ } else if (group.value.size === "text") {
+ fstyle = Style.TEXT;
+ }
+
+ var nstyle = fstyle.fracNum();
+ var dstyle = fstyle.fracDen();
+
+ var numer = buildGroup(group.value.numer, options.withStyle(nstyle));
+ var numerreset = makeSpan([fstyle.reset(), nstyle.cls()], [numer]);
+
+ var denom = buildGroup(group.value.denom, options.withStyle(dstyle));
+ var denomreset = makeSpan([fstyle.reset(), dstyle.cls()], [denom]);
+
+ var ruleWidth;
+ if (group.value.hasBarLine) {
+ ruleWidth = fontMetrics.metrics.defaultRuleThickness /
+ options.style.sizeMultiplier;
+ } else {
+ ruleWidth = 0;
+ }
+
+ // Rule 15b
+ var numShift;
+ var clearance;
+ var denomShift;
+ if (fstyle.size === Style.DISPLAY.size) {
+ numShift = fontMetrics.metrics.num1;
+ if (ruleWidth > 0) {
+ clearance = 3 * ruleWidth;
+ } else {
+ clearance = 7 * fontMetrics.metrics.defaultRuleThickness;
+ }
+ denomShift = fontMetrics.metrics.denom1;
+ } else {
+ if (ruleWidth > 0) {
+ numShift = fontMetrics.metrics.num2;
+ clearance = ruleWidth;
+ } else {
+ numShift = fontMetrics.metrics.num3;
+ clearance = 3 * fontMetrics.metrics.defaultRuleThickness;
+ }
+ denomShift = fontMetrics.metrics.denom2;
+ }
+
+ var frac;
+ if (ruleWidth === 0) {
+ // Rule 15c
+ var candiateClearance =
+ (numShift - numer.depth) - (denom.height - denomShift);
+ if (candiateClearance < clearance) {
+ numShift += 0.5 * (clearance - candiateClearance);
+ denomShift += 0.5 * (clearance - candiateClearance);
+ }
+
+ frac = buildCommon.makeVList([
+ {type: "elem", elem: denomreset, shift: denomShift},
+ {type: "elem", elem: numerreset, shift: -numShift},
+ ], "individualShift", null, options);
+ } else {
+ // Rule 15d
+ var axisHeight = fontMetrics.metrics.axisHeight;
+
+ if ((numShift - numer.depth) - (axisHeight + 0.5 * ruleWidth) <
+ clearance) {
+ numShift +=
+ clearance - ((numShift - numer.depth) -
+ (axisHeight + 0.5 * ruleWidth));
+ }
+
+ if ((axisHeight - 0.5 * ruleWidth) - (denom.height - denomShift) <
+ clearance) {
+ denomShift +=
+ clearance - ((axisHeight - 0.5 * ruleWidth) -
+ (denom.height - denomShift));
+ }
+
+ var mid = makeSpan(
+ [options.style.reset(), Style.TEXT.cls(), "frac-line"]);
+ // Manually set the height of the line because its height is
+ // created in CSS
+ mid.height = ruleWidth;
+
+ var midShift = -(axisHeight - 0.5 * ruleWidth);
+
+ frac = buildCommon.makeVList([
+ {type: "elem", elem: denomreset, shift: denomShift},
+ {type: "elem", elem: mid, shift: midShift},
+ {type: "elem", elem: numerreset, shift: -numShift},
+ ], "individualShift", null, options);
+ }
+
+ // Since we manually change the style sometimes (with \dfrac or \tfrac),
+ // account for the possible size change here.
+ frac.height *= fstyle.sizeMultiplier / options.style.sizeMultiplier;
+ frac.depth *= fstyle.sizeMultiplier / options.style.sizeMultiplier;
+
+ // Rule 15e
+ var delimSize;
+ if (fstyle.size === Style.DISPLAY.size) {
+ delimSize = fontMetrics.metrics.delim1;
+ } else {
+ delimSize = fontMetrics.metrics.getDelim2(fstyle);
+ }
+
+ var leftDelim;
+ var rightDelim;
+ if (group.value.leftDelim == null) {
+ leftDelim = makeNullDelimiter(options);
+ } else {
+ leftDelim = delimiter.customSizedDelim(
+ group.value.leftDelim, delimSize, true,
+ options.withStyle(fstyle), group.mode);
+ }
+ if (group.value.rightDelim == null) {
+ rightDelim = makeNullDelimiter(options);
+ } else {
+ rightDelim = delimiter.customSizedDelim(
+ group.value.rightDelim, delimSize, true,
+ options.withStyle(fstyle), group.mode);
+ }
+
+ return makeSpan(
+ ["mord", options.style.reset(), fstyle.cls()],
+ [leftDelim, makeSpan(["mfrac"], [frac]), rightDelim],
+ options.getColor());
+};
+
+groupTypes.array = function(group, options, prev) {
+ var r;
+ var c;
+ var nr = group.value.body.length;
+ var nc = 0;
+ var body = new Array(nr);
+
+ // Horizontal spacing
+ var pt = 1 / fontMetrics.metrics.ptPerEm;
+ var arraycolsep = 5 * pt; // \arraycolsep in article.cls
+
+ // Vertical spacing
+ var baselineskip = 12 * pt; // see size10.clo
+ // Default \arraystretch from lttab.dtx
+ // TODO(gagern): may get redefined once we have user-defined macros
+ var arraystretch = utils.deflt(group.value.arraystretch, 1);
+ var arrayskip = arraystretch * baselineskip;
+ var arstrutHeight = 0.7 * arrayskip; // \strutbox in ltfsstrc.dtx and
+ var arstrutDepth = 0.3 * arrayskip; // \@arstrutbox in lttab.dtx
+
+ var totalHeight = 0;
+ for (r = 0; r < group.value.body.length; ++r) {
+ var inrow = group.value.body[r];
+ var height = arstrutHeight; // \@array adds an \@arstrut
+ var depth = arstrutDepth; // to each tow (via the template)
+
+ if (nc < inrow.length) {
+ nc = inrow.length;
+ }
+
+ var outrow = new Array(inrow.length);
+ for (c = 0; c < inrow.length; ++c) {
+ var elt = buildGroup(inrow[c], options);
+ if (depth < elt.depth) {
+ depth = elt.depth;
+ }
+ if (height < elt.height) {
+ height = elt.height;
+ }
+ outrow[c] = elt;
+ }
+
+ var gap = 0;
+ if (group.value.rowGaps[r]) {
+ gap = group.value.rowGaps[r].value;
+ switch (gap.unit) {
+ case "em":
+ gap = gap.number;
+ break;
+ case "ex":
+ gap = gap.number * fontMetrics.metrics.emPerEx;
+ break;
+ default:
+ console.error("Can't handle unit " + gap.unit);
+ gap = 0;
+ }
+ if (gap > 0) { // \@argarraycr
+ gap += arstrutDepth;
+ if (depth < gap) {
+ depth = gap; // \@xargarraycr
+ }
+ gap = 0;
+ }
+ }
+
+ outrow.height = height;
+ outrow.depth = depth;
+ totalHeight += height;
+ outrow.pos = totalHeight;
+ totalHeight += depth + gap; // \@yargarraycr
+ body[r] = outrow;
+ }
+
+ var offset = totalHeight / 2 + fontMetrics.metrics.axisHeight;
+ var colDescriptions = group.value.cols || [];
+ var cols = [];
+ var colSep;
+ var colDescrNum;
+ for (c = 0, colDescrNum = 0;
+ // Continue while either there are more columns or more column
+ // descriptions, so trailing separators don't get lost.
+ c < nc || colDescrNum < colDescriptions.length;
+ ++c, ++colDescrNum) {
+
+ var colDescr = colDescriptions[colDescrNum] || {};
+
+ var firstSeparator = true;
+ while (colDescr.type === "separator") {
+ // If there is more than one separator in a row, add a space
+ // between them.
+ if (!firstSeparator) {
+ colSep = makeSpan(["arraycolsep"], []);
+ colSep.style.width =
+ fontMetrics.metrics.doubleRuleSep + "em";
+ cols.push(colSep);
+ }
+
+ if (colDescr.separator === "|") {
+ var separator = makeSpan(
+ ["vertical-separator"],
+ []);
+ separator.style.height = totalHeight + "em";
+ separator.style.verticalAlign =
+ -(totalHeight - offset) + "em";
+
+ cols.push(separator);
+ } else {
+ throw new ParseError(
+ "Invalid separator type: " + colDescr.separator);
+ }
+
+ colDescrNum++;
+ colDescr = colDescriptions[colDescrNum] || {};
+ firstSeparator = false;
+ }
+
+ if (c >= nc) {
+ continue;
+ }
+
+ var sepwidth;
+ if (c > 0 || group.value.hskipBeforeAndAfter) {
+ sepwidth = utils.deflt(colDescr.pregap, arraycolsep);
+ if (sepwidth !== 0) {
+ colSep = makeSpan(["arraycolsep"], []);
+ colSep.style.width = sepwidth + "em";
+ cols.push(colSep);
+ }
+ }
+
+ var col = [];
+ for (r = 0; r < nr; ++r) {
+ var row = body[r];
+ var elem = row[c];
+ if (!elem) {
+ continue;
+ }
+ var shift = row.pos - offset;
+ elem.depth = row.depth;
+ elem.height = row.height;
+ col.push({type: "elem", elem: elem, shift: shift});
+ }
+
+ col = buildCommon.makeVList(col, "individualShift", null, options);
+ col = makeSpan(
+ ["col-align-" + (colDescr.align || "c")],
+ [col]);
+ cols.push(col);
+
+ if (c < nc - 1 || group.value.hskipBeforeAndAfter) {
+ sepwidth = utils.deflt(colDescr.postgap, arraycolsep);
+ if (sepwidth !== 0) {
+ colSep = makeSpan(["arraycolsep"], []);
+ colSep.style.width = sepwidth + "em";
+ cols.push(colSep);
+ }
+ }
+ }
+ body = makeSpan(["mtable"], cols);
+ return makeSpan(["mord"], [body], options.getColor());
+};
+
+groupTypes.spacing = function(group, options, prev) {
+ if (group.value === "\\ " || group.value === "\\space" ||
+ group.value === " " || group.value === "~") {
+ // Spaces are generated by adding an actual space. Each of these
+ // things has an entry in the symbols table, so these will be turned
+ // into appropriate outputs.
+ return makeSpan(
+ ["mord", "mspace"],
+ [buildCommon.mathsym(group.value, group.mode)]
+ );
+ } else {
+ // Other kinds of spaces are of arbitrary width. We use CSS to
+ // generate these.
+ return makeSpan(
+ ["mord", "mspace",
+ buildCommon.spacingFunctions[group.value].className]);
+ }
+};
+
+groupTypes.llap = function(group, options, prev) {
+ var inner = makeSpan(
+ ["inner"], [buildGroup(group.value.body, options.reset())]);
+ var fix = makeSpan(["fix"], []);
+ return makeSpan(
+ ["llap", options.style.cls()], [inner, fix]);
+};
+
+groupTypes.rlap = function(group, options, prev) {
+ var inner = makeSpan(
+ ["inner"], [buildGroup(group.value.body, options.reset())]);
+ var fix = makeSpan(["fix"], []);
+ return makeSpan(
+ ["rlap", options.style.cls()], [inner, fix]);
+};
+
+groupTypes.op = function(group, options, prev) {
+ // Operators are handled in the TeXbook pg. 443-444, rule 13(a).
+ var supGroup;
+ var subGroup;
+ var hasLimits = false;
+ if (group.type === "supsub" ) {
+ // If we have limits, supsub will pass us its group to handle. Pull
+ // out the superscript and subscript and set the group to the op in
+ // its base.
+ supGroup = group.value.sup;
+ subGroup = group.value.sub;
+ group = group.value.base;
+ hasLimits = true;
+ }
+
+ // Most operators have a large successor symbol, but these don't.
+ var noSuccessor = [
+ "\\smallint",
+ ];
+
+ var large = false;
+ if (options.style.size === Style.DISPLAY.size &&
+ group.value.symbol &&
+ !utils.contains(noSuccessor, group.value.body)) {
+
+ // Most symbol operators get larger in displaystyle (rule 13)
+ large = true;
+ }
+
+ var base;
+ var baseShift = 0;
+ var slant = 0;
+ if (group.value.symbol) {
+ // If this is a symbol, create the symbol.
+ var style = large ? "Size2-Regular" : "Size1-Regular";
+ base = buildCommon.makeSymbol(
+ group.value.body, style, "math", options.getColor(),
+ ["op-symbol", large ? "large-op" : "small-op", "mop"]);
+
+ // Shift the symbol so its center lies on the axis (rule 13). It
+ // appears that our fonts have the centers of the symbols already
+ // almost on the axis, so these numbers are very small. Note we
+ // don't actually apply this here, but instead it is used either in
+ // the vlist creation or separately when there are no limits.
+ baseShift = (base.height - base.depth) / 2 -
+ fontMetrics.metrics.axisHeight *
+ options.style.sizeMultiplier;
+
+ // The slant of the symbol is just its italic correction.
+ slant = base.italic;
+ } else {
+ // Otherwise, this is a text operator. Build the text from the
+ // operator's name.
+ // TODO(emily): Add a space in the middle of some of these
+ // operators, like \limsup
+ var output = [];
+ for (var i = 1; i < group.value.body.length; i++) {
+ output.push(buildCommon.mathsym(group.value.body[i], group.mode));
+ }
+ base = makeSpan(["mop"], output, options.getColor());
+ }
+
+ if (hasLimits) {
+ // IE 8 clips \int if it is in a display: inline-block. We wrap it
+ // in a new span so it is an inline, and works.
+ base = makeSpan([], [base]);
+
+ var supmid;
+ var supKern;
+ var submid;
+ var subKern;
+ // We manually have to handle the superscripts and subscripts. This,
+ // aside from the kern calculations, is copied from supsub.
+ if (supGroup) {
+ var sup = buildGroup(
+ supGroup, options.withStyle(options.style.sup()));
+ supmid = makeSpan(
+ [options.style.reset(), options.style.sup().cls()], [sup]);
+
+ supKern = Math.max(
+ fontMetrics.metrics.bigOpSpacing1,
+ fontMetrics.metrics.bigOpSpacing3 - sup.depth);
+ }
+
+ if (subGroup) {
+ var sub = buildGroup(
+ subGroup, options.withStyle(options.style.sub()));
+ submid = makeSpan(
+ [options.style.reset(), options.style.sub().cls()],
+ [sub]);
+
+ subKern = Math.max(
+ fontMetrics.metrics.bigOpSpacing2,
+ fontMetrics.metrics.bigOpSpacing4 - sub.height);
+ }
+
+ // Build the final group as a vlist of the possible subscript, base,
+ // and possible superscript.
+ var finalGroup;
+ var top;
+ var bottom;
+ if (!supGroup) {
+ top = base.height - baseShift;
+
+ finalGroup = buildCommon.makeVList([
+ {type: "kern", size: fontMetrics.metrics.bigOpSpacing5},
+ {type: "elem", elem: submid},
+ {type: "kern", size: subKern},
+ {type: "elem", elem: base},
+ ], "top", top, options);
+
+ // Here, we shift the limits by the slant of the symbol. Note
+ // that we are supposed to shift the limits by 1/2 of the slant,
+ // but since we are centering the limits adding a full slant of
+ // margin will shift by 1/2 that.
+ finalGroup.children[0].style.marginLeft = -slant + "em";
+ } else if (!subGroup) {
+ bottom = base.depth + baseShift;
+
+ finalGroup = buildCommon.makeVList([
+ {type: "elem", elem: base},
+ {type: "kern", size: supKern},
+ {type: "elem", elem: supmid},
+ {type: "kern", size: fontMetrics.metrics.bigOpSpacing5},
+ ], "bottom", bottom, options);
+
+ // See comment above about slants
+ finalGroup.children[1].style.marginLeft = slant + "em";
+ } else if (!supGroup && !subGroup) {
+ // This case probably shouldn't occur (this would mean the
+ // supsub was sending us a group with no superscript or
+ // subscript) but be safe.
+ return base;
+ } else {
+ bottom = fontMetrics.metrics.bigOpSpacing5 +
+ submid.height + submid.depth +
+ subKern +
+ base.depth + baseShift;
+
+ finalGroup = buildCommon.makeVList([
+ {type: "kern", size: fontMetrics.metrics.bigOpSpacing5},
+ {type: "elem", elem: submid},
+ {type: "kern", size: subKern},
+ {type: "elem", elem: base},
+ {type: "kern", size: supKern},
+ {type: "elem", elem: supmid},
+ {type: "kern", size: fontMetrics.metrics.bigOpSpacing5},
+ ], "bottom", bottom, options);
+
+ // See comment above about slants
+ finalGroup.children[0].style.marginLeft = -slant + "em";
+ finalGroup.children[2].style.marginLeft = slant + "em";
+ }
+
+ return makeSpan(["mop", "op-limits"], [finalGroup]);
+ } else {
+ if (group.value.symbol) {
+ base.style.top = baseShift + "em";
+ }
+
+ return base;
+ }
+};
+
+groupTypes.katex = function(group, options, prev) {
+ // The KaTeX logo. The offsets for the K and a were chosen to look
+ // good, but the offsets for the T, E, and X were taken from the
+ // definition of \TeX in TeX (see TeXbook pg. 356)
+ var k = makeSpan(
+ ["k"], [buildCommon.mathsym("K", group.mode)]);
+ var a = makeSpan(
+ ["a"], [buildCommon.mathsym("A", group.mode)]);
+
+ a.height = (a.height + 0.2) * 0.75;
+ a.depth = (a.height - 0.2) * 0.75;
+
+ var t = makeSpan(
+ ["t"], [buildCommon.mathsym("T", group.mode)]);
+ var e = makeSpan(
+ ["e"], [buildCommon.mathsym("E", group.mode)]);
+
+ e.height = (e.height - 0.2155);
+ e.depth = (e.depth + 0.2155);
+
+ var x = makeSpan(
+ ["x"], [buildCommon.mathsym("X", group.mode)]);
+
+ return makeSpan(
+ ["katex-logo", "mord"], [k, a, t, e, x], options.getColor());
+};
+
+groupTypes.overline = function(group, options, prev) {
+ // Overlines are handled in the TeXbook pg 443, Rule 9.
+
+ // Build the inner group in the cramped style.
+ var innerGroup = buildGroup(group.value.body,
+ options.withStyle(options.style.cramp()));
+
+ var ruleWidth = fontMetrics.metrics.defaultRuleThickness /
+ options.style.sizeMultiplier;
+
+ // Create the line above the body
+ var line = makeSpan(
+ [options.style.reset(), Style.TEXT.cls(), "overline-line"]);
+ line.height = ruleWidth;
+ line.maxFontSize = 1.0;
+
+ // Generate the vlist, with the appropriate kerns
+ var vlist = buildCommon.makeVList([
+ {type: "elem", elem: innerGroup},
+ {type: "kern", size: 3 * ruleWidth},
+ {type: "elem", elem: line},
+ {type: "kern", size: ruleWidth},
+ ], "firstBaseline", null, options);
+
+ return makeSpan(["overline", "mord"], [vlist], options.getColor());
+};
+
+groupTypes.underline = function(group, options, prev) {
+ // Underlines are handled in the TeXbook pg 443, Rule 10.
+
+ // Build the inner group.
+ var innerGroup = buildGroup(group.value.body, options);
+
+ var ruleWidth = fontMetrics.metrics.defaultRuleThickness /
+ options.style.sizeMultiplier;
+
+ // Create the line above the body
+ var line = makeSpan(
+ [options.style.reset(), Style.TEXT.cls(), "underline-line"]);
+ line.height = ruleWidth;
+ line.maxFontSize = 1.0;
+
+ // Generate the vlist, with the appropriate kerns
+ var vlist = buildCommon.makeVList([
+ {type: "kern", size: ruleWidth},
+ {type: "elem", elem: line},
+ {type: "kern", size: 3 * ruleWidth},
+ {type: "elem", elem: innerGroup},
+ ], "top", innerGroup.height, options);
+
+ return makeSpan(["underline", "mord"], [vlist], options.getColor());
+};
+
+groupTypes.sqrt = function(group, options, prev) {
+ // Square roots are handled in the TeXbook pg. 443, Rule 11.
+
+ // First, we do the same steps as in overline to build the inner group
+ // and line
+ var inner = buildGroup(group.value.body,
+ options.withStyle(options.style.cramp()));
+
+ var ruleWidth = fontMetrics.metrics.defaultRuleThickness /
+ options.style.sizeMultiplier;
+
+ var line = makeSpan(
+ [options.style.reset(), Style.TEXT.cls(), "sqrt-line"], [],
+ options.getColor());
+ line.height = ruleWidth;
+ line.maxFontSize = 1.0;
+
+ var phi = ruleWidth;
+ if (options.style.id < Style.TEXT.id) {
+ phi = fontMetrics.metrics.xHeight;
+ }
+
+ // Calculate the clearance between the body and line
+ var lineClearance = ruleWidth + phi / 4;
+
+ var innerHeight =
+ (inner.height + inner.depth) * options.style.sizeMultiplier;
+ var minDelimiterHeight = innerHeight + lineClearance + ruleWidth;
+
+ // Create a \surd delimiter of the required minimum size
+ var delim = makeSpan(["sqrt-sign"], [
+ delimiter.customSizedDelim("\\surd", minDelimiterHeight,
+ false, options, group.mode)],
+ options.getColor());
+
+ var delimDepth = (delim.height + delim.depth) - ruleWidth;
+
+ // Adjust the clearance based on the delimiter size
+ if (delimDepth > inner.height + inner.depth + lineClearance) {
+ lineClearance =
+ (lineClearance + delimDepth - inner.height - inner.depth) / 2;
+ }
+
+ // Shift the delimiter so that its top lines up with the top of the line
+ var delimShift = -(inner.height + lineClearance + ruleWidth) + delim.height;
+ delim.style.top = delimShift + "em";
+ delim.height -= delimShift;
+ delim.depth += delimShift;
+
+ // We add a special case here, because even when `inner` is empty, we
+ // still get a line. So, we use a simple heuristic to decide if we
+ // should omit the body entirely. (note this doesn't work for something
+ // like `\sqrt{\rlap{x}}`, but if someone is doing that they deserve for
+ // it not to work.
+ var body;
+ if (inner.height === 0 && inner.depth === 0) {
+ body = makeSpan();
+ } else {
+ body = buildCommon.makeVList([
+ {type: "elem", elem: inner},
+ {type: "kern", size: lineClearance},
+ {type: "elem", elem: line},
+ {type: "kern", size: ruleWidth},
+ ], "firstBaseline", null, options);
+ }
+
+ if (!group.value.index) {
+ return makeSpan(["sqrt", "mord"], [delim, body]);
+ } else {
+ // Handle the optional root index
+
+ // The index is always in scriptscript style
+ var root = buildGroup(
+ group.value.index,
+ options.withStyle(Style.SCRIPTSCRIPT));
+ var rootWrap = makeSpan(
+ [options.style.reset(), Style.SCRIPTSCRIPT.cls()],
+ [root]);
+
+ // Figure out the height and depth of the inner part
+ var innerRootHeight = Math.max(delim.height, body.height);
+ var innerRootDepth = Math.max(delim.depth, body.depth);
+
+ // The amount the index is shifted by. This is taken from the TeX
+ // source, in the definition of `\r@@t`.
+ var toShift = 0.6 * (innerRootHeight - innerRootDepth);
+
+ // Build a VList with the superscript shifted up correctly
+ var rootVList = buildCommon.makeVList(
+ [{type: "elem", elem: rootWrap}],
+ "shift", -toShift, options);
+ // Add a class surrounding it so we can add on the appropriate
+ // kerning
+ var rootVListWrap = makeSpan(["root"], [rootVList]);
+
+ return makeSpan(["sqrt", "mord"], [rootVListWrap, delim, body]);
+ }
+};
+
+groupTypes.sizing = function(group, options, prev) {
+ // Handle sizing operators like \Huge. Real TeX doesn't actually allow
+ // these functions inside of math expressions, so we do some special
+ // handling.
+ var inner = buildExpression(group.value.value,
+ options.withSize(group.value.size), prev);
+
+ var span = makeSpan(["mord"],
+ [makeSpan(["sizing", "reset-" + options.size, group.value.size,
+ options.style.cls()],
+ inner)]);
+
+ // Calculate the correct maxFontSize manually
+ var fontSize = buildCommon.sizingMultiplier[group.value.size];
+ span.maxFontSize = fontSize * options.style.sizeMultiplier;
+
+ return span;
+};
+
+groupTypes.styling = function(group, options, prev) {
+ // Style changes are handled in the TeXbook on pg. 442, Rule 3.
+
+ // Figure out what style we're changing to.
+ var style = {
+ "display": Style.DISPLAY,
+ "text": Style.TEXT,
+ "script": Style.SCRIPT,
+ "scriptscript": Style.SCRIPTSCRIPT,
+ };
+
+ var newStyle = style[group.value.style];
+
+ // Build the inner expression in the new style.
+ var inner = buildExpression(
+ group.value.value, options.withStyle(newStyle), prev);
+
+ return makeSpan([options.style.reset(), newStyle.cls()], inner);
+};
+
+groupTypes.font = function(group, options, prev) {
+ var font = group.value.font;
+ return buildGroup(group.value.body, options.withFont(font), prev);
+};
+
+groupTypes.delimsizing = function(group, options, prev) {
+ var delim = group.value.value;
+
+ if (delim === ".") {
+ // Empty delimiters still count as elements, even though they don't
+ // show anything.
+ return makeSpan([groupToType[group.value.delimType]]);
+ }
+
+ // Use delimiter.sizedDelim to generate the delimiter.
+ return makeSpan(
+ [groupToType[group.value.delimType]],
+ [delimiter.sizedDelim(
+ delim, group.value.size, options, group.mode)]);
+};
+
+groupTypes.leftright = function(group, options, prev) {
+ // Build the inner expression
+ var inner = buildExpression(group.value.body, options.reset());
+
+ var innerHeight = 0;
+ var innerDepth = 0;
+
+ // Calculate its height and depth
+ for (var i = 0; i < inner.length; i++) {
+ innerHeight = Math.max(inner[i].height, innerHeight);
+ innerDepth = Math.max(inner[i].depth, innerDepth);
+ }
+
+ // The size of delimiters is the same, regardless of what style we are
+ // in. Thus, to correctly calculate the size of delimiter we need around
+ // a group, we scale down the inner size based on the size.
+ innerHeight *= options.style.sizeMultiplier;
+ innerDepth *= options.style.sizeMultiplier;
+
+ var leftDelim;
+ if (group.value.left === ".") {
+ // Empty delimiters in \left and \right make null delimiter spaces.
+ leftDelim = makeNullDelimiter(options);
+ } else {
+ // Otherwise, use leftRightDelim to generate the correct sized
+ // delimiter.
+ leftDelim = delimiter.leftRightDelim(
+ group.value.left, innerHeight, innerDepth, options,
+ group.mode);
+ }
+ // Add it to the beginning of the expression
+ inner.unshift(leftDelim);
+
+ var rightDelim;
+ // Same for the right delimiter
+ if (group.value.right === ".") {
+ rightDelim = makeNullDelimiter(options);
+ } else {
+ rightDelim = delimiter.leftRightDelim(
+ group.value.right, innerHeight, innerDepth, options,
+ group.mode);
+ }
+ // Add it to the end of the expression.
+ inner.push(rightDelim);
+
+ return makeSpan(
+ ["minner", options.style.cls()], inner, options.getColor());
+};
+
+groupTypes.rule = function(group, options, prev) {
+ // Make an empty span for the rule
+ var rule = makeSpan(["mord", "rule"], [], options.getColor());
+
+ // Calculate the shift, width, and height of the rule, and account for units
+ var shift = 0;
+ if (group.value.shift) {
+ shift = group.value.shift.number;
+ if (group.value.shift.unit === "ex") {
+ shift *= fontMetrics.metrics.xHeight;
+ }
+ }
+
+ var width = group.value.width.number;
+ if (group.value.width.unit === "ex") {
+ width *= fontMetrics.metrics.xHeight;
+ }
+
+ var height = group.value.height.number;
+ if (group.value.height.unit === "ex") {
+ height *= fontMetrics.metrics.xHeight;
+ }
+
+ // The sizes of rules are absolute, so make it larger if we are in a
+ // smaller style.
+ shift /= options.style.sizeMultiplier;
+ width /= options.style.sizeMultiplier;
+ height /= options.style.sizeMultiplier;
+
+ // Style the rule to the right size
+ rule.style.borderRightWidth = width + "em";
+ rule.style.borderTopWidth = height + "em";
+ rule.style.bottom = shift + "em";
+
+ // Record the height and width
+ rule.width = width;
+ rule.height = height + shift;
+ rule.depth = -shift;
+
+ return rule;
+};
+
+groupTypes.accent = function(group, options, prev) {
+ // Accents are handled in the TeXbook pg. 443, rule 12.
+ var base = group.value.base;
+
+ var supsubGroup;
+ if (group.type === "supsub") {
+ // If our base is a character box, and we have superscripts and
+ // subscripts, the supsub will defer to us. In particular, we want
+ // to attach the superscripts and subscripts to the inner body (so
+ // that the position of the superscripts and subscripts won't be
+ // affected by the height of the accent). We accomplish this by
+ // sticking the base of the accent into the base of the supsub, and
+ // rendering that, while keeping track of where the accent is.
+
+ // The supsub group is the group that was passed in
+ var supsub = group;
+ // The real accent group is the base of the supsub group
+ group = supsub.value.base;
+ // The character box is the base of the accent group
+ base = group.value.base;
+ // Stick the character box into the base of the supsub group
+ supsub.value.base = base;
+
+ // Rerender the supsub group with its new base, and store that
+ // result.
+ supsubGroup = buildGroup(
+ supsub, options.reset(), prev);
+ }
+
+ // Build the base group
+ var body = buildGroup(
+ base, options.withStyle(options.style.cramp()));
+
+ // Calculate the skew of the accent. This is based on the line "If the
+ // nucleus is not a single character, let s = 0; otherwise set s to the
+ // kern amount for the nucleus followed by the \skewchar of its font."
+ // Note that our skew metrics are just the kern between each character
+ // and the skewchar.
+ var skew;
+ if (isCharacterBox(base)) {
+ // If the base is a character box, then we want the skew of the
+ // innermost character. To do that, we find the innermost character:
+ var baseChar = getBaseElem(base);
+ // Then, we render its group to get the symbol inside it
+ var baseGroup = buildGroup(
+ baseChar, options.withStyle(options.style.cramp()));
+ // Finally, we pull the skew off of the symbol.
+ skew = baseGroup.skew;
+ // Note that we now throw away baseGroup, because the layers we
+ // removed with getBaseElem might contain things like \color which
+ // we can't get rid of.
+ // TODO(emily): Find a better way to get the skew
+ } else {
+ skew = 0;
+ }
+
+ // calculate the amount of space between the body and the accent
+ var clearance = Math.min(body.height, fontMetrics.metrics.xHeight);
+
+ // Build the accent
+ var accent = buildCommon.makeSymbol(
+ group.value.accent, "Main-Regular", "math", options.getColor());
+ // Remove the italic correction of the accent, because it only serves to
+ // shift the accent over to a place we don't want.
+ accent.italic = 0;
+
+ // The \vec character that the fonts use is a combining character, and
+ // thus shows up much too far to the left. To account for this, we add a
+ // specific class which shifts the accent over to where we want it.
+ // TODO(emily): Fix this in a better way, like by changing the font
+ var vecClass = group.value.accent === "\\vec" ? "accent-vec" : null;
+
+ var accentBody = makeSpan(["accent-body", vecClass], [
+ makeSpan([], [accent])]);
+
+ accentBody = buildCommon.makeVList([
+ {type: "elem", elem: body},
+ {type: "kern", size: -clearance},
+ {type: "elem", elem: accentBody},
+ ], "firstBaseline", null, options);
+
+ // Shift the accent over by the skew. Note we shift by twice the skew
+ // because we are centering the accent, so by adding 2*skew to the left,
+ // we shift it to the right by 1*skew.
+ accentBody.children[1].style.marginLeft = 2 * skew + "em";
+
+ var accentWrap = makeSpan(["mord", "accent"], [accentBody]);
+
+ if (supsubGroup) {
+ // Here, we replace the "base" child of the supsub with our newly
+ // generated accent.
+ supsubGroup.children[0] = accentWrap;
+
+ // Since we don't rerun the height calculation after replacing the
+ // accent, we manually recalculate height.
+ supsubGroup.height = Math.max(accentWrap.height, supsubGroup.height);
+
+ // Accents should always be ords, even when their innards are not.
+ supsubGroup.classes[0] = "mord";
+
+ return supsubGroup;
+ } else {
+ return accentWrap;
+ }
+};
+
+groupTypes.phantom = function(group, options, prev) {
+ var elements = buildExpression(
+ group.value.value,
+ options.withPhantom(),
+ prev
+ );
+
+ // \phantom isn't supposed to affect the elements it contains.
+ // See "color" for more details.
+ return new buildCommon.makeFragment(elements);
+};
+
+/**
+ * buildGroup is the function that takes a group and calls the correct groupType
+ * function for it. It also handles the interaction of size and style changes
+ * between parents and children.
+ */
+var buildGroup = function(group, options, prev) {
+ if (!group) {
+ return makeSpan();
+ }
+
+ if (groupTypes[group.type]) {
+ // Call the groupTypes function
+ var groupNode = groupTypes[group.type](group, options, prev);
+ var multiplier;
+
+ // If the style changed between the parent and the current group,
+ // account for the size difference
+ if (options.style !== options.parentStyle) {
+ multiplier = options.style.sizeMultiplier /
+ options.parentStyle.sizeMultiplier;
+
+ groupNode.height *= multiplier;
+ groupNode.depth *= multiplier;
+ }
+
+ // If the size changed between the parent and the current group, account
+ // for that size difference.
+ if (options.size !== options.parentSize) {
+ multiplier = buildCommon.sizingMultiplier[options.size] /
+ buildCommon.sizingMultiplier[options.parentSize];
+
+ groupNode.height *= multiplier;
+ groupNode.depth *= multiplier;
+ }
+
+ return groupNode;
+ } else {
+ throw new ParseError(
+ "Got group of unknown type: '" + group.type + "'");
+ }
+};
+
+/**
+ * Take an entire parse tree, and build it into an appropriate set of HTML
+ * nodes.
+ */
+var buildHTML = function(tree, options) {
+ // buildExpression is destructive, so we need to make a clone
+ // of the incoming tree so that it isn't accidentally changed
+ tree = JSON.parse(JSON.stringify(tree));
+
+ // Build the expression contained in the tree
+ var expression = buildExpression(tree, options);
+ var body = makeSpan(["base", options.style.cls()], expression);
+
+ // Add struts, which ensure that the top of the HTML element falls at the
+ // height of the expression, and the bottom of the HTML element falls at the
+ // depth of the expression.
+ var topStrut = makeSpan(["strut"]);
+ var bottomStrut = makeSpan(["strut", "bottom"]);
+
+ topStrut.style.height = body.height + "em";
+ bottomStrut.style.height = (body.height + body.depth) + "em";
+ // We'd like to use `vertical-align: top` but in IE 9 this lowers the
+ // baseline of the box to the bottom of this strut (instead staying in the
+ // normal place) so we use an absolute value for vertical-align instead
+ bottomStrut.style.verticalAlign = -body.depth + "em";
+
+ // Wrap the struts and body together
+ var htmlNode = makeSpan(["katex-html"], [topStrut, bottomStrut, body]);
+
+ htmlNode.setAttribute("aria-hidden", "true");
+
+ return htmlNode;
+};
+
+module.exports = buildHTML;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildMathML.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildMathML.js
new file mode 100644
index 000000000..7bf38e86a
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/buildMathML.js
@@ -0,0 +1,533 @@
+/**
+ * This file converts a parse tree into a cooresponding MathML tree. The main
+ * entry point is the `buildMathML` function, which takes a parse tree from the
+ * parser.
+ */
+
+var buildCommon = require("./buildCommon");
+var fontMetrics = require("./fontMetrics");
+var mathMLTree = require("./mathMLTree");
+var ParseError = require("./ParseError");
+var symbols = require("./symbols");
+var utils = require("./utils");
+
+var makeSpan = buildCommon.makeSpan;
+var fontMap = buildCommon.fontMap;
+
+/**
+ * Takes a symbol and converts it into a MathML text node after performing
+ * optional replacement from symbols.js.
+ */
+var makeText = function(text, mode) {
+ if (symbols[mode][text] && symbols[mode][text].replace) {
+ text = symbols[mode][text].replace;
+ }
+
+ return new mathMLTree.TextNode(text);
+};
+
+/**
+ * Returns the math variant as a string or null if none is required.
+ */
+var getVariant = function(group, options) {
+ var font = options.font;
+ if (!font) {
+ return null;
+ }
+
+ var mode = group.mode;
+ if (font === "mathit") {
+ return "italic";
+ }
+
+ var value = group.value;
+ if (utils.contains(["\\imath", "\\jmath"], value)) {
+ return null;
+ }
+
+ if (symbols[mode][value] && symbols[mode][value].replace) {
+ value = symbols[mode][value].replace;
+ }
+
+ var fontName = fontMap[font].fontName;
+ if (fontMetrics.getCharacterMetrics(value, fontName)) {
+ return fontMap[options.font].variant;
+ }
+
+ return null;
+};
+
+/**
+ * Functions for handling the different types of groups found in the parse
+ * tree. Each function should take a parse group and return a MathML node.
+ */
+var groupTypes = {};
+
+groupTypes.mathord = function(group, options) {
+ var node = new mathMLTree.MathNode(
+ "mi",
+ [makeText(group.value, group.mode)]);
+
+ var variant = getVariant(group, options);
+ if (variant) {
+ node.setAttribute("mathvariant", variant);
+ }
+ return node;
+};
+
+groupTypes.textord = function(group, options) {
+ var text = makeText(group.value, group.mode);
+
+ var variant = getVariant(group, options) || "normal";
+
+ var node;
+ if (/[0-9]/.test(group.value)) {
+ // TODO(kevinb) merge adjacent nodes
+ // do it as a post processing step
+ node = new mathMLTree.MathNode("mn", [text]);
+ if (options.font) {
+ node.setAttribute("mathvariant", variant);
+ }
+ } else {
+ node = new mathMLTree.MathNode("mi", [text]);
+ node.setAttribute("mathvariant", variant);
+ }
+
+ return node;
+};
+
+groupTypes.bin = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ return node;
+};
+
+groupTypes.rel = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ return node;
+};
+
+groupTypes.open = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ return node;
+};
+
+groupTypes.close = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ return node;
+};
+
+groupTypes.inner = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ return node;
+};
+
+groupTypes.punct = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value, group.mode)]);
+
+ node.setAttribute("separator", "true");
+
+ return node;
+};
+
+groupTypes.ordgroup = function(group, options) {
+ var inner = buildExpression(group.value, options);
+
+ var node = new mathMLTree.MathNode("mrow", inner);
+
+ return node;
+};
+
+groupTypes.text = function(group, options) {
+ var inner = buildExpression(group.value.body, options);
+
+ var node = new mathMLTree.MathNode("mtext", inner);
+
+ return node;
+};
+
+groupTypes.color = function(group, options) {
+ var inner = buildExpression(group.value.value, options);
+
+ var node = new mathMLTree.MathNode("mstyle", inner);
+
+ node.setAttribute("mathcolor", group.value.color);
+
+ return node;
+};
+
+groupTypes.supsub = function(group, options) {
+ var children = [buildGroup(group.value.base, options)];
+
+ if (group.value.sub) {
+ children.push(buildGroup(group.value.sub, options));
+ }
+
+ if (group.value.sup) {
+ children.push(buildGroup(group.value.sup, options));
+ }
+
+ var nodeType;
+ if (!group.value.sub) {
+ nodeType = "msup";
+ } else if (!group.value.sup) {
+ nodeType = "msub";
+ } else {
+ nodeType = "msubsup";
+ }
+
+ var node = new mathMLTree.MathNode(nodeType, children);
+
+ return node;
+};
+
+groupTypes.genfrac = function(group, options) {
+ var node = new mathMLTree.MathNode(
+ "mfrac",
+ [buildGroup(group.value.numer, options),
+ buildGroup(group.value.denom, options)]);
+
+ if (!group.value.hasBarLine) {
+ node.setAttribute("linethickness", "0px");
+ }
+
+ if (group.value.leftDelim != null || group.value.rightDelim != null) {
+ var withDelims = [];
+
+ if (group.value.leftDelim != null) {
+ var leftOp = new mathMLTree.MathNode(
+ "mo", [new mathMLTree.TextNode(group.value.leftDelim)]);
+
+ leftOp.setAttribute("fence", "true");
+
+ withDelims.push(leftOp);
+ }
+
+ withDelims.push(node);
+
+ if (group.value.rightDelim != null) {
+ var rightOp = new mathMLTree.MathNode(
+ "mo", [new mathMLTree.TextNode(group.value.rightDelim)]);
+
+ rightOp.setAttribute("fence", "true");
+
+ withDelims.push(rightOp);
+ }
+
+ var outerNode = new mathMLTree.MathNode("mrow", withDelims);
+
+ return outerNode;
+ }
+
+ return node;
+};
+
+groupTypes.array = function(group, options) {
+ return new mathMLTree.MathNode(
+ "mtable", group.value.body.map(function(row) {
+ return new mathMLTree.MathNode(
+ "mtr", row.map(function(cell) {
+ return new mathMLTree.MathNode(
+ "mtd", [buildGroup(cell, options)]);
+ }));
+ }));
+};
+
+groupTypes.sqrt = function(group, options) {
+ var node;
+ if (group.value.index) {
+ node = new mathMLTree.MathNode(
+ "mroot", [
+ buildGroup(group.value.body, options),
+ buildGroup(group.value.index, options),
+ ]);
+ } else {
+ node = new mathMLTree.MathNode(
+ "msqrt", [buildGroup(group.value.body, options)]);
+ }
+
+ return node;
+};
+
+groupTypes.leftright = function(group, options) {
+ var inner = buildExpression(group.value.body, options);
+
+ if (group.value.left !== ".") {
+ var leftNode = new mathMLTree.MathNode(
+ "mo", [makeText(group.value.left, group.mode)]);
+
+ leftNode.setAttribute("fence", "true");
+
+ inner.unshift(leftNode);
+ }
+
+ if (group.value.right !== ".") {
+ var rightNode = new mathMLTree.MathNode(
+ "mo", [makeText(group.value.right, group.mode)]);
+
+ rightNode.setAttribute("fence", "true");
+
+ inner.push(rightNode);
+ }
+
+ var outerNode = new mathMLTree.MathNode("mrow", inner);
+
+ return outerNode;
+};
+
+groupTypes.accent = function(group, options) {
+ var accentNode = new mathMLTree.MathNode(
+ "mo", [makeText(group.value.accent, group.mode)]);
+
+ var node = new mathMLTree.MathNode(
+ "mover",
+ [buildGroup(group.value.base, options),
+ accentNode]);
+
+ node.setAttribute("accent", "true");
+
+ return node;
+};
+
+groupTypes.spacing = function(group) {
+ var node;
+
+ if (group.value === "\\ " || group.value === "\\space" ||
+ group.value === " " || group.value === "~") {
+ node = new mathMLTree.MathNode(
+ "mtext", [new mathMLTree.TextNode("\u00a0")]);
+ } else {
+ node = new mathMLTree.MathNode("mspace");
+
+ node.setAttribute(
+ "width", buildCommon.spacingFunctions[group.value].size);
+ }
+
+ return node;
+};
+
+groupTypes.op = function(group) {
+ var node;
+
+ // TODO(emily): handle big operators using the `largeop` attribute
+
+ if (group.value.symbol) {
+ // This is a symbol. Just add the symbol.
+ node = new mathMLTree.MathNode(
+ "mo", [makeText(group.value.body, group.mode)]);
+ } else {
+ // This is a text operator. Add all of the characters from the
+ // operator's name.
+ // TODO(emily): Add a space in the middle of some of these
+ // operators, like \limsup.
+ node = new mathMLTree.MathNode(
+ "mi", [new mathMLTree.TextNode(group.value.body.slice(1))]);
+ }
+
+ return node;
+};
+
+groupTypes.katex = function(group) {
+ var node = new mathMLTree.MathNode(
+ "mtext", [new mathMLTree.TextNode("KaTeX")]);
+
+ return node;
+};
+
+groupTypes.font = function(group, options) {
+ var font = group.value.font;
+ return buildGroup(group.value.body, options.withFont(font));
+};
+
+groupTypes.delimsizing = function(group) {
+ var children = [];
+
+ if (group.value.value !== ".") {
+ children.push(makeText(group.value.value, group.mode));
+ }
+
+ var node = new mathMLTree.MathNode("mo", children);
+
+ if (group.value.delimType === "open" ||
+ group.value.delimType === "close") {
+ // Only some of the delimsizing functions act as fences, and they
+ // return "open" or "close" delimTypes.
+ node.setAttribute("fence", "true");
+ } else {
+ // Explicitly disable fencing if it's not a fence, to override the
+ // defaults.
+ node.setAttribute("fence", "false");
+ }
+
+ return node;
+};
+
+groupTypes.styling = function(group, options) {
+ var inner = buildExpression(group.value.value, options);
+
+ var node = new mathMLTree.MathNode("mstyle", inner);
+
+ var styleAttributes = {
+ "display": ["0", "true"],
+ "text": ["0", "false"],
+ "script": ["1", "false"],
+ "scriptscript": ["2", "false"],
+ };
+
+ var attr = styleAttributes[group.value.style];
+
+ node.setAttribute("scriptlevel", attr[0]);
+ node.setAttribute("displaystyle", attr[1]);
+
+ return node;
+};
+
+groupTypes.sizing = function(group, options) {
+ var inner = buildExpression(group.value.value, options);
+
+ var node = new mathMLTree.MathNode("mstyle", inner);
+
+ // TODO(emily): This doesn't produce the correct size for nested size
+ // changes, because we don't keep state of what style we're currently
+ // in, so we can't reset the size to normal before changing it. Now
+ // that we're passing an options parameter we should be able to fix
+ // this.
+ node.setAttribute(
+ "mathsize", buildCommon.sizingMultiplier[group.value.size] + "em");
+
+ return node;
+};
+
+groupTypes.overline = function(group, options) {
+ var operator = new mathMLTree.MathNode(
+ "mo", [new mathMLTree.TextNode("\u203e")]);
+ operator.setAttribute("stretchy", "true");
+
+ var node = new mathMLTree.MathNode(
+ "mover",
+ [buildGroup(group.value.body, options),
+ operator]);
+ node.setAttribute("accent", "true");
+
+ return node;
+};
+
+groupTypes.underline = function(group, options) {
+ var operator = new mathMLTree.MathNode(
+ "mo", [new mathMLTree.TextNode("\u203e")]);
+ operator.setAttribute("stretchy", "true");
+
+ var node = new mathMLTree.MathNode(
+ "munder",
+ [buildGroup(group.value.body, options),
+ operator]);
+ node.setAttribute("accentunder", "true");
+
+ return node;
+};
+
+groupTypes.rule = function(group) {
+ // TODO(emily): Figure out if there's an actual way to draw black boxes
+ // in MathML.
+ var node = new mathMLTree.MathNode("mrow");
+
+ return node;
+};
+
+groupTypes.llap = function(group, options) {
+ var node = new mathMLTree.MathNode(
+ "mpadded", [buildGroup(group.value.body, options)]);
+
+ node.setAttribute("lspace", "-1width");
+ node.setAttribute("width", "0px");
+
+ return node;
+};
+
+groupTypes.rlap = function(group, options) {
+ var node = new mathMLTree.MathNode(
+ "mpadded", [buildGroup(group.value.body, options)]);
+
+ node.setAttribute("width", "0px");
+
+ return node;
+};
+
+groupTypes.phantom = function(group, options, prev) {
+ var inner = buildExpression(group.value.value, options);
+ return new mathMLTree.MathNode("mphantom", inner);
+};
+
+/**
+ * Takes a list of nodes, builds them, and returns a list of the generated
+ * MathML nodes. A little simpler than the HTML version because we don't do any
+ * previous-node handling.
+ */
+var buildExpression = function(expression, options) {
+ var groups = [];
+ for (var i = 0; i < expression.length; i++) {
+ var group = expression[i];
+ groups.push(buildGroup(group, options));
+ }
+ return groups;
+};
+
+/**
+ * Takes a group from the parser and calls the appropriate groupTypes function
+ * on it to produce a MathML node.
+ */
+var buildGroup = function(group, options) {
+ if (!group) {
+ return new mathMLTree.MathNode("mrow");
+ }
+
+ if (groupTypes[group.type]) {
+ // Call the groupTypes function
+ return groupTypes[group.type](group, options);
+ } else {
+ throw new ParseError(
+ "Got group of unknown type: '" + group.type + "'");
+ }
+};
+
+/**
+ * Takes a full parse tree and settings and builds a MathML representation of
+ * it. In particular, we put the elements from building the parse tree into a
+ * tag so we can also include that TeX source as an annotation.
+ *
+ * Note that we actually return a domTree element with a `
',s)
+ res = ''
+ res += '\n\n'
+ temp_item = ''
+ for c in s:
+ temp_item += c
+ if c == '>' and not re.search(r'\$', temp_item):
+ res += temp_item + '\n'
+ temp_item = ''
+ return res + '\n'
+
+
+def merge_table(md):
+ table_temp = ''
+ for line in md:
+ table_temp += line
+ return convert_table_str(table_temp)
+
+
+def find_md_table_mode(line):
+ if re.search(r'-*?:', line) or re.search(r'---', line) or re.search(r':-*?', line):
+ return True
+ return False
+
+
+def delete_table_and_body(input_list):
+ res = []
+ for line in input_list:
+ if not re.search(r'', line):
+ res.append(line)
+ return res
+
+
+def merge_tables(input_str):
+ # Delete HTML comments
+ input_str = re.sub(r'', '', input_str)
+
+ # Use regex to find each
block
+ table_blocks = re.findall(r'
[\s\S]*?
', input_str)
+
+ # Process each
block, replace
with
+ output_lines = []
+ for block in table_blocks:
+ block_lines = block.split('\n')
+ for i, line in enumerate(block_lines):
+ if '
+
+
](https://khan.github.io/KaTeX/) [](https://travis-ci.org/Khan/KaTeX)
+
+[](https://gitter.im/Khan/KaTeX?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+
+KaTeX is a fast, easy-to-use JavaScript library for TeX math rendering on the web.
+
+ * **Fast:** KaTeX renders its math synchronously and doesn't need to reflow the page. See how it compares to a competitor in [this speed test](http://jsperf.com/katex-vs-mathjax/).
+ * **Print quality:** KaTeX’s layout is based on Donald Knuth’s TeX, the gold standard for math typesetting.
+ * **Self contained:** KaTeX has no dependencies and can easily be bundled with your website resources.
+ * **Server side rendering:** KaTeX produces the same output regardless of browser or environment, so you can pre-render expressions using Node.js and send them as plain HTML.
+
+KaTeX supports all major browsers, including Chrome, Safari, Firefox, Opera, and IE 8 - IE 11. A list of supported commands can be on the [wiki](https://github.com/Khan/KaTeX/wiki/Function-Support-in-KaTeX).
+
+## Usage
+
+You can [download KaTeX](https://github.com/khan/katex/releases) and host it on your server or include the `katex.min.js` and `katex.min.css` files on your page directly from a CDN:
+
+```html
+
+
+```
+
+#### In-browser rendering
+
+Call `katex.render` with a TeX expression and a DOM element to render into:
+
+```js
+katex.render("c = \\pm\\sqrt{a^2 + b^2}", element);
+```
+
+If KaTeX can't parse the expression, it throws a `katex.ParseError` error.
+
+#### Server side rendering or rendering to a string
+
+To generate HTML on the server or to generate an HTML string of the rendered math, you can use `katex.renderToString`:
+
+```js
+var html = katex.renderToString("c = \\pm\\sqrt{a^2 + b^2}");
+// '...'
+```
+
+Make sure to include the CSS and font files, but there is no need to include the JavaScript. Like `render`, `renderToString` throws if it can't parse the expression.
+
+#### Rendering options
+
+You can provide an object of options as the last argument to `katex.render` and `katex.renderToString`. Available options are:
+
+- `displayMode`: `boolean`. If `true` the math will be rendered in display mode, which will put the math in display style (so `\int` and `\sum` are large, for example), and will center the math on the page on its own line. If `false` the math will be rendered in inline mode. (default: `false`)
+- `throwOnError`: `boolean`. If `true`, KaTeX will throw a `ParseError` when it encounters an unsupported command. If `false`, KaTeX will render the unsupported command as text in the color given by `errorColor`. (default: `true`)
+- `errorColor`: `string`. A color string given in the format `"#XXX"` or `"#XXXXXX"`. This option determines the color which unsupported commands are rendered in. (default: `#cc0000`)
+
+For example:
+
+```js
+katex.render("c = \\pm\\sqrt{a^2 + b^2}", element, { displayMode: true });
+```
+
+#### Automatic rendering of math on a page
+
+Math on the page can be automatically rendered using the auto-render extension. See [the Auto-render README](contrib/auto-render/README.md) for more information.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md)
+
+## License
+
+KaTeX is licensed under the [MIT License](http://opensource.org/licenses/MIT).
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js
new file mode 100644
index 000000000..b64de377c
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/cli.js
@@ -0,0 +1,32 @@
+#!/usr/bin/env node
+// Simple CLI for KaTeX.
+// Reads TeX from stdin, outputs HTML to stdout.
+/* eslint no-console:0 */
+
+var katex = require("./");
+var input = "";
+
+// Skip the first two args, which are just "node" and "cli.js"
+var args = process.argv.slice(2);
+
+if (args.indexOf("--help") !== -1) {
+ console.log(process.argv[0] + " " + process.argv[1] +
+ " [ --help ]" +
+ " [ --display-mode ]");
+
+ console.log("\n" +
+ "Options:");
+ console.log(" --help Display this help message");
+ console.log(" --display-mode Render in display mode (not inline mode)");
+ process.exit();
+}
+
+process.stdin.on("data", function(chunk) {
+ input += chunk.toString();
+});
+
+process.stdin.on("end", function() {
+ var options = { displayMode: args.indexOf("--display-mode") !== -1 };
+ var output = katex.renderToString(input, options);
+ console.log(output);
+});
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js
new file mode 100644
index 000000000..4d64606bf
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/katex.js
@@ -0,0 +1,74 @@
+/* eslint no-console:0 */
+/**
+ * This is the main entry point for KaTeX. Here, we expose functions for
+ * rendering expressions either to DOM nodes or to markup strings.
+ *
+ * We also expose the ParseError class to check if errors thrown from KaTeX are
+ * errors in the expression, or errors in javascript handling.
+ */
+
+var ParseError = require("./src/ParseError");
+var Settings = require("./src/Settings");
+
+var buildTree = require("./src/buildTree");
+var parseTree = require("./src/parseTree");
+var utils = require("./src/utils");
+
+/**
+ * Parse and build an expression, and place that expression in the DOM node
+ * given.
+ */
+var render = function(expression, baseNode, options) {
+ utils.clearNode(baseNode);
+
+ var settings = new Settings(options);
+
+ var tree = parseTree(expression, settings);
+ var node = buildTree(tree, expression, settings).toNode();
+
+ baseNode.appendChild(node);
+};
+
+// KaTeX's styles don't work properly in quirks mode. Print out an error, and
+// disable rendering.
+if (typeof document !== "undefined") {
+ if (document.compatMode !== "CSS1Compat") {
+ typeof console !== "undefined" && console.warn(
+ "Warning: KaTeX doesn't work in quirks mode. Make sure your " +
+ "website has a suitable doctype.");
+
+ render = function() {
+ throw new ParseError("KaTeX doesn't work in quirks mode.");
+ };
+ }
+}
+
+/**
+ * Parse and build an expression, and return the markup for that.
+ */
+var renderToString = function(expression, options) {
+ var settings = new Settings(options);
+
+ var tree = parseTree(expression, settings);
+ return buildTree(tree, expression, settings).toMarkup();
+};
+
+/**
+ * Parse an expression and return the parse tree.
+ */
+var generateParseTree = function(expression, options) {
+ var settings = new Settings(options);
+ return parseTree(expression, settings);
+};
+
+module.exports = {
+ render: render,
+ renderToString: renderToString,
+ /**
+ * NOTE: This method is not currently recommended for public use.
+ * The internal tree representation is unstable and is very likely
+ * to change. Use at your own risk.
+ */
+ __parse: generateParseTree,
+ ParseError: ParseError,
+};
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js
new file mode 100644
index 000000000..5cb8d3d4e
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Lexer.js
@@ -0,0 +1,162 @@
+/**
+ * The Lexer class handles tokenizing the input in various ways. Since our
+ * parser expects us to be able to backtrack, the lexer allows lexing from any
+ * given starting point.
+ *
+ * Its main exposed function is the `lex` function, which takes a position to
+ * lex from and a type of token to lex. It defers to the appropriate `_innerLex`
+ * function.
+ *
+ * The various `_innerLex` functions perform the actual lexing of different
+ * kinds.
+ */
+
+var matchAt = require("../../match-at");
+
+var ParseError = require("./ParseError");
+
+// The main lexer class
+function Lexer(input) {
+ this._input = input;
+}
+
+// The resulting token returned from `lex`.
+function Token(text, data, position) {
+ this.text = text;
+ this.data = data;
+ this.position = position;
+}
+
+/* The following tokenRegex
+ * - matches typical whitespace (but not NBSP etc.) using its first group
+ * - matches symbol combinations which result in a single output character
+ * - does not match any control character \x00-\x1f except whitespace
+ * - does not match a bare backslash
+ * - matches any ASCII character except those just mentioned
+ * - does not match the BMP private use area \uE000-\uF8FF
+ * - does not match bare surrogate code units
+ * - matches any BMP character except for those just described
+ * - matches any valid Unicode surrogate pair
+ * - matches a backslash followed by one or more letters
+ * - matches a backslash followed by any BMP character, including newline
+ * Just because the Lexer matches something doesn't mean it's valid input:
+ * If there is no matching function or symbol definition, the Parser will
+ * still reject the input.
+ */
+var tokenRegex = new RegExp(
+ "([ \r\n\t]+)|(" + // whitespace
+ "---?" + // special combinations
+ "|[!-\\[\\]-\u2027\u202A-\uD7FF\uF900-\uFFFF]" + // single codepoint
+ "|[\uD800-\uDBFF][\uDC00-\uDFFF]" + // surrogate pair
+ "|\\\\(?:[a-zA-Z]+|[^\uD800-\uDFFF])" + // function name
+ ")"
+);
+
+var whitespaceRegex = /\s*/;
+
+/**
+ * This function lexes a single normal token. It takes a position and
+ * whether it should completely ignore whitespace or not.
+ */
+Lexer.prototype._innerLex = function(pos, ignoreWhitespace) {
+ var input = this._input;
+ if (pos === input.length) {
+ return new Token("EOF", null, pos);
+ }
+ var match = matchAt(tokenRegex, input, pos);
+ if (match === null) {
+ throw new ParseError(
+ "Unexpected character: '" + input[pos] + "'",
+ this, pos);
+ } else if (match[2]) { // matched non-whitespace
+ return new Token(match[2], null, pos + match[2].length);
+ } else if (ignoreWhitespace) {
+ return this._innerLex(pos + match[1].length, true);
+ } else { // concatenate whitespace to a single space
+ return new Token(" ", null, pos + match[1].length);
+ }
+};
+
+// A regex to match a CSS color (like #ffffff or BlueViolet)
+var cssColor = /#[a-z0-9]+|[a-z]+/i;
+
+/**
+ * This function lexes a CSS color.
+ */
+Lexer.prototype._innerLexColor = function(pos) {
+ var input = this._input;
+
+ // Ignore whitespace
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ var match;
+ if ((match = matchAt(cssColor, input, pos))) {
+ // If we look like a color, return a color
+ return new Token(match[0], null, pos + match[0].length);
+ } else {
+ throw new ParseError("Invalid color", this, pos);
+ }
+};
+
+// A regex to match a dimension. Dimensions look like
+// "1.2em" or ".4pt" or "1 ex"
+var sizeRegex = /(-?)\s*(\d+(?:\.\d*)?|\.\d+)\s*([a-z]{2})/;
+
+/**
+ * This function lexes a dimension.
+ */
+Lexer.prototype._innerLexSize = function(pos) {
+ var input = this._input;
+
+ // Ignore whitespace
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ var match;
+ if ((match = matchAt(sizeRegex, input, pos))) {
+ var unit = match[3];
+ // We only currently handle "em" and "ex" units
+ // if (unit !== "em" && unit !== "ex") {
+ // throw new ParseError("Invalid unit: '" + unit + "'", this, pos);
+ // }
+ return new Token(match[0], {
+ number: +(match[1] + match[2]),
+ unit: unit,
+ }, pos + match[0].length);
+ }
+
+ throw new ParseError("Invalid size", this, pos);
+};
+
+/**
+ * This function lexes a string of whitespace.
+ */
+Lexer.prototype._innerLexWhitespace = function(pos) {
+ var input = this._input;
+
+ var whitespace = matchAt(whitespaceRegex, input, pos)[0];
+ pos += whitespace.length;
+
+ return new Token(whitespace[0], null, pos);
+};
+
+/**
+ * This function lexes a single token starting at `pos` and of the given mode.
+ * Based on the mode, we defer to one of the `_innerLex` functions.
+ */
+Lexer.prototype.lex = function(pos, mode) {
+ if (mode === "math") {
+ return this._innerLex(pos, true);
+ } else if (mode === "text") {
+ return this._innerLex(pos, false);
+ } else if (mode === "color") {
+ return this._innerLexColor(pos);
+ } else if (mode === "size") {
+ return this._innerLexSize(pos);
+ } else if (mode === "whitespace") {
+ return this._innerLexWhitespace(pos);
+ }
+};
+
+module.exports = Lexer;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js
new file mode 100644
index 000000000..39ff37bfc
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Options.js
@@ -0,0 +1,189 @@
+/**
+ * This file contains information about the options that the Parser carries
+ * around with it while parsing. Data is held in an `Options` object, and when
+ * recursing, a new `Options` object can be created with the `.with*` and
+ * `.reset` functions.
+ */
+
+/**
+ * This is the main options class. It contains the style, size, color, and font
+ * of the current parse level. It also contains the style and size of the parent
+ * parse level, so size changes can be handled efficiently.
+ *
+ * Each of the `.with*` and `.reset` functions passes its current style and size
+ * as the parentStyle and parentSize of the new options class, so parent
+ * handling is taken care of automatically.
+ */
+function Options(data) {
+ this.style = data.style;
+ this.color = data.color;
+ this.size = data.size;
+ this.phantom = data.phantom;
+ this.font = data.font;
+
+ if (data.parentStyle === undefined) {
+ this.parentStyle = data.style;
+ } else {
+ this.parentStyle = data.parentStyle;
+ }
+
+ if (data.parentSize === undefined) {
+ this.parentSize = data.size;
+ } else {
+ this.parentSize = data.parentSize;
+ }
+}
+
+/**
+ * Returns a new options object with the same properties as "this". Properties
+ * from "extension" will be copied to the new options object.
+ */
+Options.prototype.extend = function(extension) {
+ var data = {
+ style: this.style,
+ size: this.size,
+ color: this.color,
+ parentStyle: this.style,
+ parentSize: this.size,
+ phantom: this.phantom,
+ font: this.font,
+ };
+
+ for (var key in extension) {
+ if (extension.hasOwnProperty(key)) {
+ data[key] = extension[key];
+ }
+ }
+
+ return new Options(data);
+};
+
+/**
+ * Create a new options object with the given style.
+ */
+Options.prototype.withStyle = function(style) {
+ return this.extend({
+ style: style,
+ });
+};
+
+/**
+ * Create a new options object with the given size.
+ */
+Options.prototype.withSize = function(size) {
+ return this.extend({
+ size: size,
+ });
+};
+
+/**
+ * Create a new options object with the given color.
+ */
+Options.prototype.withColor = function(color) {
+ return this.extend({
+ color: color,
+ });
+};
+
+/**
+ * Create a new options object with "phantom" set to true.
+ */
+Options.prototype.withPhantom = function() {
+ return this.extend({
+ phantom: true,
+ });
+};
+
+/**
+ * Create a new options objects with the give font.
+ */
+Options.prototype.withFont = function(font) {
+ return this.extend({
+ font: font,
+ });
+};
+
+/**
+ * Create a new options object with the same style, size, and color. This is
+ * used so that parent style and size changes are handled correctly.
+ */
+Options.prototype.reset = function() {
+ return this.extend({});
+};
+
+/**
+ * A map of color names to CSS colors.
+ * TODO(emily): Remove this when we have real macros
+ */
+var colorMap = {
+ "katex-blue": "#6495ed",
+ "katex-orange": "#ffa500",
+ "katex-pink": "#ff00af",
+ "katex-red": "#df0030",
+ "katex-green": "#28ae7b",
+ "katex-gray": "gray",
+ "katex-purple": "#9d38bd",
+ "katex-blueA": "#c7e9f1",
+ "katex-blueB": "#9cdceb",
+ "katex-blueC": "#58c4dd",
+ "katex-blueD": "#29abca",
+ "katex-blueE": "#1c758a",
+ "katex-tealA": "#acead7",
+ "katex-tealB": "#76ddc0",
+ "katex-tealC": "#5cd0b3",
+ "katex-tealD": "#55c1a7",
+ "katex-tealE": "#49a88f",
+ "katex-greenA": "#c9e2ae",
+ "katex-greenB": "#a6cf8c",
+ "katex-greenC": "#83c167",
+ "katex-greenD": "#77b05d",
+ "katex-greenE": "#699c52",
+ "katex-goldA": "#f7c797",
+ "katex-goldB": "#f9b775",
+ "katex-goldC": "#f0ac5f",
+ "katex-goldD": "#e1a158",
+ "katex-goldE": "#c78d46",
+ "katex-redA": "#f7a1a3",
+ "katex-redB": "#ff8080",
+ "katex-redC": "#fc6255",
+ "katex-redD": "#e65a4c",
+ "katex-redE": "#cf5044",
+ "katex-maroonA": "#ecabc1",
+ "katex-maroonB": "#ec92ab",
+ "katex-maroonC": "#c55f73",
+ "katex-maroonD": "#a24d61",
+ "katex-maroonE": "#94424f",
+ "katex-purpleA": "#caa3e8",
+ "katex-purpleB": "#b189c6",
+ "katex-purpleC": "#9a72ac",
+ "katex-purpleD": "#715582",
+ "katex-purpleE": "#644172",
+ "katex-mintA": "#f5f9e8",
+ "katex-mintB": "#edf2df",
+ "katex-mintC": "#e0e5cc",
+ "katex-grayA": "#fdfdfd",
+ "katex-grayB": "#f7f7f7",
+ "katex-grayC": "#eeeeee",
+ "katex-grayD": "#dddddd",
+ "katex-grayE": "#cccccc",
+ "katex-grayF": "#aaaaaa",
+ "katex-grayG": "#999999",
+ "katex-grayH": "#555555",
+ "katex-grayI": "#333333",
+ "katex-kaBlue": "#314453",
+ "katex-kaGreen": "#639b24",
+};
+
+/**
+ * Gets the CSS color of the current options object, accounting for the
+ * `colorMap`.
+ */
+Options.prototype.getColor = function() {
+ if (this.phantom) {
+ return "transparent";
+ } else {
+ return colorMap[this.color] || this.color;
+ }
+};
+
+module.exports = Options;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js
new file mode 100644
index 000000000..320f0bd69
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/ParseError.js
@@ -0,0 +1,40 @@
+/**
+ * This is the ParseError class, which is the main error thrown by KaTeX
+ * functions when something has gone wrong. This is used to distinguish internal
+ * errors from errors in the expression that the user provided.
+ */
+function ParseError(message, lexer, position) {
+ var error = "KaTeX parse error: " + message;
+
+ if (lexer !== undefined && position !== undefined) {
+ // If we have the input and a position, make the error a bit fancier
+
+ // Prepend some information
+ error += " at position " + position + ": ";
+
+ // Get the input
+ var input = lexer._input;
+ // Insert a combining underscore at the correct position
+ input = input.slice(0, position) + "\u0332" +
+ input.slice(position);
+
+ // Extract some context from the input and add it to the error
+ var begin = Math.max(0, position - 15);
+ var end = position + 15;
+ error += input.slice(begin, end);
+ }
+
+ // Some hackery to make ParseError a prototype of Error
+ // See http://stackoverflow.com/a/8460753
+ var self = new Error(error);
+ self.name = "ParseError";
+ self.__proto__ = ParseError.prototype;
+
+ self.position = position;
+ return self;
+}
+
+// More hackery
+ParseError.prototype.__proto__ = Error.prototype;
+
+module.exports = ParseError;
diff --git a/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js
new file mode 100644
index 000000000..aca6cd291
--- /dev/null
+++ b/vlmeval/dataset/MDPBench/metrics/cdm/modules/tokenize_latex/third_party/katex/src/Parser.js
@@ -0,0 +1,798 @@
+/* eslint no-constant-condition:0 */
+var functions = require("./functions");
+var environments = require("./environments");
+var Lexer = require("./Lexer");
+var symbols = require("./symbols");
+var utils = require("./utils");
+
+var parseData = require("./parseData");
+var ParseError = require("./ParseError");
+
+global_str = ""
+
+/**
+ * This file contains the parser used to parse out a TeX expression from the
+ * input. Since TeX isn't context-free, standard parsers don't work particularly
+ * well.
+ *
+ * The strategy of this parser is as such:
+ *
+ * The main functions (the `.parse...` ones) take a position in the current
+ * parse string to parse tokens from. The lexer (found in Lexer.js, stored at
+ * this.lexer) also supports pulling out tokens at arbitrary places. When
+ * individual tokens are needed at a position, the lexer is called to pull out a
+ * token, which is then used.
+ *
+ * The parser has a property called "mode" indicating the mode that
+ * the parser is currently in. Currently it has to be one of "math" or
+ * "text", which denotes whether the current environment is a math-y
+ * one or a text-y one (e.g. inside \text). Currently, this serves to
+ * limit the functions which can be used in text mode.
+ *
+ * The main functions then return an object which contains the useful data that
+ * was parsed at its given point, and a new position at the end of the parsed
+ * data. The main functions can call each other and continue the parsing by
+ * using the returned position as a new starting point.
+ *
+ * There are also extra `.handle...` functions, which pull out some reused
+ * functionality into self-contained functions.
+ *
+ * The earlier functions return ParseNodes.
+ * The later functions (which are called deeper in the parse) sometimes return
+ * ParseFuncOrArgument, which contain a ParseNode as well as some data about
+ * whether the parsed object is a function which is missing some arguments, or a
+ * standalone object which can be used as an argument to another function.
+ */
+
+/**
+ * Main Parser class
+ */
+function Parser(input, settings) {
+ // Make a new lexer
+ this.lexer = new Lexer(input);
+ // Store the settings for use in parsing
+ this.settings = settings;
+}
+
+var ParseNode = parseData.ParseNode;
+
+/**
+ * An initial function (without its arguments), or an argument to a function.
+ * The `result` argument should be a ParseNode.
+ */
+function ParseFuncOrArgument(result, isFunction) {
+ this.result = result;
+ // Is this a function (i.e. is it something defined in functions.js)?
+ this.isFunction = isFunction;
+}
+
+/**
+ * Checks a result to make sure it has the right type, and throws an
+ * appropriate error otherwise.
+ *
+ * @param {boolean=} consume whether to consume the expected token,
+ * defaults to true
+ */
+Parser.prototype.expect = function(text, consume) {
+ if (this.nextToken.text !== text) {
+ throw new ParseError(
+ "Expected '" + text + "', got '" + this.nextToken.text + "'",
+ this.lexer, this.nextToken.position
+ );
+ }
+ if (consume !== false) {
+ this.consume();
+ }
+};
+
+/**
+ * Considers the current look ahead token as consumed,
+ * and fetches the one after that as the new look ahead.
+ */
+Parser.prototype.consume = function() {
+ this.pos = this.nextToken.position;
+
+ global_str = global_str + " " + this.nextToken.text
+ this.nextToken = this.lexer.lex(this.pos, this.mode);
+};
+
+/**
+ * Main parsing function, which parses an entire input.
+ *
+ * @return {?Array.