[v1.1] Fix the Format in the SDA-BERT/DE-NAS BERT/ViT/HF Demos (#285)

* Fix the typos in the SDA BERT demo

* format the SDA-BERT/DENAS BERT/HF/ViT demos

* Add DE-NAS HF train conf and component for the demo

* refine the configuration

* Reformat the DE-NAS HF demo in the training part

* Fix the typos

Author: tianyil1

conf/denas/hf/e2eaiok_denas_train_bert.conf

@@ -0,0 +1,45 @@
+# conf for Bert-based DE-NAS
+domain: hf
+task_name: squad1
+task_type: classification
+supernet: bert-base-uncased
+tokenizer: bert-base-uncased
+optimizer: BertAdam
+criterion: CrossEntropyQALoss
+lr_scheduler: warmup_linear
+eval_metric: qa_f1
+dist_backend: gloo
+input_id: input_ids attention_mask token_type_ids
+data_set: SQuADv1.1
+
+best_model_structure: /home/vmagent/app/e2eaiok/e2eAIOK/DeNas/best_model_structure.txt
+model: /home/vmagent/app/dataset/bert-base-uncased/
+model_dir: /home/vmagent/app/dataset/bert-base-uncased/
+data_dir: /home/vmagent/app/dataset/SQuAD/
+output_dir: /home/vmagent/app/e2eaiok/e2eAIOK/DeNas/thirdparty/
+
+hidden_size: 640
+gradient_accumulation_steps: 1
+warmup_proportion: 0.1
+learning_rate: 0.00006
+weight_decay: 0.01
+initializer_range: 0.02
+train_epochs: 2
+max_seq_length: 384
+doc_stride: 128
+train_batch_size: 32
+eval_batch_size: 8
+eval_step: 500
+n_best_size: 20
+max_answer_length: 30
+max_query_length: 64
+version_2_with_negative: 0
+null_score_diff_threshold: 0.0
+num_labels: 2
+num_workers: 1
+
+pin_mem: True
+verbose_logging: False
+no_cuda: True
+do_lower_case: True
+metric_threshold: None

demo/builtin/bert/BERT_DEMO.ipynb

@@ -29,7 +29,7 @@
       },
       "source": [
         "# Content\n",
-        "* [Overview](#ovewview)\n",
+        "* [Overview](#overview)\n",
         "    * [Model Architecture](#model-architecture)\n",
         "    * [Optimizations](#optimizations)\n",
         "    * [Performance](#performance)\n",
@@ -47,7 +47,7 @@
         "id": "pS__BjqrCZqn"
       },
       "source": [
-        "# Ovewview"
+        "# Overview"
       ]
     },
     {
@@ -61,7 +61,7 @@
         "\n",
         "### Natural Language Processing\n",
         "\n",
-        "<img src=\"https://github.com/intel/e2eAIOK/blob/main/demo/builtin/bert/img/QA.png?raw=1\" width=\"600\"/><figure>Question Answer Task</figure>\n",
+        "<img src=\"https://github.com/intel/e2eAIOK/blob/main/demo/builtin/bert/img/QA.png?raw=1\" width=\"400\"/><figure>Question Answer Task</figure>\n",
         "\n",
         "* Natural language processing (NLP) is the intersection of computer science, linguistics and machine learning, where the pre-trained language model BERT is the most representive model in a wide area of NLP tasks, like question and answer.\n",
         "* The end-to-end NLP system is a BERT-based network that uses the pretrained model weight for the downstream question answer task SQuAD (v1.1). In the question-answer task, given the input question and paragraph/context sequence, the BERT aims to predict the start/end index in the paragraph to indicate the answer span.\n",
@@ -246,7 +246,7 @@
         "\n",
         "## 1. Environment Setup\n",
         "\n",
-        "### Option 1 Setup Environment with Pip"
+        "### (Option 1) Use Pip install"
       ]
     },
     {
@@ -264,7 +264,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "### Option 2 Setup Environment with Docker\n",
+        "### (Option 2) Use Docker\n",
         "\n",
         "Step1. prepare code\n",
         "``` bash\n",
@@ -345,6 +345,16 @@
         "## 4. Train"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Launch SDA training process on NLP domain BERT, and will produce the fine-tuned BERT performance on SQuADv1.1 task.\n",
+        "\n",
+        "Noted: Below performance result is using sample dataset and small iterations to demonstrate its function. The actual performance result please refers to the [performance section](#performance)."
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,

demo/denas/bert/DENAS_BERT_DEMO.ipynb

@@ -87,7 +87,7 @@
     "\n",
     "## 1. Environment Setup\n",
     "\n",
-    "### Option 1 Setup Environment with Pip"
+    "### (Option 1) Use Pip install"
    ]
   },
   {
@@ -104,7 +104,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Option 2 Setup Environment with Docker\n",
+    "### (Option 2) Use Docker\n",
     "\n",
     "Step1. prepare code\n",
     "\n",
@@ -413,10 +413,13 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Launch DE-NAS training process on NLP domain BERT with the input of overall training configuration `/home/vmagent/app/e2eaiok/conf/denas/nlp/e2eaiok_denas_train_bert.conf`, and will produce the fine-tuned BERT performance on SQuADv1.1 task."
+    "Launch DE-NAS training process on NLP domain BERT with the input of overall training configuration `/home/vmagent/app/e2eaiok/conf/denas/nlp/e2eaiok_denas_train_bert.conf`, and will produce the fine-tuned BERT performance on SQuADv1.1 task.\n",
+    "\n",
+    "Noted: Below performance result is using sample dataset and small iterations to demonstrate its function. The actual performance result please refers to the [performance section](#performance)."
    ]
   },
   {

demo/denas/computer_vision/DENAS_ViT_DEMO.ipynb

@@ -65,11 +65,11 @@
     "\n",
     "## Performance\n",
     "<center>\n",
-    "<img src=\"./img/denas_vit_stock.png\" width=\"800\"/><figure>DE-NAS ViT performance over stock model</figure>\n",
+    "<img src=\"./img/denas_vit_stock.png\" width=\"500\"/><figure>DE-NAS ViT performance over stock model</figure>\n",
     "</center>\n",
     "\n",
     "<center>\n",
-    "<img src=\"./img/denas_vit_autoformer.png\" width=\"800\"/><figure>DE-NAS ViT performance over Autoformer</figure>\n",
+    "<img src=\"./img/denas_vit_autoformer.png\" width=\"500\"/><figure>DE-NAS ViT performance over Autoformer</figure>\n",
     "</center>\n",
     "\n",
     "* Testing methodology\n",
@@ -91,7 +91,7 @@
     "\n",
     "## 1. Environment Setup\n",
     "\n",
-    "### Option 1 Setup Environment with Pip"
+    "### (Option 1) Use Pip install"
    ]
   },
   {
@@ -110,7 +110,7 @@
    "id": "def012f3",
    "metadata": {},
    "source": [
-    "### Option 2 Setup Environmental with Docker\n",
+    "### (Option 2) Use Docker\n",
     "\n",
     "Step1. prepare code\n",
     "``` bash\n",
@@ -444,12 +444,15 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "ad47332c",
    "metadata": {},
    "source": [
     "### Train the best searched ViT model\n",
-    "The input is the configuration for VIT domain best model train, edit the configuration file of `e2eaiok_denas_train_vit.conf` for VIT best model train, and then run below commond line"
+    "The input is the configuration for VIT domain best model train, edit the configuration file of `e2eaiok_denas_train_vit.conf` for VIT best model train.\n",
+    "\n",
+    "Noted: Below performance result is using sample dataset and small iterations to demonstrate its function. The actual performance result please refers to the [performance section](#performance)."
    ]
   },
   {