Install (#35)

* remove conda pynini install

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added pynini install note

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix text

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

---------

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: yzhang123

README.md

@@ -54,6 +54,11 @@ Use this installation mode if you want the latest released version.
 pip install nemo_text_processing
 ```
 
+**_NOTE:_** This should work on any Linux OS with x86_64. Pip installation on MacOS and Windows are not supported due to the dependency [Pynini](https://www.openfst.org/twiki/bin/view/GRM/Pynini). On a platform other than Linux x86_64, installing from Pip tries to compile Pynini from scratch, and requires OpenFst headers and libraries to be in the expected place. So if it's working for you, it's because you happen to have installed OpenFst in the right way in the right place. So if you want to Pip install Pynini on MacOS, you have to have pre-compiled and pre-installed OpenFst. The Pynini README for that version should tell you which version it needs and what `--enable-foo` flags to use.
+Instead, we recommend you to use conda-forge to install Pynini on MacOS or Windows:
+`conda install -c conda-forge pynini=2.1.5`.
+
+
 ###  Pip from source
 
 Use this installation mode if you want the a version from particular GitHub branch (e.g main).

install_pynini.sh

@@ -21,7 +21,4 @@ else
     ${PIP} install "${DIST_FILE}[all]"
 fi
 
-echo 'Installing additional pynini dependency'
-bash install_pynini.sh > /dev/null 2>&1 && echo "nemo_text_processing installed!" || echo "nemo_text_processing could not be installed!"
-
 echo 'All done!'

reinstall.sh

@@ -3,6 +3,7 @@ inflect
 jiwer>2.3.0
 joblib
 pandas
+pynini==2.1.5
 regex
 sacremoses>=0.0.43
 setuptools==59.5.0

requirements/requirements.txt

@@ -2,6 +2,7 @@ black==19.10b0
 click==8.0.2
 isort[requirements] < 5
 parameterized
+pynini==2.1.5
 pytest
 pytest-runner
 ruamel.yaml

requirements/requirements_test.txt

@@ -61,11 +61,7 @@
    "source": [
     "## Install NeMo, which installs both nemo and nemo_text_processing package\n",
     "BRANCH = 'main'\n",
-    "!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[nlp]\n",
-    "\n",
-    "# install Pynini for text normalization\n",
-    "! wget https://raw.githubusercontent.com/NVIDIA/NeMo/main/nemo_text_processing/install_pynini.sh\n",
-    "! bash install_pynini.sh"
+    "!python -m pip install git+https://github.com/NVIDIA/NeMo-text-processing.git@$BRANCH#egg=nemo_text_processing"
    ]
   },
   {
@@ -142,7 +138,7 @@
     "print(normalized)"
    ]
   },
-    {
+  {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -446,7 +442,7 @@
    "toc_visible": true
   },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.8.10 64-bit",
    "language": "python",
    "name": "python3"
   },
@@ -460,7 +456,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.7"
+   "version": "3.8.10"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
+   }
   }
  },
  "nbformat": 4,

tutorials/Text_(Inverse)_Normalization.ipynb

@@ -28,7 +28,7 @@
    "source": [
     "\n",
     "## Prerequisites:\n",
-    "1. Please make sure to read the [Text Processing Documentation](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/text_normalization/wfst/intro.html) and [Text Normalization Introduction Tutorial](https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/text_processing/Text_(Inverse)_Normalization.ipynb) **before** this notebook. This notebook is a in-depth tutorial on how to customize and develop your own text normalization or inverse text normalization grammars.\n",
+    "1. Please make sure to read the [Text Processing Documentation](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/text_normalization/wfst/intro.html) and [Text Normalization Introduction Tutorial](https://colab.research.google.com/github/NVIDIA/NeMo-text-processing/blob/main/tutorials/Text_(Inverse)_Normalization.ipynb) **before** this notebook. This notebook is a in-depth tutorial on how to customize and develop your own text normalization or inverse text normalization grammars.\n",
     "2. download NeMo source code"
    ]
   },
@@ -40,11 +40,7 @@
    "source": [
     "## Install NeMo, which installs both nemo and nemo_text_processing package\n",
     "BRANCH = 'main'\n",
-    "!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[nemo_text_processing]\n",
-          "\n",
-          "# install Pynini for text normalization\n",
-          "! wget https://raw.githubusercontent.com/NVIDIA/NeMo/main/nemo_text_processing/install_pynini.sh\n",
-          "! bash install_pynini.sh"
+    "!python -m pip install git+https://github.com/NVIDIA/NeMo-text-processing.git@$BRANCH#egg=nemo_text_processing"
    ]
   },
   {
@@ -7028,11 +7024,8 @@
    "provenance": [],
    "toc_visible": true
   },
-  "interpreter": {
-   "hash": "fbc643a332f9d7801191710b24a8a955d342df4f32791f7fb65121dc4784751f"
-  },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.8.10 64-bit",
    "language": "python",
    "name": "python3"
   },
@@ -7046,7 +7039,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.7"
+   "version": "3.8.10"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
+   }
   }
  },
  "nbformat": 4,