Skip to content

Commit ca008c9

Browse files
brosenberg42jrobble
brosenberg42
and
jrobble
authored
Update docs for Python 3. (#105)
* Update docs for Python 3. * Update formatting. * Update Python3 install instructions. * Fix typo Co-authored-by: jrobble <jrobble@mitre.org>
1 parent 4281e33 commit ca008c9

7 files changed

Lines changed: 344 additions & 124 deletions

File tree

‎docs/docs/Development-Environment-Guide.md‎

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,7 @@ Install [CentOS 7](https://www.centos.org/download/). Most developers use a virt
5555

5656
Use `yum` to install packages:
5757

58-
`sudo yum install -y asciidoc autoconf automake boost boost-devel cmake3 curl freetype-devel gcc-c++ git graphviz gstreamer-plugins-base-devel gtk2-devel gtkglext-devel gtkglext-libs jasper jasper-devel libavc1394-devel libcurl-devel libdc1394-devel libffi-devel libICE-devel libjpeg-turbo-devel libpng-devel libSM-devel libtiff-devel libtool libv4l-devel libXinerama-devel libXmu-devel libXt-devel log4cplus log4cplus-devel log4cxx log4cxx-devel make mercurial mesa-libGL-devel mesa-libGLU-devel nasm ncurses-devel numpy openssl-devel pangox-compat pangox-compat-devel perl-CPAN-Meta-YAML perl-DBI perl-Digest-MD5 perl-File-Find-Rule perl-File-Find-Rule-Perl perl-JSON perl-JSON-PP perl-List-Compare perl-Number-Compare perl-Params-Util perl-Parse-CPAN-Meta php pkgconfig python-devel python-httplib2 python-jinja2 python-keyczar python2-paramiko python2-pip python-setuptools python-six PyYAML qt qt-devel qt-x11 redis rpm-build sshpass tbb tbb-devel tree unzip uuid-devel wget yasm yum-utils zlib-devel`
59-
60-
The version of `pip` available in the yum repository is old and must be upgraded with the following command:
61-
62-
`sudo pip install --upgrade pip`
58+
`sudo yum install -y asciidoc autoconf automake boost boost-devel cmake3 curl freetype-devel gcc-c++ git graphviz gstreamer-plugins-base-devel gtk2-devel gtkglext-devel gtkglext-libs jasper jasper-devel libavc1394-devel libcurl-devel libdc1394-devel libffi-devel libICE-devel libjpeg-turbo-devel libpng-devel libSM-devel libtiff-devel libtool libv4l-devel libXinerama-devel libXmu-devel libXt-devel log4cplus log4cplus-devel log4cxx log4cxx-devel make mercurial mesa-libGL-devel mesa-libGLU-devel nasm ncurses-devel numpy openssl-devel pangox-compat pangox-compat-devel perl-CPAN-Meta-YAML perl-DBI perl-Digest-MD5 perl-File-Find-Rule perl-File-Find-Rule-Perl perl-JSON perl-JSON-PP perl-List-Compare perl-Number-Compare perl-Params-Util perl-Parse-CPAN-Meta php pkgconfig qt qt-devel qt-x11 redis rpm-build sshpass tbb tbb-devel tree unzip uuid-devel wget yasm yum-utils zlib-devel`
6359

6460

6561
# Get the OpenMPF Source Code
@@ -69,7 +65,7 @@ The version of `pip` available in the yum repository is old and must be upgraded
6965
2. `git clone https://github.com/openmpf/openmpf-projects.git --recursive`
7066

7167
2. Install the OpenMPF command line tools:
72-
<br> `sudo pip install /home/mpf/openmpf-projects/openmpf/trunk/bin/mpf-scripts`
68+
<br> `sudo pip3 install /home/mpf/openmpf-projects/openmpf/trunk/bin/mpf-scripts`
7369

7470
3. Copy the mpf user profile script from the extracted source code:
7571
<br> `sudo cp /home/mpf/openmpf-projects/openmpf/trunk/mpf-install/src/main/scripts/mpf-profile.sh /etc/profile.d/mpf.sh`
@@ -155,6 +151,40 @@ For reference only: <http://ant.apache.org>
155151
<br>`sudo -i -u postgres createdb -O mpf mpf`
156152
157153
154+
## Python 3.8
155+
1. Install build dependencies:
156+
<br> `sudo yum install -y yum-utils`
157+
<br>`sudo yum-builddep -y python3`
158+
159+
2. Download the source code:
160+
<br>`curl https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz | tar --extract --xz`
161+
162+
3. Build Python:
163+
<br> `cd Python-3.8.2`
164+
<br> `./configure --enable-optimizations --with-lto --enable-shared`
165+
<br> `make -j8`
166+
<br> `sudo make install`
167+
<br> `sudo ln -s /usr/local/lib/libpython3.8.so.1.0 /usr/lib64/libpython3.8.so.1.0`
168+
<br> `sudo ln -sf /usr/local/bin/python3 /bin/python3`
169+
<br> `sudo ln -sf /usr/local/bin/python3.8 /bin/python3.8`
170+
<br> `sudo ln -sf /usr/local/bin/pip3 /bin/pip3`
171+
<br> `sudo ln -sf /usr/local/bin/pip3.8 /bin/pip3.8`
172+
173+
4. Make sure the output of running `python3 --version` is `Python 3.8.2`.
174+
175+
5. Make sure the output of running `sudo python3 --version` is `Python 3.8.2`.
176+
177+
6. Make sure the output of running `pip3 --version` ends with `(python 3.8)`.
178+
179+
7. Make sure the output of running `sudo pip3 --version` ends with `(python 3.8)`.
180+
181+
8. Upgrade pip:
182+
<br> `sudo pip3 install --upgrade pip`
183+
184+
9. Install wheel:
185+
<br> `sudo pip3 install wheel`
186+
187+
158188
# Configure System Dependencies
159189
160190
# Configure ActiveMQ

‎docs/docs/Python-Batch-Component-API.md‎

Lines changed: 109 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -138,8 +138,8 @@ If you have openmpf-build-tools, then you can run:
138138
```
139139
To setup the libraries manually you can run:
140140
```bash
141-
pip wheel -w ~/mpf-sdk-install/python/wheelhouse ~/openmpf-projects/openmpf-python-component-sdk/detection/api
142-
pip wheel -w ~/mpf-sdk-install/python/wheelhouse ~/openmpf-projects/openmpf-python-component-sdk/detection/component_util
141+
pip3 wheel -w ~/mpf-sdk-install/python/wheelhouse ~/openmpf-projects/openmpf-python-component-sdk/detection/api
142+
pip3 wheel -w ~/mpf-sdk-install/python/wheelhouse ~/openmpf-projects/openmpf-python-component-sdk/detection/component_util
143143
```
144144

145145

@@ -159,7 +159,7 @@ ComponentName
159159
├── descriptor
160160
│ └── descriptor.json
161161
└── wheelhouse # optional
162-
└── my_prebuilt_lib-0.1-py2-none-any.whl
162+
└── my_prebuilt_lib-0.1-py3-none-any.whl
163163
```
164164

165165
**1\. Create directory structure:**
@@ -257,11 +257,11 @@ MyComponent
257257
├── descriptor
258258
│ └── descriptor.json
259259
└── wheelhouse
260-
├── MyComponent-0.1-py2-none-any.whl
261-
├── mpf_component_api-0.1-py2-none-any.whl
262-
├── mpf_component_util-0.1-py2-none-any.whl
263-
├── numpy-1.14.4-cp27-cp27mu-manylinux1_x86_64.whl
264-
└── opencv_python-3.4.1.15-cp27-cp27mu-manylinux1_x86_64.whl
260+
├── MyComponent-0.1-py3-none-any.whl
261+
├── mpf_component_api-0.1-py3-none-any.whl
262+
├── mpf_component_util-0.1-py3-none-any.whl
263+
├── numpy-1.18.4-cp38-cp38-manylinux1_x86_64.whl
264+
└── opencv_python-4.2.0.34-cp38-cp38-manylinux1_x86_64.whl
265265
```
266266

267267
To create the plugin packages you can run the build script as follows:
@@ -273,7 +273,7 @@ The plugin package can also be built manually using the following commands:
273273
```bash
274274
mkdir -p plugin-packages/MyComponent/wheelhouse
275275
cp -r MyComponent/plugin-files/* plugin-packages/MyComponent/
276-
pip wheel -w plugin-packages/MyComponent/wheelhouse -f ~/mpf-sdk-install/python/wheelhouse -f plugin-packages/MyComponent/wheelhouse ./MyComponent/
276+
pip3 wheel -w plugin-packages/MyComponent/wheelhouse -f ~/mpf-sdk-install/python/wheelhouse -f plugin-packages/MyComponent/wheelhouse ./MyComponent/
277277
cd plugin-packages
278278
tar -zcf MyComponent.tar.gz MyComponent
279279
```
@@ -510,7 +510,7 @@ def __init__(self, x_left_upper, y_left_upper, width, height, confidence=-1.0, d
510510
| width | `int` | The width of the detected object. |
511511
| height | `int` | The height of the detected object. |
512512
| confidence | `float` | Represents the "quality" of the detection. The range depends on the detection algorithm. 0.0 is lowest quality. Higher values are higher quality. Using a standard range of [0.0 - 1.0] is advised. If the component is unable to supply a confidence value, it should return -1.0. |
513-
| detection_properties | `mpf_component_api.Properties` | Dict-like object with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
513+
| detection_properties | `dict[str, str]` | A dict with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
514514

515515
* Example:
516516

@@ -713,8 +713,8 @@ def __init__(self, start_frame, stop_frame, confidence=-1.0, frame_locations=Non
713713
| start_frame | `int` | The first frame number (0-based index) that contained the detected object. |
714714
| stop_frame | `int` | The last frame number (0-based index) that contained the detected object. |
715715
| confidence | `float` | Represents the "quality" of the detection. The range depends on the detection algorithm. 0.0 is lowest quality. Higher values are higher quality. Using a standard range of [0.0 - 1.0] is advised. If the component is unable to supply a confidence value, it should return -1.0. |
716-
| frame_locations | `mpf_component_api.FrameLocationMap` | A dict-like object of individual detections. The key for each entry is the frame number where the detection was generated, and the value is a `mpf_component_api.ImageLocation` calculated as if that frame was a still image. Note that a key-value pair is *not* required for every frame between the track start frame and track stop frame. |
717-
| detection_properties | `mpf_component_api.Properties` | Dict-like object with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
716+
| frame_locations | `dict[int, mpf_component_api.ImageLocation]` | A dict of individual detections. The key for each entry is the frame number where the detection was generated, and the value is a `mpf_component_api.ImageLocation` calculated as if that frame was a still image. Note that a key-value pair is *not* required for every frame between the track start frame and track stop frame. |
717+
| detection_properties | `dict[str, str]` | A dict with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
718718

719719
> **NOTE:** Currently, `mpf_component_api.VideoTrack.detection_properties` do not show up in the JSON output object or
720720
> are used by the WFM in any way.
@@ -927,7 +927,7 @@ def __init__(self, start_time, stop_time, confidence, detection_properties=None)
927927
| start_time | `int` | The time (0-based index, in ms) when the audio detection event started. |
928928
| stop_time | `int` | The time (0-based index, in ms) when the audio detection event stopped. |
929929
| confidence | `float` | Represents the "quality" of the detection. The range depends on the detection algorithm. 0.0 is lowest quality. Higher values are higher quality. Using a standard range of [0.0 - 1.0] is advised. If the component is unable to supply a confidence value, it should return -1.0. |
930-
| detection_properties | `mpf_component_api.Properties` | Dict-like object with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
930+
| detection_properties | `dict[str, str]` | A dict with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
931931

932932
> **NOTE:** Currently, `mpf_component_api.AudioTrack.detection_properties` do not show up in the JSON output object or
933933
> are used by the WFM in any way.
@@ -1030,38 +1030,110 @@ def __init__(self, confidence=-1.0, detection_properties=None):
10301030
| Member | Data Type | Description |
10311031
|----------------------|-----------|-------------|
10321032
| confidence | `float` | Represents the "quality" of the detection. The range depends on the detection algorithm. 0.0 is lowest quality. Higher values are higher quality. Using a standard range of [0.0 - 1.0] is advised. If the component is unable to supply a confidence value, it should return -1.0. |
1033-
| detection_properties | `mpf_component_api.Properties` | Dict-like object with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
1033+
| detection_properties | `dict[str, str]` | A dict with keys and values of type `str` containing optional additional information about the detected object. For best practice, keys should be in all CAPS. |
10341034

10351035

10361036
# Python Component Build Environment
1037-
All Python components must work with CPython 2.7. Also, Python components must work with the Linux version that is
1037+
All Python components must work with CPython 3.8.2. Also, Python components must work with the Linux version that is
10381038
used by the OpenMPF Component Executable. At this writing, OpenMPF runs on CentOS 7.4.1708 (kernel version 3.10.0-693).
10391039
Pure Python code should work on any OS, but incompatibility issues can arise when using Python libraries that include
10401040
compiled extension modules. Python libraries are typically distributed as wheel files. The wheel format requires that
10411041
the file name follows the pattern of
10421042
`<dist_name>-<version>-<python_tag>-<abi_tag>-<platform_tag>.whl`. `<python_tag>-<abi_tag>-<platform_tag>` are called
1043-
[compatibility tags](https://www.python.org/dev/peps/pep-0425). For example, `mpf_component_api` is pure Python
1044-
so the name of its wheel file is `mpf_component_api-0.1-py2-none-any.whl`. `py2` means it will work with any Python 2
1043+
[compatibility tags](https://www.python.org/dev/peps/pep-0425). For example, `mpf_component_api` is pure Python,
1044+
so the name of its wheel file is `mpf_component_api-0.1-py3-none-any.whl`. `py3` means it will work with any Python 3
10451045
implementation because it does not use any implementation-specific features. `none` means that it does not use the
10461046
Python ABI. `any` means it will work on any platform.
10471047

1048-
Supported Python Tags:
1049-
1050-
* `py2`
1051-
* `py27`
1052-
* `cp2`
1053-
* `cp27`
1054-
1055-
Supported ABI Tags:
1056-
1057-
* `none`
1058-
* `cp27mu`
1059-
1060-
Supported Platform Tags:
1061-
1062-
* `any`
1063-
* `manylinux1_x86_64`
1064-
1048+
The following combinations of compatibility tags are supported:
1049+
1050+
* `cp32-abi3-linux_x86_64`
1051+
* `cp32-abi3-manylinux1_x86_64`
1052+
* `cp32-abi3-manylinux2010_x86_64`
1053+
* `cp32-abi3-manylinux2014_x86_64`
1054+
* `cp33-abi3-linux_x86_64`
1055+
* `cp33-abi3-manylinux1_x86_64`
1056+
* `cp33-abi3-manylinux2010_x86_64`
1057+
* `cp33-abi3-manylinux2014_x86_64`
1058+
* `cp34-abi3-linux_x86_64`
1059+
* `cp34-abi3-manylinux1_x86_64`
1060+
* `cp34-abi3-manylinux2010_x86_64`
1061+
* `cp34-abi3-manylinux2014_x86_64`
1062+
* `cp35-abi3-linux_x86_64`
1063+
* `cp35-abi3-manylinux1_x86_64`
1064+
* `cp35-abi3-manylinux2010_x86_64`
1065+
* `cp35-abi3-manylinux2014_x86_64`
1066+
* `cp36-abi3-linux_x86_64`
1067+
* `cp36-abi3-manylinux1_x86_64`
1068+
* `cp36-abi3-manylinux2010_x86_64`
1069+
* `cp36-abi3-manylinux2014_x86_64`
1070+
* `cp37-abi3-linux_x86_64`
1071+
* `cp37-abi3-manylinux1_x86_64`
1072+
* `cp37-abi3-manylinux2010_x86_64`
1073+
* `cp37-abi3-manylinux2014_x86_64`
1074+
* `cp38-abi3-linux_x86_64`
1075+
* `cp38-abi3-manylinux1_x86_64`
1076+
* `cp38-abi3-manylinux2010_x86_64`
1077+
* `cp38-abi3-manylinux2014_x86_64`
1078+
* `cp38-cp38-linux_x86_64`
1079+
* `cp38-cp38-manylinux1_x86_64`
1080+
* `cp38-cp38-manylinux2010_x86_64`
1081+
* `cp38-cp38-manylinux2014_x86_64`
1082+
* `cp38-none-any`
1083+
* `cp38-none-linux_x86_64`
1084+
* `cp38-none-manylinux1_x86_64`
1085+
* `cp38-none-manylinux2010_x86_64`
1086+
* `cp38-none-manylinux2014_x86_64`
1087+
* `py30-none-any`
1088+
* `py30-none-linux_x86_64`
1089+
* `py30-none-manylinux1_x86_64`
1090+
* `py30-none-manylinux2010_x86_64`
1091+
* `py30-none-manylinux2014_x86_64`
1092+
* `py31-none-any`
1093+
* `py31-none-linux_x86_64`
1094+
* `py31-none-manylinux1_x86_64`
1095+
* `py31-none-manylinux2010_x86_64`
1096+
* `py31-none-manylinux2014_x86_64`
1097+
* `py32-none-any`
1098+
* `py32-none-linux_x86_64`
1099+
* `py32-none-manylinux1_x86_64`
1100+
* `py32-none-manylinux2010_x86_64`
1101+
* `py32-none-manylinux2014_x86_64`
1102+
* `py33-none-any`
1103+
* `py33-none-linux_x86_64`
1104+
* `py33-none-manylinux1_x86_64`
1105+
* `py33-none-manylinux2010_x86_64`
1106+
* `py33-none-manylinux2014_x86_64`
1107+
* `py34-none-any`
1108+
* `py34-none-linux_x86_64`
1109+
* `py34-none-manylinux1_x86_64`
1110+
* `py34-none-manylinux2010_x86_64`
1111+
* `py34-none-manylinux2014_x86_64`
1112+
* `py35-none-any`
1113+
* `py35-none-linux_x86_64`
1114+
* `py35-none-manylinux1_x86_64`
1115+
* `py35-none-manylinux2010_x86_64`
1116+
* `py35-none-manylinux2014_x86_64`
1117+
* `py36-none-any`
1118+
* `py36-none-linux_x86_64`
1119+
* `py36-none-manylinux1_x86_64`
1120+
* `py36-none-manylinux2010_x86_64`
1121+
* `py36-none-manylinux2014_x86_64`
1122+
* `py37-none-any`
1123+
* `py37-none-linux_x86_64`
1124+
* `py37-none-manylinux1_x86_64`
1125+
* `py37-none-manylinux2010_x86_64`
1126+
* `py37-none-manylinux2014_x86_64`
1127+
* `py38-none-any`
1128+
* `py38-none-linux_x86_64`
1129+
* `py38-none-manylinux1_x86_64`
1130+
* `py38-none-manylinux2010_x86_64`
1131+
* `py38-none-manylinux2014_x86_64`
1132+
* `py3-none-any`
1133+
* `py3-none-linux_x86_64`
1134+
* `py3-none-manylinux1_x86_64`
1135+
* `py3-none-manylinux2010_x86_64`
1136+
* `py3-none-manylinux2014_x86_64`
10651137

10661138
Components should be supplied as a tar file, which includes not only the component library, but any other libraries or
10671139
files needed for execution. This includes all other non-standard libraries used by the component
@@ -1082,11 +1154,9 @@ OpenMPF components should be stateless in operation and give identical output fo
10821154

10831155
## Logging
10841156
It recommended that components use the logger returned from:
1085-
<br> `mpf_component_util.configure_logging(log_file_name, is_debug=False)`. When `is_debug` is false, the log messages
1086-
will be written to
1087-
<br> `${MPF_LOG_PATH}/${THIS_MPF_NODE}/log/<log_file_name>.log` When `is_debug` is true, the log messages
1088-
will be written to standard out. Note that multiple instances of the same component can log to the same file.
1089-
Also, logging content can span multiple lines. The following log levels are supported:
1157+
<br> `mpf_component_api.configure_logging(log_file_name, debug=False, replace_existing_config=True)`.
1158+
The logger will write log messages to standard out. When `debug` is false, the log messages will also be written to `${MPF_LOG_PATH}/${THIS_MPF_NODE}/log/<log_file_name>.log` Note that multiple instances of the same component
1159+
can log to the same file. Also, logging content can span multiple lines. The following log levels are supported:
10901160
`FATAL, ERROR, WARN, INFO, DEBUG`.
10911161

10921162
The format of the log messages is:

0 commit comments

Comments
 (0)