Merge pull request #123 from lambda-feedback/tr135-update-algorithm

Tr135 update algorithm

Author: KarlLundengaard

app/docs/dev.md

@@ -1,7 +1,10 @@
 # ComparePhysicalQuantities
-This is an **EXPERIMENTAL** evaluation function with some dimensional analysis functionality.
 
-This function lacks a nice GUI, can be quite brittle, and will likely change significantly in the near future.
+Evaluation function which proveds some basic some dimensional analysis functionality.
+
+- **DEPRECATED** Comparing physical quantities **RECOMMENDED ALTERNATIVE:** CompareExpressions with the `physical_quantity` parameter set to `true`
+- Substitutions of symbols before comparison of expressions is done
+- Checking if a comma separated list of expressions can be interpreted as a set groups that satisfies the Buckingham Pi theorem
 
 **Note:** When the `quantities` grading parameter is set, this function cannot handle short form symbols for units. Thus when defining quantities all units must be given with full names in lower-case letters. For example `Nm/s` or `Newton*metre/SECOND` will not be handled correctly, but `newton*metre/second` will.
 
@@ -21,11 +24,40 @@ This function lacks a nice GUI, can be quite brittle, and will likely change sig
 
 **Note:** When running the unit test some tests are expected to take much longer than the other. These tests can be skipped by adding `skip_resource_intensive_tests` as a command line argument to improve iteration times.
 
+## Changing default feedback messages
+
+The feedback messages can be set on a per-task basis (see description of the `custom_feedback` input parameter).
+
+The default feedback messages are defined in `feedback_responses_list` defined near the top of `evaulation.py`, which contains a list of dictionaries of feedback responses that are used througout the code. All feedback messages visible to learners are defined in these dictionaries. The entries in the dictionaries are either be string of functions that return strings.
+
 ## Inputs
 All input parameters need to be supplied via the **Grading parameters** panel.
 
 There are seven optional parameters that can be set: `elementary_functions`, `substitutions`, `quantities`, `strict_syntax`, `rtol`, `atol` and `comparison`.
 
+## `custom_feedback`
+
+Custom feedback can be set on a per-task basis. **Note:** Custom feedback only supports fixed strings, this means that for some situations the custom feedback cannot be as detailed as the default feedback.
+
+The parameter must be set as a dictionary with keys from the feedback tags listed below. The value for each key can be any string.
+
+### Feedback tags for all comparisons
+- `PARSE_ERROR_WARNING` Response cannot be parsed as an expression or physical quantity.
+- `PER_FOR_DIVISION` Warns about risk of ambiguity when using `per` instead `/` for division.
+- `STRICT_SYNTAX_EXPONENTIATION` Warns that `^` cannot be used for exponentiation when `strict_syntax` is set to `true`.
+- `QUANTITIES_NOT_WRITTEN_CORRECTLY` Text in error message that appears if list of quantities could not be parsed.
+- `SUBSTITUTIONS_NOT_WRITTEN_CORRECTLY` Text in error message that appears if list of substitutions could not be parsed.
+
+### Feedback tags for `buckinghamPi` comparison
+
+- `VALID_CANDIDATE_SET` Message that is displayed when a response is found to be a valid set of groups. **Note:** setting this will not affect the Correct/Incorrect message, it will only add further text.
+- `NOT_DIMENSIONLESS` Message displayed when at least one groups is not dimensionless.
+- `MORE_GROUPS_THAN_REFERENCE_SET` Message displayed when the response contains more groups than necessary.
+- `CANDIDATE_GROUPS_NOT_INDEPENDENT` Message displayed when the groups in the response are not independent.
+- `TOO_FEW_INDEPENDENT_GROUPS` Message displayed when the response contains fewer groups than necessary.
+- `UNKNOWN_SYMBOL` Message displayed when the response contains some undefined symbol.
+- `SUM_WITH_INDEPENDENT_TERMS`  Message displayed when the response has too few groups but one (or more) of the groups is a sum with independent terms.
+
 ## `elementary_functions`
 
 When using implicit multiplication function names with mulitple characters are sometimes split and not interpreted properly. Setting `elementary_functions` to true will reserve the function names listed below and prevent them from being split. If a name is said to have one or more alternatives this means that it will accept the alternative names but the reserved name is what will be shown in the preview.
@@ -60,8 +92,6 @@ If the `comparison` parameter is set to `buckinghamPi`, then `quantities` should
 
 #### Table: Base SI units
 
-These tables list the units defined in `unit_system_conversions.py`. Note that the evaluation function uses static version of these arrays created by running `generate_unit_conversion_arrays.py` and stored in `static_unit_conversion_arrays.py`.
-
 SI base units taken from Table 1 of https://physics.nist.gov/cuu/Units/units.html
 
 Note that gram is used as a base unit instead of kilogram.
@@ -101,29 +131,29 @@ Note that degrees Celsius is omitted.
 
 Note that the function treats radians and steradians as dimensionless values.
 
-| Unit name | Symbol | Expressed in base SI units                          |
-|-----------|:-------|:----------------------------------------------------|
-| radian    |   r    | 1                                                   |
-| steradian |  sr    | 1                                                   |
-| hertz     |  Hz    | second$^{-1}$                                       |
-| newton    |   N    | metre kilogram second$^{-2}$                        |
-| pascal    |  Pa    | metre$^{-1}$ kilogram second$^{-2}$                 |
-| joule     |   J    | metre$^2$ kilogram second$^{-2}$                    |
-| watt      |   W    | metre$^2$ kilogram second$^{-3}$                    |
-| coulomb   |   C    | second ampere                                       |
-| volt      |   V    | metre$^2$ kilogram second$^{-3}$ ampere$^{-1}$      |
-| farad     |   F    | metre$^{-2}$ kilogram$^{-1}$ second$^4$ ampere$^2$  |
-| ohm       |   O    | metre$^2$ kilogram second$^{-3}$ ampere$^{-2}$      |
-| siemens   |   S    | metre$^{-2}$ kilogram$^{-1}$ second$^3$ ampere$^2$  |
-| weber     |  Wb    | metre$^2$ kilogram second$^{-2}$ ampere$^{-1}$      |
-| tesla     |   T    | kilo gram second$^{-2}$ ampere$^{-1}$               |
-| henry     |   H    | metre$^2$ kilogram second$^{-2}$ ampere$^{-2}$      |
-| lumen     |  lm    | candela                                             |
-| lux       |  lx    | metre$^{-2}$ candela                                |
-| becquerel |  Bq    | second$^{-1}$                                       |
-| gray      |  Gy    | metre$^2$ second$^{-2}$                             |
-| sievert   |  Sv    | metre$^2$ second$^{-2}$                             |
-| katal     |  kat   | mole second$^{-1}$                                  |
+| Unit name | Symbol | Expressed in base SI units                                                       |
+|-----------|:-------|:---------------------------------------------------------------------------------|
+| radian    |   r    | 1                                                                                |
+| steradian |  sr    | 1                                                                                |
+| hertz     |  Hz    | $\mathrm{second}^{-1}$                                                           |
+| newton    |   N    | $\mathrm{metre}~\mathrm{kilogram}~\mathrm{second}^{-2}$                         |
+| pascal    |  Pa    | $\mathrm{metre}^{-1}~\mathrm{kilogram}~\mathrm{second}^{-2}$                     |
+| joule     |   J    | $\mathrm{metre}^2~\mathrm{kilogram~second}^{-2}$                                 |
+| watt      |   W    | $\mathrm{metre}^2~\mathrm{kilogram~second}^{-3}$                                 |
+| coulomb   |   C    | $\mathrm{second~ampere}$                                                         |
+| volt      |   V    | $\mathrm{metre}^2~\mathrm{kilogram second}^{-3}~\mathrm{ampere}^{-1}$            |
+| farad     |   F    | $\mathrm{metre}^{-2}~\mathrm{kilogram}^{-1}~\mathrm{second}^4~\mathrm{ampere}^2$ |
+| ohm       |   O    | $\mathrm{metre}^2~\mathrm{kilogram second}^{-3}~\mathrm{ampere}^{-2}$            |
+| siemens   |   S    | $\mathrm{metre}^{-2}~\mathrm{kilogram}^{-1}~\mathrm{second}^3~\mathrm{ampere}^2$ |
+| weber     |  Wb    | $\mathrm{metre}^2~\mathrm{kilogram~second}^{-2}~\mathrm{ampere}^{-1}$            |
+| tesla     |   T    | $\mathrm{kilogram~second}^{-2} \mathrm{ampere}^{-1}$                            |
+| henry     |   H    | $\mathrm{metre}^2~\mathrm{kilogram~second}^{-2}~\mathrm{ampere}^{-2}$            |
+| lumen     |  lm    | $\mathrm{candela}$                                                               |
+| lux       |  lx    | $\mathrm{metre}^{-2}~\mathrm{candela}$                                            |
+| becquerel |  Bq    | $\mathrm{second}^{-1}$                                                            |
+| gray      |  Gy    | $\mathrm{metre}^2~\mathrm{second}^{-2}$                                          |
+| sievert   |  Sv    | $\mathrm{metre}^2~\mathrm{second}^{-2}$                                          |
+| katal     |  kat   | $\mathrm{mole~second}^{-1}$                                                      |
 
 #### Table: Common non-SI units
 
@@ -133,35 +163,54 @@ Note that the function treats angles, neper and bel as dimensionless values.
 
 Note that only the first table in this section has short form symbols defined, the second table does not.
 
+| Unit name         | Symbol | Expressed in SI units                      |
+|-------------------|:-------|:-------------------------------------------|
+| minute            |  min   | $60~\mathrm{second}$                       |
+| hour              |   h    | $3600~\mathrm{second}$                     |
+| degree            |  deg   | $\frac{\pi}{180}$                          |
+| liter             |   l    | $10^{-3}~\mathrm{metre}^3$                 |
+| metric_ton        |   t    | $10^3~\mathrm{kilogram}$                   |
+| neper             |  Np    | $1$                                        |
+| bel               |   B    | $\frac{1}{2}~\ln(10)$                      |
+| electronvolt      |  eV    | $1.60218 \cdot 10^{-19}~\mathrm{joule}$    |
+| atomic_mass_unit  |   u    | $1.66054 \cdot 10^{-27}~\mathrm{kilogram}$ |
+| angstrom          |   å    | $10^{-10}~\mathrm{metre}$                  |
+
+| Unit name        | Expressed in SI units                                |
+|------------------|:-----------------------------------------------------|
+| day              | $86400~\mathrm{second}$                              |
+| angleminute      | $\frac{\pi}{10800}$                                  |
+| anglesecond      | $\frac{\pi}{648000}$                                 |
+| astronomicalunit | $149597870700~\mathrm{metre}$                        |
+| nauticalmile     | $1852~\mathrm{metre}$                                |
+| knot             | $\frac{1852}{3600}~\mathrm{metre~second}^{-1}$       |
+| are              | $10^2~\mathrm{metre}^2$                              |
+| hectare          | $10^4~\mathrm{metre}^2$                              |
+| bar              | $10^5~\mathrm{pascal}$                               |
+| barn             | $10^{-28}~\mathrm{metre}$                            |
+| curie            | $3.7 \cdot 10^{10}~\mathrm{becquerel}                |
+| roentgen         | $2.58 \cdot 10^{-4}~\mathrm{kelvin~(kilogram)}^{-1}$ |
+| rad              | $10^{-2}~\mathrm{gray}$                              |
+| rem              | $10^{-2}~\mathrm{sievert}$                           |
+
+#### Table: Imperial units
+
+Commonly imperial units taken from https://en.wikipedia.org/wiki/Imperial_units
+
 | Unit name         | Symbol | Expressed in SI units                         |
 |-------------------|:-------|:----------------------------------------------|
-| minute            |  min   | 60 second                                     |
-| hour              |   h    | 3600 second                                   |
-| degree            |  deg   | $\frac{\pi}{180}$                             |
-| liter             |   l    | $10^{-3}$ metre$^3$                           |
-| metric_ton        |   t    | $10^3$ kilo gram                              |
-| neper             |  Np    | 1                                             |
-| bel               |   B    | $\frac{1}{2} \ln(10)$                         |
-| electronvolt      |  eV    | $1.60218 \cdot 10^{-19}$ joule                |
-| atomic_mass_unit  |   u    | $1.66054 \cdot 10^{-27}$ kilogram             |
-| angstrom          |   å    | $10^{-10}$ metre                              |
-
-| Unit name        | Expressed in SI units                         |
-|------------------|:----------------------------------------------|
-| day              | $86400$ second                                |
-| angleminute      | $\frac{pi}{10800}$                            |
-| anglesecond      | $\frac{pi}{648000}$                           |
-| astronomicalunit | $149597870700$ metre                          |
-| nauticalmile     | $1852$ metre                                  |
-| knot             | $\frac{1852}{3600}$ metre second$^{-1}$       |
-| are              | $10^2$ metre$^2$                              |
-| hectare          | $10^4$ metre$^2$                              |
-| bar              | $10^5$ pascal                                 |
-| barn             | $10^{-28}$ metre                              |
-| curie            | $3.7 \cdot 10^{10}$ becquerel                 |
-| roentgen         | $2.58 \cdot 10^{-4}$ kelvin (kilogram)$^{-1}$ |
-| rad              | $10^{-2}$ gray                                |
-| rem              | $10^{-2}$ sievert                             |
+| inch              |   in   | $0.0254~\mathrm{metre}$                       |
+| foot              |   ft   | $0.3048~\mathrm{metre}$                       |
+| yard              |   yd   | $0.9144~\mathrm{metre}$                       |
+| mile              |   mi   | $1609.344~\mathrm{metre}$                     |
+| fluid ounce       |  fl oz | $28.4130625~\mathrm{millilitre}$              |
+| gill              |   gi   | $142.0653125~\mathrm{millilitre}$             |
+| pint              |   pt   | $568.26125~\mathrm{millilitre}$               |
+| quart             |   qt   | $1.1365225~\mathrm{litre}$                    |
+| gallon            |   gal  | $4546.09~\mathrm{litre}$                      |
+| ounce             |   oz   | $28.349523125~\mathrm{gram}$                  |
+| pound             |   lb   | $0.45359237~\mathrm{kilogram}$                |
+| stone             |   st   | $6.35029318~\mathrm{kilogram}$                |
 
 ### `strict_syntax`
 
@@ -188,7 +237,7 @@ Parameter that determines what kind of comparison is done. There are four possib
  - `dimensions` Checks that the answer and response have the same dimensions, does not compare the values of the physical quantities.
  - `buckinghamPi` Checks that the set of quantities in the response matches the set of quantities in the sense given by the Buckingham Pi theorem.
 
-For more details on each options see the description below and the correspondig examples.
+For more details on each options see the description below and the corresponding examples.
 
 If `comparison` is not specified it defaults to `expression`.
 
@@ -213,17 +262,8 @@ With this option the quantities (specified by the `quantities` parameter) can be
 Checks that the set of quantities in the response matches the set of quantities in the sense given by the Buckingham Pi theorem.
 
 There are three different ways of supplying this function with the necessary information.
-- In the answer, supply a list of groups on the form `['` first group `','` second group `','` ... `','` last group `']`. Note that if there is only one group it still needs to be written as a list `['` group `']`. When used this way the function assumes that the given list is correct and contains at least the minimum number of groups.
+- In the answer, provide an example set of groups as a comma seprated list. When used this way the function assumes that the given list is correct and contains at least the minimum number of groups.
 - In the `quantities` parameter, supply a list of what the dimensions for each quantity is and set answer to `-`. The function will then compute a list of sufficiently many independen dimensionless quantities and compare to the response.
 - In the `quantities` parameter, supply a list of what the dimensions for each quantity is and in the answer, supply a list of groups as in the first option. The function will then check that the supplied answer is dimensionless and has a sufficient number of independent groups before comparing it to the response.
 
-## Outputs
-Outputs vary depending on chosen comparison options. This is likely to change in near future. Below is the minimum common set of outputs.
-```json
-{
-  "command": "eval",
-  "result": {
-    "is_correct": "<bool>",
-  }
-}
-```
+Note that in lists of groups the items should ideally be written on the form $q_1^{c_1} \cdot q_2^{c_2} \cdots q_n^{c_n}$ where $q_1, q_2 \ldots q_n$ are quantities and $c_1, c_2 \ldots c_n$ are integers, but the function can also handle item that are sums with terms written on the form $a \cdot q_1^{c_1} \cdot q_2^{c_2} \cdots q_n^{c_n}$ where $q_1, q_2 \ldots q_n$ are quantities, $c_1, c_2 \ldots c_n$ rational numbers and $a$ a constant. If the total number of groups is less than required the set of groups is considered invalid, even if there is a sufficient number of terms with independent power products in the response.