Reintroduce Python 3.9 testing in build_and_test.yml

[rickecon]

PR #11 incorporates updates to the OpenFisca-UK package into get_micro_data.py and the corresponding tests. For some reason, the updates in that PR caused the code to fail the Python 3.9 tests in build_and_test.py. As a result, PR #11 removed the Python 3.9 tests. We should figure out what the cause of these errors are and reintroduce the Python 3.9 testing requirement.

cc: @jdebacker @nikhilwoodruff

[jdebacker]

PR #2 also failed on 3.9. It looks like it had to do with the tax function estimation finding slightly different parameter values when using the later version of Python:

18s
1
Run pytest -m 'not full_run and not regression' --cov=./ --cov-report=xml
6
/usr/share/miniconda/envs/oguk-calibrate-dev/lib/python3.9/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
7
============================= test session starts ==============================
8
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
9
platform linux -- Python 3.9.2, pytest-5.4.3, py-1.10.0, pluggy-0.13.1
10
rootdir: /home/runner/work/OG-UK-Calibration/OG-UK-Calibration
11
plugins: anyio-2.2.0, cov-2.11.1, forked-1.3.0, xdist-2.2.1, pycodestyle-2.2.0, pep8-1.0.6
12
collected 27 items / 4 deselected / 23 selected
13

14
og_uk_calibrate/tests/test_get_micro_data.py ..                          [  8%]
15
og_uk_calibrate/tests/test_txfunc.py ......F..............               [100%]
16

17
=================================== FAILURES ===================================
18
__________________________ test_txfunc_est_on_GH[GS] ___________________________
19

20
rate_type = 'etr', tax_func_type = 'GS', numparams = 3
21
expected_tuple = (array([1.29769044e-01, 4.36139091e+00, 4.44767848e-07]), 20323.465971499016, 3798)
22
tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_txfunc_est_on_GH_GS_0')
23

24
    @pytest.mark.parametrize(
25
        "rate_type,tax_func_type,numparams,expected_tuple",
26
        [
27
            ("etr", "linear", 1, expected_tuple_linear),
28
            ("etr", "GS", 3, expected_tuple_GS),
29
            ("mtrx", "linear", 1, expected_tuple_linear_mtrx),
30
            ("mtry", "linear", 1, expected_tuple_linear_mtry),
31
        ],
32
        ids=["linear", "GS", "linear, mtrx", "linear, mtry"],
33
    )
34
    def test_txfunc_est_on_GH(
35
        rate_type, tax_func_type, numparams, expected_tuple, tmpdir
36
    ):
37
        """
38
        Test txfunc.txfunc_est() function.  The test is that given
39
        inputs from previous run, the outputs are unchanged.
40
        """
41
        input_tuple = safe_read_pickle(
42
            os.path.join(CUR_PATH, "test_io_data", "txfunc_est_inputs.pkl")
43
        )
44
        (df, s, t, _, output_dir, graph) = input_tuple
45
        output_dir = tmpdir
46
        # Put old df variables into new df var names
47
        df.rename(
48
            columns={
49
                "MTR labor income": "mtr_labinc",
50
                "MTR capital income": "mtr_capinc",
51
                "Total labor income": "total_labinc",
52
                "Total capital income": "total_capinc",
53
                "ETR": "etr",
54
                "expanded_income": "market_income",
55
                "Weights": "weight",
56
            },
57
            inplace=True,
58
        )
59
        test_tuple = txfunc.txfunc_est(
60
            df, s, t, rate_type, tax_func_type, numparams, output_dir, True
61
        )
62
    
63
        for i, v in enumerate(expected_tuple):
64
>           assert np.allclose(test_tuple[i], v)
65
E           assert False
66
E            +  where False = <function allclose at 0x7f506d24adc0>(array([1.29769079e-01, 4.36130866e+00, 4.44902398e-07]), array([1.29769044e-01, 4.36139091e+00, 4.44767848e-07]))
67
E            +    where <function allclose at 0x7f506d24adc0> = np.allclose
68

69
og_uk_calibrate/tests/test_txfunc.py:297: AssertionError
70

One solution would be to set the tolerance for np.allclose to something other than the default. One would want to be careful with this, though, as some parameter estimates are pretty small, so using a relative tolerance could be useful.