updated for OpenRefine 2.8 and 3.0-beta

[daniel-butler]

Updates to python2 2.8 and 3.0-beta server updates

[psychemedia]

Trying this PR, I don't seem to be able to create a project with a given name? (All projects are created as Untitled?)

Example notebook here that can run via MyBinder on that repo.

I also note that docstrings etc are missing from the Library. Is the original repo still open to PRs?

With OpenRefine seemingly back on track as an active project, it'd be good to be able to work up the Python wrapper a bit more.

[felixlohmeier]

The test files have been changed much and that needs to be reviewed I think.

Anyway, test results for Python 3.7.3 for https://github.com/dbutlerdb/refine-client-py/tree/master (as of today, identical to OpenRefine3.0Beta)

• 2.0-java6: FAILED (failures=3, errors=3)
• 2.1-java6: FAILED (failures=3, errors=3)
• 2.5-java6: FAILED (failures=2, errors=1)
• 2.7: OK! just 1 failure in test_version because 2.7 is not listed yet.
• 2.8: OK!
• 3.0: OK! just 1 failure in test_version because 3.0 is not listed yet.
• 3.1: FAILED (failures=1, errors=18)
• 3.2: FAILED (failures=1, errors=18)

project creation fails with OpenRefine 3.1 and 3.2:

Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/refinetest.py", line 44, in setUp
    **self.project_options
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 127, in new_project
    project_id = self.get_project_id(response.url)
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 175, in get_project_id
    raise Exception('Project not created')
Exception: Project not created

errors/failures in 2.5

maybe because of changes to test files?

======================================================================
ERROR: test_web_scraping (tests.test_tutorial.TutorialTestWebScraping)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_tutorial.py", line 442, in test_web_scraping
    self.project.split_column('key', separator=':')
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 552, in split_column
    'removeOriginalColumn': remove_original_column
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 331, in do_json
    response = self.server.urlopen_json(command, params=params, data=data)
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 80, in urlopen_json
    self.check_response_ok(response)
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 88, in check_response_ok
    raise Exception(error_message)
Exception: server error: No column named key

======================================================================
FAIL: test_apply_operations (tests.test_commands.TestCommands)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_commands.py", line 34, in setUp
    self.assertEqual(response.facets[0].choices[True].count, 2)
AssertionError: 3 != 2

======================================================================
FAIL: test_get_operations (tests.test_commands.TestCommands)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_commands.py", line 34, in setUp
    self.assertEqual(response.facets[0].choices[True].count, 2)
AssertionError: 3 != 2

errors/failures in 2.0 and 2.1

======================================================================
ERROR: test_transpose_variable_number_of_rows_into_columns (tests.test_tutorial.TutorialTestTransposeVariableNumberOfRowsIntoColumns)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_tutorial.py", line 385, in test_transpose_variable_number_of_rows_into_columns
    'Column', 'First Line', 'if(value.contains(" on "), value, null)')
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 527, in add_column
    column_insert_index = self.column_order[column] + 1
KeyError: 'Column'

======================================================================
ERROR: test_open_export (tests.test_refine.RefineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_refine.py", line 62, in test_open_export
    content = refine.RefineProject(self.project.project_url()).export()
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 411, in export
    url = ('export-rows/' + urllib.parse.quote(self.project_name()) + '.' + export_format)
  File "/usr/lib64/python3.7/urllib/parse.py", line 827, in quote
    return quote_from_bytes(string, safe)
  File "/usr/lib64/python3.7/urllib/parse.py", line 852, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

======================================================================
ERROR: test_open_export_csv (tests.test_refine.RefineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_refine.py", line 70, in test_open_export_csv
    fp = refine.RefineProject(self.project.project_url()).export()
  File "/home/felix/Desktop/refine-client-py/open/refine/refine.py", line 411, in export
    url = ('export-rows/' + urllib.parse.quote(self.project_name()) + '.' + export_format)
  File "/usr/lib64/python3.7/urllib/parse.py", line 827, in quote
    return quote_from_bytes(string, safe)
  File "/usr/lib64/python3.7/urllib/parse.py", line 852, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

======================================================================
FAIL: test_transpose_fixed_number_of_rows_into_columns (tests.test_tutorial.TutorialTestTransposeFixedNumberOfRowsIntoColumns)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_tutorial.py", line 312, in test_transpose_fixed_number_of_rows_into_columns
    self.assertTrue('Column' in self.project.column_order)
AssertionError: False is not true

======================================================================
FAIL: test_apply_operations (tests.test_commands.TestCommands)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_commands.py", line 34, in setUp
    self.assertEqual(response.facets[0].choices[True].count, 2)
AssertionError: 3 != 2

======================================================================
FAIL: test_get_operations (tests.test_commands.TestCommands)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/felix/Desktop/refine-client-py/tests/test_commands.py", line 34, in setUp
    self.assertEqual(response.facets[0].choices[True].count, 2)
AssertionError: 3 != 2