Skip to content

Feature/corpus settings#2103

Merged
JeltevanBoheemen merged 11 commits into
developfrom
feature/corpus-settings
Jul 15, 2026
Merged

Feature/corpus settings#2103
JeltevanBoheemen merged 11 commits into
developfrom
feature/corpus-settings

Conversation

@lukavdplas

Copy link
Copy Markdown
Contributor

Finally got around to implementing this. Changes the corpus settings format according to #2099

The new format can exist side by side with the old one (you could even use both formats in the same corpus) but it should fully replace it.

For some corpora, I've changed the definition so they support the new format as well as the old one (but there is a log warning if you use the old setting). This adds a significant expansion of code, which is temporary. For example, this was the ParliamentCanada definition:

class ParliamentCanada(Parliament, CSVCorpusDefinition):
    title = 'People & Parliament (Canada)'
    description = "Speeches from House of Commons"
    min_date = datetime(year=1901, month=1, day=1)
    data_directory = settings.PP_CANADA_DATA
    es_index = getattr(settings, 'PP_CANADA_INDEX', 'parliament-canada')
    word_model_path = getattr(settings, 'PP_CANADA_WM', None)

Now changed to:

class ParliamentCanada(Parliament, CSVCorpusDefinition):
    title = 'People & Parliament (Canada)'
    description = "Speeches from House of Commons"
    min_date = datetime(year=1901, month=1, day=1)

    @property
    def data_directory(self):
        return get_deprecated_setting('PP_CANADA_DATA') or super().data_directory

    @property
    def es_index(self):
        return get_deprecated_setting('PP_CANADA_INDEX') or 'parliament-canada'

    @property
    def word_model_path(self):
        return get_deprecated_setting('PP_CANADA_WM') or super().word_model_path

After deployment, this can be changed to:

class ParliamentCanada(Parliament, CSVCorpusDefinition):
    title = 'People & Parliament (Canada)'
    description = "Speeches from House of Commons"
    min_date = datetime(year=1901, month=1, day=1)
    es_index = 'parliament-canada'

Changes to Django settings

This adds the CORPUS_SETTINGS Django setting. No immediate change is required, but the log will show a warning if a corpus can be switched to CORPUS_SETTINGS and this has not been done in the environment.

@lukavdplas lukavdplas added code quality code & performance improvements that do not affect user functionality backend changes to the django backend affects-deployment changes that require an update in the deployment module labels Jun 24, 2026
@JeltevanBoheemen
JeltevanBoheemen merged commit bcfc43e into develop Jul 15, 2026
3 checks passed
@JeltevanBoheemen
JeltevanBoheemen deleted the feature/corpus-settings branch July 15, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

affects-deployment changes that require an update in the deployment module backend changes to the django backend code quality code & performance improvements that do not affect user functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants