Corpus settings format

[lukavdplas]

Currently, the way to make a Python corpus configurable is to import Django settings in the Python class definition, e.g.

from django.conf import settings
from addcorpus.pyton_corpora.corpus import CSVCorpusDefinition

class MyCorpus(CSVCorpusDefinition):
	title = getattr(settings, 'MY_CORPUS_TITLE', 'My Corpus')
	description = 'This is an example'
	data_directory = settings.MY_CORPUS_DATA

	# ....

For this corpus, title can be configured but has a default value, description is not configurable, and data_directory is a required configuration.

This approach has some issues:

• The keys for these settings follow a loose naming convention, but this is not standardised. Related to that, it is not easily recognisable which constants in the backend settings are corpus settings.
• Attributes are only configurable if the corpus class defines them as such. Configurable attributes (like title here) are more verbose, so to keep things simple, we don't make everything configurable.
• Required settings cause issues in module imports. In this example, the setting MY_CORPUS_DATA is presumably necessary to use the corpus, but with this setup, it's also required to import the module, even if the corpus is not instantiated. This can be a problem with related corpora. There is no real problem if MyOtherCorpus imports some values from this module without defining a data directory for MyCorpus, but doing so would raise an AttributeError.
• Every new corpus definition requires the logic to import settings; this would be simpler without.
• The keys chosen for corpora might overlap with other Django settings, or other corpora. I don't think this is an issue right now, but it would be if you wanted to make the application more modular or re-usable. Another implication is that you cannot have two corpora based on the same definition but with different settings, which can be useful.

My suggestion:

• The configuration of the corpus should happen in __init_() or be called post-init by the loadcorpus module (not sure which). So Django settings are only injected when the corpus is instantiated, and can override class attributes.
• The logic to get attribute values from the settings should be in the CorpusDefinition base class, or in the loadcorpus module. A corpus definition module should, generally speaking, not import Django settings directly.

Implementation details:

One option is to keep the settings format more-or-less the same, but standardised. For instance, the title of any corpus could be overridden with the setting {corpus.name.upper()}_TITLE.

I would prefer to transition to a different format, like this:

CORPORA = {
	'times': 'corpora.times.times.Times',
	'parliament-nl': 'corpora.parliament.netherlands.ParliamentNetherlands',
}

CORPUS_SETTINGS = {
	'times': {
		'data_directory': '/path/to/times/data/'
	},
	'parliament-nl': {
		'title': 'Dutch Parliamentary Debates',
		'data_directory': '/path/to/parliament/nl/data/',
		'word_models_path': '/path/to/word/models/',
	},
}

Here, CORPUS_SETTINGS provides a straightforward attribute-value map for each corpus, with the attributes you want to override. Compared to the current format, I think this is easier to follow and creates less clutter in the project settings.