Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 40 additions & 26 deletions docs/data_services/create_dataservice.rst
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,22 @@ into a model object with ``create_foo_from_query()``.
Depending on the specifics of your data service, it may be reasonable to call the ``query_foo()`` methods independently,
and/or part of ``query_targets``.

Including Aliases:
++++++++++++++++++
Often when ingesting a new target from a dataservice, there will be multiple names or references for the target that you
will want to include to avoid confusion and help with disambiguation within a TOM. This can be done in one of two ways:

1. ``target_result``: Including an ``aliases`` key in any target results output by ``query_targets`` with a value that is
a list of strings will automatically ingest those aliases as ``TargetNames``.

2. ``query_aliases``: Sometimes, getting aliases can be more complex, possibly even requiring its own separate DB query. For
These instances, you will need to include a ``query_aliases`` method that returns a list of strings, each string being a
name for the target that is different from the primary name stored in ``Target.name``. See the
:doc:`dataservices documentation <../api/tom_dataservices/data_services>` for more information.

Note: By default a TOM's Match Filters apply to alias creation, so if an alias matches any existing name or alias already in the DB
it will not be created.

Querying Reduced Datums:
++++++++++++++++++++++++
Data from a dataservice that needs to be stored as a ``ReducedDatum`` should be handled a little differently.
Expand Down Expand Up @@ -293,9 +309,10 @@ We will start by creating our query:
``DataService.create_reduced_datums_from_query``
================================================

To create the ``ReducedDatum``s we will need a ``create_reduced_datums_from_query()`` method. This should take all of the data
types and convert them into ``ReducedDatum`` objects. Be sure to use ``ReducedDatum.objects.get_or_create()`` to prevent
re-creating existing objects.
To create the ``ReducedDatum`` we will need a ``create_reduced_datums_from_query()`` method. This should take all of the data
types and convert them into ``ReducedDatum`` objects. Be sure to use ``XXXXXReducedDatum.objects.get_or_create()`` to prevent
re-creating existing objects where "XXXXX" represents the specific model you wish to use (such as Photometry, Spectroscopy, or Astrometry.)
Details for the different types of data models can be found in the :doc:`API documentation <../api/tom_dataproducts/models>`.

.. code-block:: python
:caption: my_dataservice.MyDataService
Expand All @@ -318,19 +335,16 @@ re-creating existing objects.
# We might have some specific things we want to include based on type.
# For Photometry, for example, we need a magnitude, error, and filter to be displayed in the
# photometry plot on the target detail page.
datum_details['magnitude'] = datum['my_mag']
datum_details['error'] = datum['my_magerr']
datum_details['limit'] = datum['my_maglim']
datum_details['filter'] = datum['my_passband']

reduced_datum, __ = ReducedDatum.objects.get_or_create(
target=target,
timestamp=Time(datum['time'], format='iso', scale='utc').datetime,
data_type=data_type,
source_name=self.name,
value=datum_details
)
reduced_datums.append(reduced_datum)
reduced_datum, __ = PhotometryReducedDatum.objects.get_or_create(
target=target,
timestamp=Time(datum['time'], format='iso', scale='utc').datetime,
source_name=self.name,
brightness=datum['my_mag'],
brightness_error=datum['my_magerr'],
limit=datum['my_maglim'],
bandpass=datum['my_passband']
)
reduced_datums.append(reduced_datum)
return reduced_datums


Expand All @@ -357,16 +371,16 @@ exists, in several places where we want to update an existing target based on da
:param target: A target object to be queried
:return: query_parameters (usually a dict) that can be understood by `query_service()`
"""
if 'first' in target.name:
form_fields = {'first_field': target.name}
query_parameters = self.build_query_parameters(form_fields)
else:
query_parameters= {
'ra_field': target.ra,
'dec_field': target.dec,
'radius': 0.5
}
return query_parameters
if 'first' in target.name:
form_fields = {'first_field': target.name}
query_parameters = self.build_query_parameters(form_fields)
else:
query_parameters= {
'ra_field': target.ra,
'dec_field': target.dec,
'radius': 0.5
}
return query_parameters


Polishing Your Data Service:
Expand Down
5 changes: 4 additions & 1 deletion tom_dataservices/dataservices.py
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,10 @@ def to_target(self, target_result=None, **kwargs):
else:
logger.warning(f"The target, {target.name}, already exists. Any new data will be ingested.")
# Save Aliases
self.to_aliases(target, target_result.get('aliases', []))
alias_list = target_result.get('aliases', []) or self.query_aliases(self.query_parameters,
target,
**kwargs)
self.to_aliases(target, alias_list)
return target

def create_target_from_query(self, target_result, **kwargs):
Expand Down
16 changes: 14 additions & 2 deletions tom_dataservices/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -321,12 +321,24 @@ def get_context_data(self, *args, **kwargs):
:rtype: dict
"""
context = super().get_context_data()
data_service_name = self.object.data_service

simple_form = context['form'].get_simple_form_partial()
advanced_form = context['form'].get_advanced_form_partial()
form = context['form']
simple_form = form.get_simple_form_partial()
advanced_form = form.get_advanced_form_partial()

context['simple_fields'] = []
if not simple_form and form.simple_fields():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would the form not have as simple partial, but also have simple fields?

I trust that this works, but it just reads a little strange.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I describe this in the docs but I should definitely add a comment here.
Basically, there is a basic partial that takes the fields in simple_fields and makes a super basic form. OR you can can make a simple partial that does whatever you want. So if no partial is given, but there are simple fields we will just use those in the built in default partial.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One would not necessarily expect there to be BOTH a simple_fields and a user made simple_partial, since the user made simple partial could be much more specific to the dataservice's form fields.

for field in form.simple_fields():
context['simple_fields'].append(form[field])
simple_form = 'tom_dataservices/partials/basic_simple_form.html'
context['simple_form'] = simple_form
context['advanced_form'] = advanced_form
context['object'] = self.object
context['app_link'] = get_data_service_class(data_service_name).app_link
context['app_version'] = get_data_service_class(data_service_name).app_version
context['verbose_name'] = get_data_service_class(data_service_name).verbose_name
context['info_url'] = get_data_service_class(data_service_name).info_url
return context


Expand Down
Loading