Skip to content

Tickets/sp 2882 #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
christinawilliams merged 5 commits into from
Mar 19, 2026
Merged

Tickets/sp 2882 #127

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c362cee
made minor changes
christinawilliams Mar 18, 2026
496ac33
made minor changes
christinawilliams Mar 19, 2026
d7dc218
made minor changes
christinawilliams Mar 19, 2026
a1578f2
made minor changes
christinawilliams Mar 19, 2026
384cd2f
last updates
christinawilliams Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
220 changes: 111 additions & 109 deletions DP1/200_Data_Products/206_Deblender_Products/206_2_Deblender_footprints.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"Data Release: DP1 <br>\n",
Comment thread
christinawilliams marked this conversation as resolved.
"Container Size: Large <br>\n",
"LSST Science Pipelines version: Weekly 29.2.0 <br>\n",
"Last verified to run: 2025-10-06 <br>\n",
"Last verified to run: 2026-03-19 <br>\n",
"Repository: <a href=\"https://github.com/lsst/tutorial-notebooks\">github.com/lsst/tutorial-notebooks</a> <br>"
]
},
Expand Down Expand Up @@ -72,7 +72,7 @@
"\n",
"\n",
"**Related tutorials:**\n",
"A notebook introducing the deblender data products stored in the `Object` table is available as 206.1 Introduction to deblender outputs. \n"
"A notebook introducing the deblender data products stored in the `Object` table is available as 206.1 Introduction to deblender outputs. A notebook introducing image subsets and defining bounding boxes is available as 104.5 titled Image subsets with the butler.\n"
]
},
{
Expand All @@ -82,7 +82,7 @@
"source": [
"### 1.1. Import packages\n",
"\n",
"`lsst.afw.image` provides access to some of the extended products created by the deblender pipeline. The `lsst.afw.display` library provides access to image visualization routines and the `lsst.daf.butler` library is used to access data products via the butler. Finally, `lsst.scarlet.lite` and `lsst.meas.extensions.scarlet` are two packages containing the scarlet software infrastructure, to enable displaying and analyzing deblended models and object footprints. "
"`lsst.afw.image` provides access to some of the extended products created by the deblender pipeline. The `lsst.afw.display` library provides access to image visualization routines and the `lsst.daf.butler` library is used to access data products via the butler. Finally, `lsst.scarlet` and `lsst.meas.extensions.scarlet` are two packages containing the scarlet software infrastructure, to enable displaying and analyzing deblended models and object footprints. "
]
},
{
Expand All @@ -96,15 +96,16 @@
"import matplotlib.pyplot as plt\n",
"\n",
"from lsst.daf.butler import Butler\n",
"\n",
"from lsst.rsp import get_tap_service\n",
"import lsst.geom as geom\n",
"\n",
"import lsst.afw.display as afwDisplay\n",
"from lsst.afw.image import MultibandExposure\n",
"import lsst.meas.extensions.scarlet as mes\n",
"\n",
"import lsst.scarlet as scarlet\n",
"import lsst.scarlet.lite as sl\n",
"import lsst.meas.extensions.scarlet as mes\n"
"from lsst.scarlet.lite import Box, Blend, Observation"
]
},
{
Expand All @@ -115,6 +116,48 @@
"### 1.2. Define parameters and functions\n"
]
},
{
"cell_type": "markdown",
"id": "f0c1e8da-8196-4c1d-a929-0feb83a7287e",
"metadata": {},
"source": [
"An issue exists with an LSST pipeline class called `minimal_data_to_blend` to access the `scarlet` blend model specifically when using DP1 data. This will be fixed in the future. In the meantime, the cell below defines a function with a temporary fix for DP1 that updates the `minimal_data_to_blend` LSST pipeline class (temporarily while running this notebook)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbda1096-cf1c-433f-ae5c-1ad761683f09",
"metadata": {},
"outputs": [],
"source": [
"def minimal_data_to_blend(cls, model_psf: np.ndarray, dtype: np.typing.DTypeLike) -> Blend:\n",
"\n",
" \"\"\"Convert the storage data model into a scarlet lite blend\n",
"\n",
" Parameters\n",
" ----------\n",
" model_psf:\n",
" PSF in model space (usually a nyquist sampled circular Gaussian).\n",
" dtype:\n",
" The data type of the model that is generated.\n",
"\n",
" Returns\n",
" -------\n",
" blend:\n",
" A scarlet blend model extracted from persisted data.\n",
" \"\"\"\n",
" model_box = Box(cls.shape, origin=cls.origin)\n",
" observation = Observation.empty(\n",
" bands=cls.bands,\n",
" psfs=cls.psf,\n",
" model_psf=model_psf,\n",
" bbox=model_box,\n",
" dtype=dtype,\n",
" )\n",
" return cls.to_blend(observation)"
]
},
{
"cell_type": "markdown",
"id": "a527bd20-372c-478a-98c4-9c0b0b9f876e",
Expand Down Expand Up @@ -176,7 +219,7 @@
"id": "16eeb034-6728-406e-bfc8-7b875c3a9416",
"metadata": {},
"source": [
"## 2. Identify a blend\n",
"## 2. Identify a blend\n",
"\n",
"This first section will identify a blend in DP1, query the `Object` table for the deblended child objects, and visualize the blend.\n"
]
Expand All @@ -201,7 +244,7 @@
"parentId = 611256447031839519\n",
"\n",
"ra = 53.2\n",
"dec = -27.5\n"
"dec = -27.5"
]
},
{
Expand All @@ -223,7 +266,7 @@
" \"FROM dp1.Object \" + \\\n",
" \"WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec ), \" + \\\n",
" \"CIRCLE('ICRS', \" + str(ra) + \", \" + str(dec) + \", .1)) = 1 \" + \\\n",
" \"AND parentObjectId = \" + str(parentId)\n"
" \"AND parentObjectId = \" + str(parentId)"
]
},
{
Expand Down Expand Up @@ -280,94 +323,85 @@
"id": "4288252b-6639-42de-82aa-70db8252f284",
"metadata": {},
"source": [
"To visualize the blend, first define a function to generate an image cutout. "
"To visualize the blend, plot a `deep_coadd` image subset that contains the blend with children identified. First, find the patch and tract containing the blend, and set the band to be i-band."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a1bf0635-8a88-42ae-b48d-79e69213f710",
"id": "245bec12-edcf-4564-811e-9de0d06fd3e5",
"metadata": {},
"outputs": [],
"source": [
"def cutout_coadd(butler, ra, dec, band='i', datasetType='deepCoadd',\n",
" skymap=None, cutoutSideLength=51, **kwargs):\n",
" \"\"\"\n",
" Produce a cutout from a coadd at the given ra, dec position.\n",
"\n",
" Adapted from DC2 tutorial notebook by Michael Wood-Vasey.\n",
"\n",
" Parameters\n",
" ----------\n",
" butler: lsst.daf.persistence.Butler\n",
" Helper object providing access to a data repository\n",
" ra: float\n",
" Right ascension of the center of the cutout, in degrees\n",
" dec: float\n",
" Declination of the center of the cutout, in degrees\n",
" band: string\n",
" Filter of the image to load\n",
" datasetType: string ['deepCoadd']\n",
" Which type of coadd to load. Doesn't support 'calexp'\n",
" skymap: lsst.afw.skyMap.SkyMap [optional]\n",
" Pass in to avoid the Butler read. Useful if you have lots of them.\n",
" cutoutSideLength: float [optional]\n",
" Size of the cutout region in pixels.\n",
"\n",
" Returns\n",
" -------\n",
" MaskedImage\n",
" \"\"\"\n",
" radec = geom.SpherePoint(ra, dec, geom.degrees)\n",
" cutoutSize = geom.ExtentI(cutoutSideLength, cutoutSideLength)\n",
"\n",
" if skymap is None:\n",
" skymap = butler.get(\"skyMap\")\n",
"radec = geom.SpherePoint(children['coord_ra'][0], children['coord_dec'][0], geom.degrees)\n",
"skymap = butler.get(\"skyMap\")\n",
"\n",
" tractInfo = skymap.findTract(radec)\n",
" patchInfo = tractInfo.findPatch(radec)\n",
" xy = geom.PointI(tractInfo.getWcs().skyToPixel(radec))\n",
" bbox = geom.BoxI(xy - cutoutSize // 2, cutoutSize)\n",
" print(\"bbox = \", bbox, \"xy = \", xy, \"cutoutSize = \", cutoutSize)\n",
" patch = tractInfo.getSequentialPatchIndex(patchInfo)\n",
" tract = tractInfo.getId()\n",
"tractInfo = skymap.findTract(radec)\n",
"\n",
" parameters = {'bbox': bbox}\n",
"\n",
" query = \"\"\"band.name = '{}' AND patch = {} AND tract = {}\n",
" \"\"\".format(band, patch, tract)\n",
" print(query)\n",
"\n",
" dataset_refs = butler.query_datasets(\"deep_coadd\", where=query)\n",
"\n",
" cutout_image = butler.get(dataset_refs[0], parameters=parameters)\n",
"\n",
" return cutout_image"
"tract = tractInfo.getId()\n",
"patch = tractInfo.findPatch(radec).getSequentialIndex()\n",
"band = 'i'"
]
},
{
"cell_type": "markdown",
"id": "5546408c-6009-4230-9ef4-2faf1c302940",
"id": "80f1ea90-55d5-43b4-b6da-0256baf8702c",
"metadata": {},
"source": [
"Make a cutout of the `deep_coadd` imaging, using the `cutout_coadd` function to visualize blend and its child objects. Set the size of the cutout to be 5x the square root of the total `footprintArea` of the blend.\n",
" "
"Next, define the dimensions of the image subset to be 5x the square root of the total `footprintArea` of the blend."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b2f6c287-7502-4343-89e3-e76cb3e98e87",
"id": "f845cc95-c731-40d2-a202-eea9e2681afe",
"metadata": {},
"outputs": [],
"source": [
"total_footprint_area = np.sum(children['footprintArea'])\n",
"\n",
"cutout = cutout_coadd(butler, children['coord_ra'][0],\n",
" children['coord_dec'][0],\n",
" band='i', datasetType='deepCoadd',\n",
" cutoutSideLength=5.0\n",
" * np.sqrt(total_footprint_area))"
"cutoutSideLength = 5.0 * np.sqrt(total_footprint_area)"
]
},
{
"cell_type": "markdown",
"id": "a1922530-e638-4514-bce0-7087f6696c32",
"metadata": {},
"source": [
"Define the bounding box for the image subset. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ab876741-2e6a-491e-bdec-de5ca72646bf",
"metadata": {},
"outputs": [],
"source": [
"cutoutSize = geom.ExtentI(cutoutSideLength, cutoutSideLength)\n",
"xy = geom.PointI(tractInfo.getWcs().skyToPixel(radec))\n",
"bbox = geom.BoxI(xy - cutoutSize // 2, cutoutSize)\n",
"parameters = {'bbox': bbox}"
]
},
{
"cell_type": "markdown",
"id": "0ffb1405-2389-41a4-b91c-3a65729f21fb",
"metadata": {},
"source": [
"Finally, send the bounding box to the butler and request the image subset from the `deep_coadd` to visualize blend and its child objects. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8fdd4188-33ae-4b30-8513-52d82e0dce9d",
"metadata": {},
"outputs": [],
"source": [
"query = \"\"\"band.name = '{}' AND patch = {} AND tract = {}\n",
" \"\"\".format(band, patch, tract)\n",
"dataset_refs = butler.query_datasets(\"deep_coadd\", where=query)\n",
"cutout = butler.get(dataset_refs[0], parameters=parameters)"
]
},
{
Expand Down Expand Up @@ -428,7 +462,7 @@
"registry = butler.registry\n",
"\n",
"for dt in sorted(registry.queryDatasetTypes('*scarlet*')):\n",
" print(dt)\n"
" print(dt)"
]
},
{
Expand All @@ -448,7 +482,7 @@
"source": [
"query = \"\"\"band.name = '{}' AND patch.region OVERLAPS POINT({}, {})\n",
" \"\"\".format('i', children['coord_ra'][0], children['coord_dec'][0])\n",
"print(query)\n"
"print(query)"
]
},
{
Expand All @@ -467,7 +501,7 @@
"outputs": [],
"source": [
"dataset_refs = butler.query_datasets(\"deep_coadd\", where=query)\n",
"dataId = dataset_refs[0].dataId\n"
"dataId = dataset_refs[0].dataId"
]
},
{
Expand Down Expand Up @@ -520,7 +554,7 @@
"source": [
"### 3.2 Convert to a usable blend instance\n",
"\n",
"First, define a monkey patch that temporarily updates the `minimal_data_to_blend` LSST pipeline class in order to fix an issue with DP1 (temporarily while running this notebook). Once the fix enters the pipeline version in use, the following cell defining the monkey patch can simply be deleted without additional alteration to the notebook. The following cells will use this monkey patch version to load the blend.\n"
"Section 1.2 defined a function with a temporary fix that updates the `minimal_data_to_blend` LSST pipeline class in order to fix an issue with DP1 (temporarily while running this notebook). Once the fix enters the pipeline version in use, the following cell defining the temporary fix (and the function from Section 1.2) can simply be deleted without additional alteration to the notebook. The following cells will use this temporary fixed version to load the blend."
]
},
{
Expand All @@ -530,38 +564,6 @@
"metadata": {},
"outputs": [],
"source": [
"from numpy.typing import DTypeLike\n",
"from lsst.scarlet.lite import Box, Blend, Observation\n",
"\n",
"\n",
"def minimal_data_to_blend(cls, model_psf: np.ndarray, dtype: DTypeLike) -> Blend:\n",
"\n",
" \"\"\"Convert the storage data model into a scarlet lite blend\n",
"\n",
" Parameters\n",
" ----------\n",
" model_psf:\n",
" PSF in model space (usually a nyquist sampled circular Gaussian).\n",
" dtype:\n",
" The data type of the model that is generated.\n",
"\n",
" Returns\n",
" -------\n",
" blend:\n",
" A scarlet blend model extracted from persisted data.\n",
" \"\"\"\n",
" model_box = Box(cls.shape, origin=cls.origin)\n",
" observation = Observation.empty(\n",
" bands=cls.bands,\n",
" psfs=cls.psf,\n",
" model_psf=model_psf,\n",
" bbox=model_box,\n",
" dtype=dtype,\n",
" )\n",
" return cls.to_blend(observation)\n",
"\n",
"\n",
"import lsst.scarlet as scarlet\n",
"scarlet.lite.io.ScarletBlendData.minimal_data_to_blend = minimal_data_to_blend"
]
},
Expand All @@ -570,7 +572,7 @@
"id": "0a8baf45-b282-467c-ae7e-62cfb2241c4b",
"metadata": {},
"source": [
"Now recreate the blend model using the monkey patch."
"Now recreate the blend model using the temporary fix function."
]
},
{
Expand Down Expand Up @@ -842,7 +844,7 @@
"outputs": [],
"source": [
"object1 = blend.sources[1]\n",
"print(object1.record_id)\n"
"print(object1.record_id)"
]
},
{
Expand Down
Loading