task/WI-454 - DPM Merge into Main by shayanaijaz · Pull Request #1332 · TACC/Core-Portal

shayanaijaz · 2026-07-07T19:14:57Z

Overview

DPM features and changes being brought to parity with core.

Changes

Lots of changes and additions included. The goal for the changes was to keep them as portal agnostic as possible. There are minimal changes to how core portals operate. I have created a document which highlights the changes and additions that were made:

DPM Branch: Configuration and Customization Reference for Core Portal

Testing

Test core portal functionality, there shouldn't be any changes to the UI and operations and everything should work same as before

UI

Notes

- Added citation box and citation modal for publications - Removed Last Modified column on publication listing - Changed Created to Published Date - Added data view modal to view related project information

…rocks

…cted (#1008)

* WC-332 Add project admin group functionality and related client access checks - Introduced `check_project_admin_group` and `get_project_client` functions to manage project admin access. - Updated various views to utilize the new client access logic for project admins. - Enhanced project metadata handling to include project admin group checks. - Added unit tests for project admin group functionality and client retrieval. * updates to project admin access --------- Co-authored-by: Shayan Khan <skhan@tacc.utexas.edu>

This reverts commit 31349a9.

codecov · 2026-07-07T19:17:50Z

Codecov Report

❌ Patch coverage is 22.54075% with 2756 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.37%. Comparing base (4ca8af2) to head (a7bdc51).

Files with missing lines	Patch %	Lines
...anagement/commands/migrate_drp_project_metadata.py	0.00%	273 Missing ⚠️
...ts/workspace_operations/project_meta_operations.py	18.03%	200 Missing ⚠️
...workspace_operations/project_publish_operations.py	19.66%	192 Missing ⚠️
server/portal/apps/publications/views.py	19.65%	139 Missing ⚠️
server/portal/libs/files/file_processing.py	12.41%	127 Missing ⚠️
...s/management/commands/migrate_drp_project_files.py	0.00%	117 Missing ⚠️
...FilesProjectPublishWizardSteps/ProjectTreeView.jsx	1.69%	97 Missing and 19 partials ⚠️
.../projects/workspace_operations/graph_operations.py	16.54%	111 Missing ⚠️
...omponents/_common/Form/DynamicForm/DynamicForm.jsx	0.90%	97 Missing and 12 partials ⚠️
...PublishedDatasets/PublishedDatasetEntityDetail.jsx	2.67%	92 Missing and 17 partials ⚠️
... and 76 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1332      +/-   ##
==========================================
- Coverage   64.10%   55.37%   -8.74%     
==========================================
  Files         463      512      +49     
  Lines       13828    17184    +3356     
  Branches     2759     3444     +685     
==========================================
+ Hits         8865     9516     +651     
- Misses       4609     7154    +2545     
- Partials      354      514     +160

Flag	Coverage Δ
javascript	`59.40% <22.02%> (-7.72%)`	⬇️
unittests	`51.52% <22.87%> (-9.41%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...aFiles/CombinedBreadcrumbs/CombinedBreadcrumbs.jsx	`100.00% <ø> (ø)`
...iles/DataFilesBreadcrumbs/DataFilesBreadcrumbs.jsx	`68.57% <100.00%> (+0.45%)`	⬆️
...s/DataFiles/DataFilesModals/DataFilesCopyModal.jsx	`78.84% <100.00%> (+0.84%)`	⬆️
...ents/DataFiles/DataFilesModals/DataFilesModals.jsx	`0.00% <ø> (ø)`
...taFiles/DataFilesModals/DataFilesShowPathModal.jsx	`91.89% <100.00%> (+0.98%)`	⬆️
...ts/DataFiles/DataFilesSidebar/DataFilesSidebar.jsx	`88.00% <100.00%> (+1.63%)`	⬆️
...ts/DataFiles/fixtures/DataFiles.systems.fixture.js	`100.00% <ø> (ø)`
client/src/components/Submissions/Submissions.jsx	`53.22% <ø> (ø)`
client/src/components/Workbench/Workbench.jsx	`90.47% <ø> (-0.44%)`	⬇️
client/src/components/_common/Expand/Expand.jsx	`87.50% <100.00%> (ø)`
... and 109 more

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This reverts commit 06b4405, reversing changes made to 5a67abe.

wesleyboar

This is impressive. I am proud of your work to make this Core-mergable. ~~The _custom/ dir gives Portal opportunity to allow custom code via Dockerfile.¹²~~

I worry only about long-term maintenance of CMSWrapper.jsx and cms_ui.html, but Core-Portal v4 likely introduces a change that can retire them.³

^{Update: I have stricken out my _custom dir thoughts, because another peer review's comment suggests it might be moved to become a standard component.}

Like how TUP-UI, APCD-CMS, and Texascale-CMS are Core-CMS with custom functionality. ↩ ↩²
~~A Dockerfile is used add/change CMS files (example). @rstijerina, what do you think about a TACC/Core-Portal-Template (à la TACC/Core-CMS-Template)?~~ ↩
Core-Portal v4 UI might be served via the Core-CMS¹, so CMS assets and markup would already be present, so no need to load/toggle them ad hoc. ↩

wesleyboar · 2026-07-08T15:54:59Z

Explanation: This <CMSWrapper> and cms_ui.html toggle CMS CSS and swap footer HTML¹ on the fly. They communicate through custom JavaScript event cms-styles-activated/cms-styles-deactivated. It exists because DPM has pages that mimic CMS UI.

^{Alternative was serving DPM (or the entire Portal) as app within CMS, which would be too big a scope change just to mimic CMS UI.}

Footnotes

The footer HTML swap is exactly like main's header HTML swap. ↩

rstijerina · 2026-07-08T14:35:11Z

    shouldTruncateValues ? 'value-truncated' : ''
  }`;

+  const compareFn = (entry1, entry2) => {


Can we make this more descriptive - what it does and what it's for?

rstijerina · 2026-07-08T14:37:14Z

can we add a docstring on how this is used?

rstijerina · 2026-07-08T14:37:40Z


 .root.as-link {
-  font-size: inherit;
+  font-size: 0.75rem;


does this have downstream effects?

Yes, at least least one — it increases size of text on Move/Copy modal. This change (#986) did fix specific links that were too small (WP-50), but that fix can likely be done without downstream effects.

Why use inherit?

The inherit is meant to make any "button that should look like a link" have style, and font-size, of a link (because a link — an <a> — inherits font-size).

rstijerina · 2026-07-08T14:38:53Z

      {label && hasAddon ? <FieldLabel /> : null}
      <FormFieldWrapper type={wrapperType}>
        {label && !hasAddon ? <FieldLabel /> : null}
+        {!hasAddon ? <FieldNote /> : null}


moved for a particular reason?

rstijerina · 2026-07-08T14:45:04Z

+          <Button attr="submit" type="primary" onClick={onContinue}>
+            Continue
+          </Button>
+          {/* <Button attr="submit" type="secondary" onClick={onSkip}>


Should the commented code in this file be removed? Or uncommented but disabled with flag?

rstijerina · 2026-07-08T17:05:01Z

+                adv_image = conf_tiff(file)
+            else:
+                adv_image = conf_raw(value, file)
+        except Exception as e:


Should we add more explicit exceptions?

rstijerina · 2026-07-08T17:11:03Z

+        except: 
+            pass


This is a large try/except, what are we trying to check against?

rstijerina · 2026-07-08T17:17:08Z

  useTapisToken();
-  // Get the time remaining in the user's session and redirect to the homepage if their
-  // browser is open when it expires.
-  useRedirectOnSessionExpired({ location: '/' });


merge issue?

rstijerina · 2026-07-08T17:18:39Z

    "@emotion/react": "^11.13.3",
    "@emotion/styled": "^11.13.0",
    "@mui/material": "^6.1.1",
+    "@mui/x-tree-view": "^7.26.0",


note as good candidate to replace with new core-components

rstijerina · 2026-07-08T17:19:20Z

+numpy = "<2.0"
+matplotlib = "^3.10.0"
+tifffile = "^2025.1.10"
+mysqlclient = "^2.2.7"


do we need mysqlclient?

fnets · 2026-07-08T18:36:11Z

I think we want to keep all these files, since @jarosenb just added them in a PR a week ago

fnets · 2026-07-17T20:01:03Z

It's likely that these projectfile directories are going to need a flag to turn them on and off, since I don't see a way to reconcile them with existing Shared Systems in Core.

fnets · 2026-07-17T20:03:38Z

Are we going to attempt to bring publications into other Core Portals as well? If so, this should also have a separate flag to turn on and off.

fnets · 2026-07-17T20:07:40Z

Is this still going to be necessary after a merge?

fnets · 2026-07-17T20:09:18Z

I think we'll want to decide which of these styles we want to go with, since it will change the portals across the board.

fnets · 2026-07-17T20:11:36Z

  const sharedWorkspaces = systems.find((e) => e.scheme === 'projects');
  const isPortalProject = scheme === 'projects';
-  const hideSearchBar = isPortalProject && sharedWorkspaces.hideSearchBar;
+  const hideSearchBar = isPortalProject && sharedWorkspaces?.hideSearchBar;


Do we have portals without shared workspaces in core?

fnets · 2026-07-17T20:14:43Z

For a public view, do we want to allow tapis operations?

fnets

Just went through this again to try and understand it a little more and flagged some questions/changes.

fnets · 2026-07-17T20:59:52Z

+import { Link } from 'react-router-dom';
+
+export default function CMSBreadcrumbs({ breadcrumbs = [] }) {
+  /* TODO: Move to …module.css OR into Core-Styles */


This seems like a good opportunity to knock this out.

Aye. Should move style to Core-Styles x-breadcrumbs.css¹, specifically.

Detailed Steps

Move style to Core-Styles x-breadcrumbs.css.¹

Release new Core-Styles.

Delete from Core-CMS.

Test on CMS.

Release new Core-CMS.

Delete from here.

Test again here.

Wow… this smells like an architecture that is too coupled.
— you

Yes, I know. There is a plan to load Core-Styles better.
— me

Footnotes

x- signifies a mixin. The x-breadcrumbs mixin is used by .s-breadcrumbs and tacc-docs.css and bootstrap3.breadcrumbs.css (for DesignSafe). ↩ ↩²

fnets · 2026-07-17T21:06:33Z

-      entry.comment.toLowerCase().includes('password') &&
-      entry.comment.toLowerCase().includes('change')
+  const passwordChanged = h.filter((entry) =>
+    entry.comment.includes('Password changed')


I'm not familiar with these functions, but it seems like they will find different sets of entries based on the parameters.

fnets · 2026-07-17T21:08:47Z

-export async function fetchProjectsListing(queryString) {
+export async function fetchProjectsListing(queryString, rootSystem) {
  const q = queryStringParser.stringify({ query_string: queryString });
+  const url = rootSystem ? `api/projects/${rootSystem}` : `/api/projects/`;


Are the portals with shared systems that don't have a root system?

fnets · 2026-07-17T21:12:46Z

+from portal.apps.projects.workspace_operations.project_meta_operations import (add_file_associations, create_file_obj, get_entity, get_file_obj, get_ordered_value, get_value, patch_entity_and_node,
+                                                                               patch_file_association, remove_file_obj_by_path)
+from portal.apps.projects.schema_models import constants
+from portal.apps.projects.workspace_operations.graph_operations import get_or_create_trash_entity, get_root_node, get_node_from_path


Suggested change

from portal.apps.projects.workspace_operations.project_meta_operations import (add_file_associations, create_file_obj, get_entity, get_file_obj, get_ordered_value, get_value, patch_entity_and_node,

patch_file_association, remove_file_obj_by_path)

from portal.apps.projects.schema_models import constants

from portal.apps.projects.workspace_operations.graph_operations import get_or_create_trash_entity, get_root_node, get_node_from_path

from portal.apps.projects.workspace_operations.project_meta_operations import (add_file_associations, create_file_obj, get_entity, get_file_obj, get_ordered_value, get_value, patch_entity_and_node, patch_file_association, remove_file_obj_by_path)

from portal.apps.projects.schema_models import constants

from portal.apps.projects.workspace_operations.graph_operations import get_or_create_trash_entity, get_root_node, get_node_from_path

shayanaijaz and others added 30 commits November 12, 2024 12:09

Publication UI improvements

f5becf8

- Added citation box and citation modal for publications - Removed Last Modified column on publication listing - Changed Created to Published Date - Added data view modal to view related project information

DRP Publication/Review changes

6136154

Added publication search functionality

d1b16a5

Added metadata sync for externally added files

8eb1c0a

Added validation for links

c504a91

Close upload modal after successful upload (#1007)

5c0c98c

minor bug fixes

23cd979

Merge branch 'task/WC-151--DRP-Form-Fixes' into task/digital-rocks

c1b8072

Merge branch 'task/WC-136--DRP-External-Files' into task/digital-rocks

7f93454

Merge branch 'task/WC-137--DRP-Publication-Search' into task/digital-…

f6bf61d

…rocks

Fix for publication date

1611b6f

Fixed project system naming, improved project/pub state handling

1dbb304

fix for projects breadcrumbs Go To button

0e5319a

Added View Authors modal, added extra logic for View Team button

ea5e6ca

front end linting

9c5543c

front end linting

5f7f8b9

Merge branch 'task/WC-157--DRP-Projects-Bug' into task/digital-rocks

4f5a310

Merge branch 'task/digital-rocks' into task/WC-168--DRP-View-Team-Modal

933ea44

Merge branch 'task/WC-168--DRP-View-Team-Modal' into task/digital-rocks

5eafb87

Email project authors when their publication request is accepted/reje…

6e19162

…cted (#1008)

Changed curator name to email link when requesting publications

941a2d2

Added support for community data system and project

e79c3be

updated and optimized loading of addon components

1754f24

front end linting

9b7e8e3

improved trash entity handling

404971f

Added support for adding guest users to a project

a2d3fd7

small bug fix

c414dde

update default settings

e536d7b

Merge branch 'task/WC-160--DRP-Guest-Users' into task/digital-rocks

efee296

project publish bug fix

de6d2e1

van-go and others added 15 commits June 18, 2026 14:03

Revert "WC-332 (#1314)"

400f8e8

This reverts commit 31349a9.

Merge branch 'task/digital-rocks' into task/WI-454-DPM-Merge

7b02774

WIP on task/WI-454-DPM-Merge

78a1cfd

Working state for DRP and CEP

1cb3cd9

unused imports cleanup

25ca03b

Cleanup migrations and unused models, added frontend flag for metadata

b06c8b5

moving formatLabel to be a generic shared util function

142dddc

Removing hardcoded DRP naming from publication flow

8c25bd9

lift citations into a common component

363d7ef

added util for default project system name, misc fixes

9d25647

fixes for existing front-end tests

f03feb9

Fixed some hardcoded drp labels

2273c87

cleanup settings

84c95f2

fix for workspace operations

0d15f6d

shayanaijaz added 3 commits July 7, 2026 16:13

backend test fixes, package-lock file update

5a67abe

Merge branch 'main' into task/WI-454-DPM-Merge

06b4405

Revert "Merge branch 'main' into task/WI-454-DPM-Merge"

a7bdc51

This reverts commit 06b4405, reversing changes made to 5a67abe.

wesleyboar reviewed Jul 8, 2026

View reviewed changes

rstijerina reviewed Jul 8, 2026

View reviewed changes

fnets reviewed Jul 8, 2026

View reviewed changes

fnets reviewed Jul 9, 2026

View reviewed changes

Comment thread client/src/styles/settings/font.css

fnets reviewed Jul 17, 2026

View reviewed changes

Uh oh!

Conversation

shayanaijaz commented Jul 7, 2026

Overview

Related

Changes

Testing

UI

Notes

Uh oh!

codecov Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wesleyboar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Footnotes

Uh oh!

wesleyboar Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Footnotes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fnets left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Footnotes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

codecov Bot commented Jul 7, 2026 •

edited

Loading

wesleyboar left a comment •

edited

Loading

wesleyboar Jul 8, 2026 •

edited

Loading