276 upgrade temp ids referenced in rerum data (#296)

* Development (#234)

* update metadata (#171)

* update metadata

* modify route name and db.update

* update db.update dependent

* cleanup

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* quickfix

* cleanup

* modify db.update to receive one param {data, collection}

* nodiff

* restore action(data, collection) structure

* undiff

* Changed Collections Parameter for Save()

* Removing /:id put Call

* Removing the Limit from express.json()

* Removing the Limit from express.json()

* getting started with Vault (#190)

* getting started with Vault

* newer Vault

* tests

* removing redundant code

* proper tests passing

* sample Vault

* Bryan's refusal to .jsonld makes this not work as expected

* loading resources

This is incomplete by design. We need to ask Vault to add any resources we want resolved

* expanding the Manifest a bit

* touch up for merge

* touch up for merge

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* Current Project IIIF manifest Creation (#187)

* Current Project IIIF manifest Creation

* Refactored the exportManifest() and endpoint name

* Changed the exportManifest() function

* TPEN ID error handled

* endpoint to move the manifest.json to TPEN-Static-Dev

* Changed @id to id

* Getting all Ids

* Console Clean up

* Added env variables

* Adding Imports

* Adding Imports

* Deleting Ids

* Updating Logic Added

* User not a member cannot change the manifest

* Removing project2

* Adding comments to functions

* add hotkeys service (#184)

* add hotkeys service

* hotkey endpoints

* aggregate hotkeys during project retrieval

* specify hotkey fields to include

* cleanup

* Update Hotkeys.js

* Update ProjectFactory.mjs

* Return hotkeys as an Array of Strings

* aligning with Class changes

* remove create, since .save is not acting correctly

* cleanup and drop .post

* tests restored

no Jest here, just checking exists.

* tests and sinon upgrade

* no db tests directly

* Update exists_unit.test.mjs

* putting post back in...

* adding create back with safety

* adding upsert to accomodate bad errors

* Update Hotkeys.js

* uncatch to let errors through

* expect the  errors to come back

* switch to jest tests

---------

Co-authored-by: cubap <cubap@slu.edu>

* hotifx

* hotfix for symbols.

* delete enabled

* Create API.md

* collaborators and users

* add markdown reader

* package for markdown

* Update API.md

* Update API.md

* touch

* ah codes

* proxy for internal use (#201)

* Endpoint to save changes for the new layer (#199)

* Adding endpoint to save changes from the layer

* Adding New Layer

* No Empty Label and no Annotations

* Updated new Layer

* Adding Items to partOf

* Changing id convention

* Removing updating layer

* Annotation Change

* Adding Delete Endpoint

* not sure...

This type of thing?

* Label Change

* Added Layer Class

* Adding rerum ids to delete and add

* Changing tests

* Update exists.test.mjs

* Changing LayerLabel

* Changes AnnotationCollection Structure

* Update Pages API

* Updating partOf Ids in case of any change

* Adding Layer Metadata Label Change

* example results as a base for comments, delete these files before merge.

* Adding AddLayer Commenting Rest of the APIs

* Added DeleteLayer Back

* Deleting json files

* Updates to the comments

* Making Project to Layer Changes

* Renaming layerAnnotationCollection

* Removing UpdateOne() changes

* Removing UpdateOne() changes

* Removing UpdateOne() changes

* Error Handled for no ProjectID or LayerID

* Clear Code

* send() instead of json()

---------

Co-authored-by: cubap <cubap@slu.edu>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* Create sample.env

* Update sample.env

GitHub per @mepripri

* Update CODEOWNERS

* dev it and hotkeys

* 188 epic middleware to upgrade imported manifests (#209)

* bring in vault

improve Project building from Manifest import

* fixes #206

* add singleton vault as utility

fix #208

* cleanup logs

* 139 factor out type type dependencies (#211)

* determine data type by content

* Removing type dependencies

- Took controller and type out of controller
- Added a function to detect the data types and assign the correct collection

* matching tests to code move

* align with main

* Update driver.mjs (#217)

* 188 epic middleware to upgrade imported manifests (#218)

* bring in vault

improve Project building from Manifest import

* fixes #206

* add singleton vault as utility

fix #208

* cleanup logs

* rename redone

* This is Vault now

* test objects don't validate

* hulk smash 👊🏽

* _sub is no longer missing on invitees

* add temp sub to new user

* Update cd_dev.yaml

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Removing fileSystem from Github API (#214)

* Removing fileSystem from Github API

* Update ProjectFactory.mjs

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* 220 services for bug reporting and feedback (#221)

* file dump init

* services for feedback

* /feedback/feedback is not a good route

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* test results

* added docs and ES6 conversion

* what the Golden AI hell is adding imports from my docs?

* cicd (#222)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update ci_dev.yaml (#219)

* Update ci_dev.yaml

* Update cd_prod.yaml

* Update ci_prod.yaml

* 220 services for bug reporting and feedback (#224)

* file dump init

* services for feedback

* /feedback/feedback is not a good route

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* test results

* added docs and ES6 conversion

* what the Golden AI hell is adding imports from my docs?

* explicitly adding CORS

* API call to Update Profile (#223)

* API call to Update Profile

* existingEmail and existingName

* Changes to comments

* Update User.mjs

* Save AnnotationCollection, Pages and Annotations to RERUM (#215)

* saveCollection to RERUM

* Adding Tinypen to Create RERUM Object

* Update exists.test.mjs

* Update exists.test.mjs

* Update Page.mjs

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* starting some adjustments

* better the tests

* headed home

* multiple ways to extract the data

* retest with suggestions

* layer/page halos

* percolating deletes

* setting up routes

---------

Co-authored-by: Patrick Cuba <cubap@slu.edu>

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Import TPEN28 (#226)

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Test restoration (#229)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* cleanup missing properties, changed method names

* just this route

* id shouldn't be optional here.

out of date test files dropped

* This should never break. What's up?

* bad merge

* Update exists_unit.test.mjs

* test is ugly

The page router needs a projectId as well to actually work

* Update end_to_end_unit.test.mjs

These two cannot work without a corresponding project, so it will need to be rewritten

* nested in router now

* Update exists.test.mjs

* un-mjs

* Refactor all .mjs files to .js and update imports. Closes #194 (#228)

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update cd_dev.yaml

* stop if things are missing

* adding verify on main

* Update ci_dev.yaml (#219)

* Update ci_dev.yaml

* Update cd_prod.yaml

* Update ci_prod.yaml

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Refactor all .mjs files to .js and update imports. Closes #194

* un-mjs

* npm update

* Hey I heard you like tests, so I put tests in your tests

* mjs > js

* no mjs, bad mjs

* fine

* habesroxx

* how bout now

* hide, Jest is coming

* runner love

* Update package-lock.json

* jest no like to run

* null != undefined

* default not defaulting

* fixes "id is not defined"

shoulda wrote test for this

* out of scope, out of effs

---------

Co-authored-by: Priyal Patel <mepripri0712@gmail.com>

* Update package-lock.json

* Using UID to get User Projects

* Update index.mjs

* no label is fine for Pages

* Origin Fetch

* SetHeader Origin

* Update index.js

* Getting User Projects (#237)

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Update index.mjs

* Using UID to get User Projects

* Update index.mjs

* Origin Fetch

* SetHeader Origin

* Update index.js

* Update index.js

* Update index.js

* Localhost URLS Validating (#242)

* Update index.js

* Update index.js

* Update validateURL.js

---------

Co-authored-by: Patrick Cuba <cubap@slu.edu>

* 231 create overwrite layer (#239)

* passing through the projectID

* some tests

* support page

* path in Page class generator

* no label is fine for Pages

* updating layers

* AI generated tests

* Update API.md

* oh auth.

* how'd we miss this?

* old tests

* adjust for tests

* default exports for tests

* wrangling AI tests

Jest mock is a nightmare.

* just skip to move on

* Update Layer.js

* Update index.js

* merged mess unwrap

* dummy

* Update Project.js

* handle labels throughout

* unerring

* update layer organized

* return changes

* avoid hard crash

* prevent crash on a page 404

* full id

After I PUT a new label (this was successful) the "id" on the layer the db obj does not the the prefix and is just the hash.

* you get it

* vaildate all singular changes

* greedier try-catch

* Squashed commit of the following:

commit 44b75cf
Author: Priyal Patel <52342511+mepripri@users.noreply.github.com>
Date:   Wed May 7 10:13:21 2025 -0500

    Localhost URLS Validating (#242)

    * Update index.js

    * Update index.js

    * Update validateURL.js

    ---------

    Co-authored-by: Patrick Cuba <cubap@slu.edu>

commit 032ef45
Merge: f977234 52edaab
Author: cubap <cubap@slu.edu>
Date:   Mon May 5 10:40:24 2025 -0500

    Merge branch 'development' into import-tpen28

commit 52edaab
Author: Priyal Patel <52342511+mepripri@users.noreply.github.com>
Date:   Fri May 2 09:36:27 2025 -0500

    Getting User Projects (#237)

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Update index.mjs

    * Using UID to get User Projects

    * Update index.mjs

    * Origin Fetch

    * SetHeader Origin

    * Update index.js

    * Update index.js

    * Update index.js

commit f977234
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 13:14:16 2025 -0500

    Update index.js

commit dde7c30
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 13:05:02 2025 -0500

    SetHeader Origin

commit d2ec198
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 12:47:32 2025 -0500

    Origin Fetch

commit 83fac3a
Merge: 6e79a8d 2e5bbbd
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Thu May 1 09:31:12 2025 -0500

    Merge branch 'development' into import-tpen28

commit 2e5bbbd
Merge: 443d0a2 9f102fb
Author: Patrick Cuba <cubap@slu.edu>
Date:   Wed Apr 30 22:38:43 2025 -0500

    Merge branch 'development' of https://github.com/CenterForDigitalHumanities/TPEN-services into development

commit 443d0a2
Author: cubap <cubap@slu.edu>
Date:   Wed Apr 30 15:23:02 2025 -0500

    no label is fine for Pages

commit 6e79a8d
Merge: ac0182f 9f102fb
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Wed Apr 30 12:06:38 2025 -0500

    Merge branch 'development' into import-tpen28

commit ac0182f
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Wed Apr 30 11:51:40 2025 -0500

    Update index.mjs

commit 673a5c5
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Tue Apr 29 17:22:08 2025 -0500

    Using UID to get User Projects

commit afe664e
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Mon Apr 28 10:16:27 2025 -0500

    Update index.mjs

commit 6bf9c9a
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:56:11 2025 -0500

    Update index.mjs

commit 0906084
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:50:54 2025 -0500

    Update index.mjs

commit 5dd077e
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 14:33:53 2025 -0500

    Update index.mjs

commit e9971bc
Author: Priyal Patel <mepripri0712@gmail.com>
Date:   Fri Apr 25 12:18:30 2025 -0500

    Update index.mjs

* 422 if no pages are there.

* Update index.js

* Update index.js

* API entries

* typo

---------

Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>

* remove unused file

---------

Co-authored-by: Onoja Victor <111019083+git-voo@users.noreply.github.com>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
Co-authored-by: Onoja <onoja.jsdev@gmail.com>
Co-authored-by: mepripri <mepripri0712@gmail.com>
Co-authored-by: Priyal Patel <52342511+mepripri@users.noreply.github.com>

* upgradeReferences on Page class

* move to shared.js

* merge goof

---------

Co-authored-by: Onoja Victor <111019083+git-voo@users.noreply.github.com>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
Co-authored-by: Onoja <onoja.jsdev@gmail.com>
Co-authored-by: mepripri <mepripri0712@gmail.com>
Co-authored-by: Priyal Patel <52342511+mepripri@users.noreply.github.com>

Author: 4 people

classes/Page/Page.js

@@ -1,5 +1,5 @@
 import dbDriver from "../../database/driver.js"
-import { handleVersionConflict, fetchUserAgent } from "../../utilities/shared.js"
+import { handleVersionConflict, fetchUserAgent, upgradeReferences } from "../../utilities/shared.js"
 import ProjectFactory from "../Project/ProjectFactory.js"
 
 const databaseTiny = new dbDriver("tiny")
@@ -30,6 +30,7 @@ export default class Page {
         Object.assign(this, { id, label, target, partOf: partOf ?? layerId, items, creator, prev, next })
         if (this.id.startsWith(process.env.RERUMIDPREFIX)) {
             this.#tinyAction = 'update'
+            upgradeReferences(this, ['partOf'])
         }
         return this
     }

utilities/shared.js

@@ -368,3 +368,23 @@ export const fetchUserAgent = async (userId) => {
       throw new Error(`Error fetching user agent: ${error.message}`)
    }
 }
+
+/**
+ * Upgrade references in an object to use the RERUMIDPREFIX for specified keys.
+ * For each key, if the value is a string containing a "/", the prefix is replaced with process.env.RERUMIDPREFIX and the last segment.
+ * For the key 'pages', if it is an array, each page object with an 'id' will have its id upgraded similarly.
+ *
+ * @param {object} obj - The object whose references should be upgraded.
+ * @param {string[]} [keys=["partOf", "next", "prev", "target", "pages"]] - The keys to upgrade. Must be valid JS object keys.
+ * @returns {object} The upgraded object.
+ */
+export function upgradeReferences(obj, keys = []) {
+   if (!obj || typeof obj !== 'object') return obj
+   keys.filter(key => typeof key === 'string' && /^[a-zA-Z_$][\w$]*$/.test(key)).forEach(key => {
+      if (!obj[key]) return
+      const hexString = obj[key].split('/').pop()
+      if (!hexString) return
+      obj[key] = `${process.env.RERUMIDPREFIX}${hexString}`
+   })
+   return obj
+}