Skip to content

Commit 9cd0c9d

Browse files
docs: rename ProcessedData to entity-oriented names across docs
- SessionAnalysis: handle-errors, distributed-computing, monitor-progress, alter-tables, delete-data, job-metadata, data-manipulation, table-declaration, type-system (blob example) - FilteredTrace: use-npy-codec (FFT computation on raw traces) - RecordingAnalysis: type-system explanation (object@ storage) - Pipeline example: RawData/Analysis -> Session/TrialStats Only the 03-data-entry tutorial notebook retains ProcessedData (self-contained tutorial with its own schema). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8c4c20b commit 9cd0c9d

11 files changed

Lines changed: 65 additions & 65 deletions

src/explanation/type-system.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,7 @@ result = np.mean(ref) # Downloads automatically
269269
Schema-addressed storage for files and folders. Path mirrors the database structure: `{schema}/{table}/{pk}/{attribute}`.
270270

271271
```python
272-
class ProcessedData(dj.Computed):
272+
class RecordingAnalysis(dj.Computed):
273273
definition = """
274274
-> Recording
275275
---

src/how-to/alter-tables.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,10 +199,10 @@ For tables created before enabling job metadata:
199199
from datajoint.migrate import add_job_metadata_columns
200200

201201
# Dry run
202-
add_job_metadata_columns(ProcessedData, dry_run=True)
202+
add_job_metadata_columns(SessionAnalysis, dry_run=True)
203203

204204
# Apply
205-
add_job_metadata_columns(ProcessedData, dry_run=False)
205+
add_job_metadata_columns(SessionAnalysis, dry_run=False)
206206
```
207207

208208
## Best Practices

src/how-to/delete-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ with dj.conn().transaction:
175175
Session.Trial.insert(corrected_trials)
176176

177177
# 3. Recompute derived data
178-
ProcessedData.populate()
178+
SessionAnalysis.populate()
179179
```
180180

181181
This ensures all derived data remains consistent with source data.

src/how-to/distributed-computing.md

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@ Use `reserve_jobs=True` to enable job coordination:
88

99
```python
1010
# Single worker (default)
11-
ProcessedData.populate()
11+
SessionAnalysis.populate()
1212

1313
# Distributed mode with job reservation
14-
ProcessedData.populate(reserve_jobs=True)
14+
SessionAnalysis.populate(reserve_jobs=True)
1515
```
1616

1717
## How It Works
@@ -26,7 +26,7 @@ With `reserve_jobs=True`:
2626

2727
```python
2828
# Use multiple processes
29-
ProcessedData.populate(reserve_jobs=True, processes=4)
29+
SessionAnalysis.populate(reserve_jobs=True, processes=4)
3030
```
3131

3232
Each process:
@@ -42,10 +42,10 @@ Run the same script on multiple machines:
4242
```python
4343
# worker_script.py - run on each machine
4444
import datajoint as dj
45-
from my_pipeline import ProcessedData
45+
from my_pipeline import SessionAnalysis
4646

4747
# Each worker reserves and processes different jobs
48-
ProcessedData.populate(
48+
SessionAnalysis.populate(
4949
reserve_jobs=True,
5050
display_progress=True,
5151
suppress_errors=True
@@ -60,13 +60,13 @@ Each auto-populated table has a jobs table (`~~table_name`):
6060

6161
```python
6262
# View job status
63-
ProcessedData.jobs
63+
SessionAnalysis.jobs
6464

6565
# Filter by status
66-
ProcessedData.jobs.pending
67-
ProcessedData.jobs.reserved
68-
ProcessedData.jobs.errors
69-
ProcessedData.jobs.completed
66+
SessionAnalysis.jobs.pending
67+
SessionAnalysis.jobs.reserved
68+
SessionAnalysis.jobs.errors
69+
SessionAnalysis.jobs.completed
7070
```
7171

7272
## Job Statuses
@@ -85,7 +85,7 @@ Sync the job queue with current key_source:
8585

8686
```python
8787
# Add new pending jobs, remove stale ones
88-
result = ProcessedData.jobs.refresh()
88+
result = SessionAnalysis.jobs.refresh()
8989
print(f"Added: {result['added']}, Removed: {result['removed']}")
9090
```
9191

@@ -95,10 +95,10 @@ Control processing order with priorities:
9595

9696
```python
9797
# Refresh with specific priority
98-
ProcessedData.jobs.refresh(priority=1) # Lower = more urgent
98+
SessionAnalysis.jobs.refresh(priority=1) # Lower = more urgent
9999

100100
# Process only high-priority jobs
101-
ProcessedData.populate(reserve_jobs=True, priority=3)
101+
SessionAnalysis.populate(reserve_jobs=True, priority=3)
102102
```
103103

104104
## Error Recovery
@@ -107,13 +107,13 @@ Handle failed jobs:
107107

108108
```python
109109
# View errors
110-
errors = ProcessedData.jobs.errors
110+
errors = SessionAnalysis.jobs.errors
111111
for job in errors.to_dicts():
112112
print(f"Key: {job}, Error: {job['error_message']}")
113113

114114
# Clear errors to retry
115115
errors.delete()
116-
ProcessedData.populate(reserve_jobs=True)
116+
SessionAnalysis.populate(reserve_jobs=True)
117117
```
118118

119119
## Orphan Detection
@@ -122,7 +122,7 @@ Jobs from crashed workers are automatically recovered:
122122

123123
```python
124124
# Refresh with orphan timeout (seconds)
125-
ProcessedData.jobs.refresh(orphan_timeout=3600)
125+
SessionAnalysis.jobs.refresh(orphan_timeout=3600)
126126
```
127127

128128
Reserved jobs older than the timeout are reset to pending.
@@ -172,10 +172,10 @@ dj.config.jobs.version_method = "git"
172172

173173
# worker.py - run on each node
174174
from config import *
175-
from my_pipeline import ProcessedData
175+
from my_pipeline import SessionAnalysis
176176

177177
while True:
178-
result = ProcessedData.populate(
178+
result = SessionAnalysis.populate(
179179
reserve_jobs=True,
180180
max_calls=100,
181181
suppress_errors=True,

src/how-to/handle-errors.md

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@ Continue processing despite individual failures:
88

99
```python
1010
# Stop on first error (default)
11-
ProcessedData.populate()
11+
SessionAnalysis.populate()
1212

1313
# Log errors but continue
14-
ProcessedData.populate(suppress_errors=True)
14+
SessionAnalysis.populate(suppress_errors=True)
1515
```
1616

1717
## View Failed Jobs
@@ -20,10 +20,10 @@ Check the jobs table for errors:
2020

2121
```python
2222
# All error jobs
23-
ProcessedData.jobs.errors
23+
SessionAnalysis.jobs.errors
2424

2525
# View error details
26-
for job in ProcessedData.jobs.errors.to_dicts():
26+
for job in SessionAnalysis.jobs.errors.to_dicts():
2727
print(f"Key: {job}")
2828
print(f"Message: {job['error_message']}")
2929
```
@@ -33,7 +33,7 @@ for job in ProcessedData.jobs.errors.to_dicts():
3333
Error stack traces are stored in the jobs table:
3434

3535
```python
36-
job = (ProcessedData.jobs.errors & key).fetch1()
36+
job = (SessionAnalysis.jobs.errors & key).fetch1()
3737
print(job['error_stack'])
3838
```
3939

@@ -43,10 +43,10 @@ Clear error status and rerun:
4343

4444
```python
4545
# Delete error records to retry
46-
ProcessedData.jobs.errors.delete()
46+
SessionAnalysis.jobs.errors.delete()
4747

4848
# Reprocess
49-
ProcessedData.populate(reserve_jobs=True)
49+
SessionAnalysis.populate(reserve_jobs=True)
5050
```
5151

5252
## Retry Specific Jobs
@@ -55,10 +55,10 @@ Target specific failed jobs:
5555

5656
```python
5757
# Clear one error
58-
(ProcessedData.jobs & key & "status='error'").delete()
58+
(SessionAnalysis.jobs & key & "status='error'").delete()
5959

6060
# Retry just that key
61-
ProcessedData.populate(key, reserve_jobs=True)
61+
SessionAnalysis.populate(key, reserve_jobs=True)
6262
```
6363

6464
## Ignore Problematic Jobs
@@ -67,10 +67,10 @@ Mark jobs to skip permanently:
6767

6868
```python
6969
# Mark job as ignored
70-
ProcessedData.jobs.ignore(key)
70+
SessionAnalysis.jobs.ignore(key)
7171

7272
# View ignored jobs
73-
ProcessedData.jobs.ignored
73+
SessionAnalysis.jobs.ignored
7474
```
7575

7676
## Error Handling in make()
@@ -79,16 +79,16 @@ Handle expected errors gracefully:
7979

8080
```python
8181
@schema
82-
class ProcessedData(dj.Computed):
82+
class SessionAnalysis(dj.Computed):
8383
definition = """
84-
-> RawData
84+
-> Session
8585
---
8686
result : float64
8787
"""
8888

8989
def make(self, key):
9090
try:
91-
data = (RawData & key).fetch1('data')
91+
data = (Session & key).fetch1('data')
9292
result = risky_computation(data)
9393
except ValueError as e:
9494
# Log and skip this key
@@ -117,7 +117,7 @@ def make(self, key):
117117
Get exception objects for programmatic handling:
118118

119119
```python
120-
result = ProcessedData.populate(
120+
result = SessionAnalysis.populate(
121121
suppress_errors=True,
122122
return_exception_objects=True
123123
)
@@ -133,7 +133,7 @@ for key, exception in result['error_list']:
133133
Track errors over time:
134134

135135
```python
136-
progress = ProcessedData.jobs.progress()
136+
progress = SessionAnalysis.jobs.progress()
137137
print(f"Pending: {progress.get('pending', 0)}")
138138
print(f"Errors: {progress.get('error', 0)}")
139139
print(f"Success: {progress.get('success', 0)}")

src/how-to/monitor-progress.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Track computation progress and job status.
77
Show progress bar during populate:
88

99
```python
10-
ProcessedData.populate(display_progress=True)
10+
SessionAnalysis.populate(display_progress=True)
1111
```
1212

1313
## Check Remaining Work
@@ -16,7 +16,7 @@ Count entries left to compute:
1616

1717
```python
1818
# What's left to compute
19-
remaining = ProcessedData.key_source - ProcessedData
19+
remaining = SessionAnalysis.key_source - SessionAnalysis
2020
print(f"{len(remaining)} entries remaining")
2121
```
2222

@@ -25,7 +25,7 @@ print(f"{len(remaining)} entries remaining")
2525
Get counts by status:
2626

2727
```python
28-
progress = ProcessedData.jobs.progress()
28+
progress = SessionAnalysis.jobs.progress()
2929
# {'pending': 100, 'reserved': 5, 'error': 3, 'success': 892}
3030

3131
for status, count in progress.items():
@@ -38,19 +38,19 @@ Access jobs by their current status:
3838

3939
```python
4040
# Pending jobs (waiting to run)
41-
ProcessedData.jobs.pending
41+
SessionAnalysis.jobs.pending
4242

4343
# Currently running
44-
ProcessedData.jobs.reserved
44+
SessionAnalysis.jobs.reserved
4545

4646
# Failed jobs
47-
ProcessedData.jobs.errors
47+
SessionAnalysis.jobs.errors
4848

4949
# Completed jobs (if keep_completed=True)
50-
ProcessedData.jobs.completed
50+
SessionAnalysis.jobs.completed
5151

5252
# Skipped jobs
53-
ProcessedData.jobs.ignored
53+
SessionAnalysis.jobs.ignored
5454
```
5555

5656
## View Job Details
@@ -59,10 +59,10 @@ Inspect specific jobs:
5959

6060
```python
6161
# All jobs for a key
62-
(ProcessedData.jobs & key).fetch1()
62+
(SessionAnalysis.jobs & key).fetch1()
6363

6464
# Recent errors
65-
ProcessedData.jobs.errors.to_dicts(
65+
SessionAnalysis.jobs.errors.to_dicts(
6666
order_by='completed_time DESC',
6767
limit=10
6868
)
@@ -73,7 +73,7 @@ ProcessedData.jobs.errors.to_dicts(
7373
See which workers are processing:
7474

7575
```python
76-
for job in ProcessedData.jobs.reserved.to_dicts():
76+
for job in SessionAnalysis.jobs.reserved.to_dicts():
7777
print(f"Key: {job}")
7878
print(f"Host: {job['host']}")
7979
print(f"PID: {job['pid']}")
@@ -86,7 +86,7 @@ Track how long jobs take:
8686

8787
```python
8888
# Average duration of completed jobs
89-
completed = ProcessedData.jobs.completed.to_arrays('duration')
89+
completed = SessionAnalysis.jobs.completed.to_arrays('duration')
9090
print(f"Average: {np.mean(completed):.1f}s")
9191
print(f"Median: {np.median(completed):.1f}s")
9292
```
@@ -112,10 +112,10 @@ This adds hidden attributes to computed tables:
112112

113113
```python
114114
import time
115-
from my_pipeline import ProcessedData
115+
from my_pipeline import SessionAnalysis
116116

117117
while True:
118-
remaining, total = ProcessedData.progress()
118+
remaining, total = SessionAnalysis.progress()
119119

120120
print(f"\rProgress: {total - remaining}/{total} ({(total - remaining) / total:.0%})", end='')
121121

@@ -130,10 +130,10 @@ For distributed mode with job tracking:
130130

131131
```python
132132
import time
133-
from my_pipeline import ProcessedData
133+
from my_pipeline import SessionAnalysis
134134

135135
while True:
136-
status = ProcessedData.jobs.progress()
136+
status = SessionAnalysis.jobs.progress()
137137

138138
print(f"\rPending: {status.get('pending', 0)} | "
139139
f"Running: {status.get('reserved', 0)} | "
@@ -152,7 +152,7 @@ while True:
152152
Check multiple tables:
153153

154154
```python
155-
tables = [RawData, ProcessedData, Analysis]
155+
tables = [Session, SessionAnalysis, TrialStats]
156156

157157
for table in tables:
158158
total = len(table.key_source)

src/how-to/use-npy-codec.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ for rec in large:
159159

160160
```python
161161
@schema
162-
class ProcessedData(dj.Computed):
162+
class FilteredTrace(dj.Computed):
163163
definition = """
164164
-> RawData
165165
---

src/reference/specs/data-manipulation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -357,7 +357,7 @@ About to delete:
357357
Subject: 1 rows
358358
Session: 5 rows
359359
Trial: 150 rows
360-
ProcessedData: 150 rows
360+
SessionAnalysis: 150 rows
361361

362362
Commit deletes? [yes, No]:
363363
```

0 commit comments

Comments
 (0)