Skip to content

Commit 0ceb792

Browse files
RenkuBotdependabot[bot]aledeganowesjdjsgaist
authored
release 2.13.0 (#4292)
* chore(deps): bump numpy from 2.3.5 to 2.4.0 in /global-images/datascience in the python group across 1 directory (#4298) * feat: add option to disable ClusterRoles * feat: add session alerts (#4267) * refactor: remove V1 resources (#4281) * chore(deps): bump numpy from 2.3.5 to 2.4.0 in /global-images/datascience in the python group across 1 directory (#4298) * feat: add option to disable ClusterRoles * refactor: remove core and graph They are obsolete and unused components * refactor: removed core and graph elements from values file * feat: add "charts" cache folder to gitignore This folder can be created when testing locally. * refactor: moved session related eleemtns from notebooks to data-service * refactor: removed obsolete notebooks elements * fix(data-service/deployment): update path to session-env-secret.yaml * refactor: removed everything enableV1Services related * refactor: add deletion information to changelog * fix: remove missed pod selector for V1 core * fix: improve changelog text Co-authored-by: Tasko Olevski <tasko.olevski@sdsc.ethz.ch> * refactor: remove additional obsolete entries from values.yaml * refactor: remove network policy for v1 sessions * refactor: remove core and kg related entries from gateway * refactor: remove obsolete core cookies handling * fix(gateway): add back kg url Explicitly hard code it and webhooks as to-be-removed as they are v1 only features * fix: postgres-password is the key to use for an existing secret --------- Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alessandro Degano <a.degano@gmail.com> Co-authored-by: Tasko Olevski <tasko.olevski@sdsc.ethz.ch> Co-authored-by: Flora Thiebaut <flora.thiebaut@sdsc.ethz.ch> * feat: upgrade data services to 0.66.0 (#4311) * feat: update amalthea to 0.23.0 (#4310) * Add user documentation about the search query (#4315) * feat: UI upgrade 4.17.0, data svc 0.67.0 (#4327) * feat: update ui to 4.17.0 * feat: update data service to 0.67.0 * feat: Upgrade Keycloak to 22.0.4 * build: bump renku-gateway to v1.9.0 (#4328) Main change: the gateway has Renku legacy support removed. * fix: docs/package-lock.json (#4333) --------- Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alessandro Degano <a.degano@gmail.com> Co-authored-by: Wes Johnson <515042+wesjdj@users.noreply.github.com> Co-authored-by: Samuel Gaist <samuel.gaist@idiap.ch> Co-authored-by: Tasko Olevski <tasko.olevski@sdsc.ethz.ch> Co-authored-by: Flora Thiebaut <flora.thiebaut@sdsc.ethz.ch> Co-authored-by: Tasko Olevski <16360283+olevski@users.noreply.github.com> Co-authored-by: eikek <701128+eikek@users.noreply.github.com>
1 parent 9017c8c commit 0ceb792

62 files changed

Lines changed: 1266 additions & 3043 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ _sidebar.rst.inc
2525
# The requirements lockfile is generated when packaging the chart
2626
helm-chart/renku/requirements.lock
2727

28+
# Charts dependencies downloads
29+
helm-chart/renku/charts
30+
2831
# Created by https://www.gitignore.io/api/vim,macos,linux,emacs,scala,python,visualstudiocode,helm
2932
# Edit at https://www.gitignore.io/?templates=vim,macos,linux,emacs,scala,python,visualstudiocode,helm
3033

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
title: High Memory Usage
3+
---
4+
5+
## Why This Alert Was Triggered
6+
7+
Your session is using more than 90% of its allocated memory capacity. This can happen when:
8+
9+
- Your code is processing large datasets
10+
- Your code has memory leaks or is accumulating data in memory without releasing it
11+
- The memory allocation for your session is too small for your workload
12+
13+
## What This Means
14+
15+
When memory usage is this high, your session is at risk of:
16+
17+
- Becoming unresponsive or slow
18+
- Being terminated by the system if it exceeds its memory limit ([Out of Memory kill](./20-session-restarted.md))
19+
- Losing unsaved work if the session terminates unexpectedly
20+
21+
## Steps to Remedy
22+
23+
### Immediate Actions
24+
25+
1. Commit and push any important changes to avoid losing them
26+
2. Free up memory:
27+
- Stop any running processes you don't need
28+
- Clear large variables from memory in your notebooks or scripts (e.g., `del variable` in Python)
29+
- Restart your Python kernel if using Jupyter notebooks
30+
31+
### Longer-Term Solutions
32+
33+
1. Optimise your code:
34+
- Process data in smaller chunks
35+
- Use generators or iterators instead of loading full datasets
36+
- Delete variables you no longer need during execution
37+
- Use memory-efficient data types and libraries
38+
39+
2. Request more memory:
40+
- Pause your current session
41+
- Modify the session resources to use a resource class with more memory, if available
42+
- Resume your session
43+
- See [Resource Pools and Classes](../30-resource-pools-and-classes.md) for more information
44+
45+
3. Profile memory usage:
46+
- Use memory profiling tools to identify which parts of your code use the most memory
47+
- For Python: `memory_profiler`, `tracemalloc`, or `guppy3`
48+
- For R: `pryr::mem_used()` or `profmem`
49+
50+
## Prevention
51+
52+
- Choose an appropriate resource class when starting sessions based on your expected workload
53+
- Test code with small datasets first to estimate memory requirements
54+
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: Session Restarted (Out of Memory)
3+
---
4+
5+
## Why This Alert Was Triggered
6+
7+
Your session ran out of memory and was automatically killed and restarted. This happens when:
8+
9+
- Your session tried to use more memory than was allocated to it.
10+
- Memory usage spiked suddenly, exceeding the limit before you could react
11+
- A process had a memory leak that gradually consumed all available memory
12+
- Multiple memory-intensive operations ran simultaneously
13+
14+
## What This Means
15+
16+
When a session is restarted:
17+
18+
- Any changes not committed to Git and pushed to remote, or saved to disk are gone
19+
- In-progress processes were terminated
20+
- Your Python kernel, R session, or other runtime environments have been reset
21+
22+
## Steps to Remedy
23+
24+
### Immediate Actions
25+
26+
1. Check what was lost:
27+
- Review your Git status to see if you have uncommitted changes: `git status`
28+
- Look for any auto-saved files or checkpoints
29+
- Check if your notebook or IDE has auto-recovery features
30+
31+
2. Prevent recurrence before restarting work:
32+
- Review what you were running when the restart occurred
33+
- Identify memory-intensive operations that need optimisation
34+
35+
### Longer-Term Solutions
36+
37+
1. Optimise memory usage:
38+
- Process data in smaller chunks instead of loading everything at once
39+
- Use memory-efficient data structures and algorithms
40+
- Stream data from disk rather than loading it all into memory
41+
- Delete large variables when you're done with them: `del variable` in Python, `rm(variable)` in R
42+
- Use generators or iterators for large datasets
43+
44+
2. Request more memory:
45+
- Pause your current session
46+
- Modify the session resources to use a resource class with more memory, if available
47+
- Resume your session
48+
- See [Resource Pools and Classes](../30-resource-pools-and-classes.md) for more information
49+
50+
3. Monitor memory usage:
51+
- Add memory profiling to your code
52+
- Use the [High Memory Usage](./10-high-memory-usage.md) alert as an early warning system
53+
54+
4. Save work frequently:
55+
- Commit and push changes to Git regularly
56+
- Save intermediate results to disk
57+
- Use checkpoint systems in long-running computations
58+
59+
## Prevention
60+
61+
- Choose an appropriate resource class when starting sessions based on your expected workload
62+
- Test code with small datasets first to estimate memory requirements
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: Low Disk Space
3+
---
4+
5+
## Why This Alert Was Triggered
6+
7+
Your session has less than 10% of its disk space remaining. This typically happens when:
8+
9+
- You've downloaded or generated large files in your session
10+
- Intermediate files have accumulated over time
11+
- Log files or cache directories have grown large
12+
13+
## What This Means
14+
15+
When disk space is this low:
16+
17+
- You may not be able to save new files
18+
- Running processes might fail
19+
- Your session may become unstable or crash
20+
21+
## Steps to Remedy
22+
23+
### Immediate Actions
24+
25+
1. Push any important changes to your Git repository or external storage to avoid data loss.
26+
27+
2. Identify large files:
28+
```bash
29+
# Find the largest files in your Renku volume
30+
du -h $RENKU_MOUNT_DIR | sort -h | tail -20
31+
```
32+
33+
3. Free up space:
34+
- Delete temporary files, old logs, or intermediate computation results
35+
- Remove downloaded datasets you no longer need
36+
37+
### Longer-Term Solutions
38+
39+
1. Manage data more efficiently:
40+
- Use a [Data Connector](../../40-data/00-data.md) to access data from external storage without copying it to your session's disk (e.g., S3, Azure Blob Storage, external data repositories)
41+
- Use data streaming instead of downloading entire datasets
42+
- Clean up output files regularly
43+
44+
2. Request more disk space:
45+
- Save any unsaved work, push all data you wish to keep to your code repositories and/or data connectors and shut down your current session
46+
- Increase the disk storage by editing the session launcher. This will only be possible if the maximum disk storage for the selected resource class size has not already been met. If it has, you may be able to switch to a different resource class with a larger maximum disk size.
47+
- Restart your session.
48+
- See [Resource Pools and Classes](../30-resource-pools-and-classes.md) for more information
49+
50+
3. Optimise your workflow:
51+
- Configure applications to use temporary directories that you can clean regularly
52+
- Write outputs to compressed formats when possible
53+
- Implement automatic cleanup of intermediate files in your scripts
54+
55+
## Prevention
56+
57+
- Monitor disk usage regularly: `df -h $RENKU_MOUNT_DIR`
58+
- Set up your code to clean up temporary files automatically
59+
- Choose an appropriate resource class with sufficient storage when starting sessions
60+
- Keep only necessary files in your session workspace
61+
- Use a [Data Connector](../../40-data/00-data.md) to store and access large datasets from external storage instead of keeping them on your session disk
62+
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
label: Alerts
2+
collapsed: true

0 commit comments

Comments
 (0)