Merge pull request #2235 from codalab/update-doc

Improve documentation

Author: ObadaS

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/Django-3.md

@@ -0,0 +1,26 @@
+!!! note "After upgrading from Codabench <1.20, you will need to rebuild containers, run Django migrations and upgrade compute workers."
+
+1. Rebuild containers
+
+```sh
+docker compose build
+docker compose up -d
+```
+
+2. Run migrations and collect static
+
+```sh
+docker compose exec django ./manage.py makemigrations
+docker compose exec django ./manage.py migrate
+docker compose exec django ./manage.py collectstatic --noinput
+```
+
+3. Upgrade compute workers
+
+For every compute worker associated to the instance (default queue) or to a custom queue, you need to update the worker:
+
+```sh
+<ssh into worker>
+docker compose down
+docker compose up -d --pull always
+```

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/Django-4.md

@@ -0,0 +1,53 @@
+!!! note "After upgrading from Codabench <1.23.0, you will need to perform manual interventions."
+
+
+## Django
+The Django version upgrade will require all the containers to be rebuilt:
+```sh
+docker compose build --no-cache
+docker compose up -d
+```
+
+Afterward, you will need to run some migrations: 
+```sh
+docker compose exec django python manage.py migrate
+```
+If it asks you to do a makemigration instead, you will have to run the following command:
+```sh
+docker compose exec django python manage.py makemigrations --merge
+```
+You might need to modify `/app/src/apps/datasets/migrations/0014_merge_20251212_0942.py`  to remove the `datagroup` changes
+```python
+# Generated by Django 4.2.23 on 2025-09-08 12:32
+
+from django.db import migrations, models
+import utils.data
+import utils.storage
+
+class Migration(migrations.Migration):
+
+        dependencies = [
+            ("datasets", "0010_auto_20250218_1100"),
+        ]
+
+        operations = [
+            migrations.AlterField(
+                model_name="data",
+                name="data_file",
+                field=models.FileField(
+                    blank=True,
+                    null=True,
+                    storage=utils.storage.PrivateStorageClass(),
+                    upload_to=utils.data.PathWrapper("dataset"),
+                ),
+            ),
+            migrations.AlterField(
+                model_name="data",
+                name="id",
+                field=models.BigAutoField(
+                    auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
+                ),
+            ),
+        ]
+```
+Notice that there are only two migrations in this file instead of 3. Simply delete the last one (it tries to change `datagroups` even after `datagroups` was deleted, which is why it fails)

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/Docker-py.md

@@ -0,0 +1,20 @@
+!!! note "After upgrading from Codabench <1.22.0, you will need to rebuild containers, run Django migrations and upgrade compute workers."
+
+
+## Main Instance
+Some of the changes will require a migration and `collectstatic` commands to be run: 
+```sh
+docker compose build && docker compose up -d
+docker compose exec django python manage.py migrate
+docker compose exec django python manage.py collectstatic --no-input
+```
+
+
+There is a new environment variable for the contact email:
+```
+CONTACT_EMAIL=info@codabench.org
+```
+Make sure to add it to your `.env` file before launching the containers
+
+## Compute Workers
+Major compute workers changes will require updating the Compute Worker images for both Docker and Podman. Podman workers will also need Podman 5.4 minimum to work on the host

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/Hide-output.md

@@ -0,0 +1,5 @@
+!!! note "After upgrading from Codabench <1.19, you will need to perform a Django migration ([#1838](https://github.com/codalab/codabench/pull/1838), [#1851](https://github.com/codalab/codabench/pull/1851))"
+
+```bash
+docker compose exec django ./manage.py migrate
+```

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/Minio-image.md

@@ -0,0 +1,33 @@
+!!! note "After upgrading from Codabench <1.21.1, you will need to migrate MinIO, rebuild containers, run Django migrations and upgrade compute workers."
+
+
+### 1. MinIO migration (depending on setup)
+If you are running MinIO locally (defined by `.env` and `docker-compose.yml`), you may require to follow the following MinIO upgrade instructions, to duplicate and mirror your buckets and convert them to the new format:
+https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-migrate-fs-gateway.html
+
+### 2. Rebuild all containers
+```sh
+docker compose build && docker compose up -d
+```
+
+### 3. Django migration
+
+```sh
+docker compose exec django ./manage.py migrate
+```
+
+### 4. Collect static files
+
+```sh
+docker compose exec django ./manage.py collectstatic --no-input
+```
+
+### 5. Upgrade compute workers
+
+For every compute worker associated to the instance (default queue) or to a custom queue, you need to update the worker:
+
+```sh
+<ssh into worker>
+docker compose down
+docker compose up -d --pull always
+```

documentation/docs/Developers_and_Administrators/Upgrading_Codabench/index.md

@@ -22,4 +22,9 @@ You can find here various manual intervention needed depending on which version
 - [Submissions and Participants Counts](Submissions-and-Participants-Counts.md) (version < 1.14.0)
 - [Homepage counters](Homepage-counters.md) (version < 1.15.0)
 - [User removal](User-removal.md) (version < 1.17.0)
-- [Database size Fix](Database-size-fixes.md) (version < 1.18.0)
+- [Database size Fix](Database-size-fixes.md) (version < 1.18.0)
+- [Hide output](Hide-output.md) (version < 1.19.0)
+- [Django 3.0](Django-3.md) (version < 1.20.0)
+- [Minio image](Minio-image.md) (version < 1.21.1)
+- [Docker-py](Docker-py.md) (version < 1.22.0)
+- [Django 4.2](Django-4.md) (version < 1.23.0)

documentation/docs/Organizers/Benchmark_Creation/Advanced-Tutorial.md

@@ -110,26 +110,7 @@ After the bundle has been uploaded, you will see the screenshot shown below.
 ![image](../../_attachments/102434028-e182bb00-404e-11eb-9de8-002e4589db66_17528513099222646.png)
 
 
-
-## Benchmark Examples
-
-Example bundles for code & dataset competition can be found here:
-
-https://github.com/codalab/competition-examples/tree/master/codabench
-
-### Iris
-
-[Iris Codabench Bundle](https://github.com/codalab/competition-examples/tree/master/codabench/iris) is a simple benchmark involving two phases, code submission and results submission.
-
-### AutoWSL
-
-Two versions of the [Automated Weakly Supervised Learning Benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl):
-- [Code submission benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl/code_submission)
-- [Dataset submission benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl/dataset_submission)
-
-### Mini-AutoML
-
-[Mini-AutoML Bundle](https://github.com/codalab/competition-examples/tree/master/codabench/mini-automl) is a benchmark template for Codabench, featuring code submission to multiple datasets (tasks).
+!!! tip "You can find benchmark templates to start your own project in [Benchmark Examples](Benchmark-Examples.md) page."
 
 
 ## How do I set up submission comments for multiple submissions?

documentation/docs/Organizers/Benchmark_Creation/Benchmark-Examples.md

@@ -0,0 +1,30 @@
+Example bundles can be found here:
+
+[https://github.com/codalab/competition-examples/tree/master/codabench](https://github.com/codalab/competition-examples/tree/master/codabench)
+
+More details are given below, to help you select the bundle that suits your own requirements. You can easily clone a template and customize it to match your project.
+
+### Iris
+
+[Iris Codabench Bundle](https://github.com/codalab/competition-examples/tree/master/codabench/iris) is a simple benchmark involving two phases, code submission and results submission.
+
+### Classify Wheat Seeds
+
+We propose three versions of the [Classify Wheat Seeds](https://github.com/codalab/competition-examples/tree/master/codabench/wheat_seeds):
+- [Result submission bundle](https://github.com/codalab/competition-examples/tree/master/codabench/wheat_seeds/results_submission_bundle), with simple submission of predictions
+- [Code submission bundle](https://github.com/codalab/competition-examples/tree/master/codabench/wheat_seeds/code_submission_bundle), with submission of Python algorithm
+- [Ingestion during scoring bundle](https://github.com/codalab/competition-examples/tree/master/codabench/wheat_seeds/ingestion_during_scoring_bundle), where ingestion and scoring run in parallel
+
+### Mini-AutoML
+
+[Mini-AutoML Bundle](https://github.com/codalab/competition-examples/tree/master/codabench/mini-automl) is a benchmark template for Codabench, featuring code submission to multiple datasets (tasks).
+
+### AutoWSL
+
+You can find two versions of the [Automated Weakly Supervised Learning Benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl):
+- [Code submission benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl/code_submission)
+- [Dataset submission benchmark](https://github.com/codalab/competition-examples/tree/master/codabench/autowsl/dataset_submission)
+
+### GPU test
+
+[GPU test bundle](https://github.com/codalab/competition-examples/tree/master/codabench/example_GPU) is an example bundle to test if GPUs are available or not. It serves as testing compute workers and does **not** contain any problem to solve.

documentation/docs/Organizers/Benchmark_Creation/Competition-docker-image.md

@@ -3,7 +3,7 @@ The competition docker image defines the docker environment in which the submiss
 ## Default competition docker image
 
 The default competition docker image is `codalab/codalab-legacy:py37`. 
-More information here: https://github.com/codalab/codalab-dockers
+More information and base images are available here: https://github.com/codalab/codalab-dockers
 
 
 ## Set up another image
@@ -14,23 +14,56 @@ You can select another docker image:
 - In the editor field "Competition Docker image" as shown in the following screenshot:
 ![](_attachments/b8124291-92ea-4d1d-b743-00fd7a35c313_17534366792118.jpg)
 
-## Building an image
-
 If the default image does not suit your needs (missing libraries, etc.), you can either:
 
-- Select an existing image from DockerHub
+- Select any existing image from DockerHub
 - Create your own image from scratch
-- Create a custom image based on the Codalab image. (more information below)
+- Create a custom image based on the CodaLab image (more information below)
+
+## Custom image based on CodaLab image
 
 If you wish to create a custom image based on the Codalab image, you can follow the steps below:
 
-1) Install Docker  
-2) Sign up to DockerHub  
-3) `docker run -itd -u root codalab/codalab-legacy:py39 /bin/bash`  
-4) Use `docker ps` to find running container id  
-5) Now run `docker  exec -it -u root <CONTAINER ID> bash`  
-6) Install anything you want at the docker container shell (`apt-get install`, `pip install`, etc.)  
-7) Exit the shell with `exit`  
-8) `docker commit <CONTAINER ID> username/image:tag`  
-9) `docker login`  
-10) `docker push username/image:tag`  
+#### Preliminary steps
+
+1) Install Docker
+2) Sign up to [DockerHub](https://hub.docker.com/)  
+
+#### Method 1: update the image from a container
+
+1) Start a container using the base image:
+```sh
+docker run -itd -u root codalab/codalab-legacy:py39 /bin/bash
+```
+2) Identify the running container ID using `docker ps`
+3) Enter inside the container:
+```sh
+docker  exec -it -u root <CONTAINER ID> bash
+```  
+4) Install anything you want at the docker container shell (`apt-get install`, `pip install`, etc.)  
+5) Exit the shell with `exit`  
+6) Push the new version to your DockerHub account:
+```sh
+docker commit <CONTAINER ID> username/image:tag
+docker login
+docker push username/image:tag
+```
+
+#### Method 2: update the Dockerfile and re-build
+
+1) Download the Dockerfile of the base image:
+https://github.com/codalab/codalab-dockers/blob/master/legacy-py39/Dockerfile
+
+2) Edit the file to include any library or program you need
+
+3) Build the image
+```sh
+docker build -t username/image:tag .
+```
+
+4) Push it to your DockerHub account
+
+```sh
+docker login
+docker push username/image:tag
+```

documentation/docs/Organizers/Benchmark_Creation/Getting-started-with-Codabench.md

@@ -8,6 +8,8 @@ This getting started tutorial shows a **simple example** of how to create a comp
 - Download the [sample competition bundle](https://github.com/codalab/competition-examples/blob/master/codabench/wheat_seeds/code_submission_bundle.zip) and the [sample submission](https://github.com/codalab/competition-examples/blob/master/codabench/wheat_seeds/sample_code_submission.zip).
 - Do not unzip them.
 
+!!! tip "You can find benchmark templates to start your own project in [Benchmark Examples](Benchmark-Examples.md) page."
+
 ## Create a competition
 - From the front page of [Codabench](https://codabench.org), in the top menu, go to the **Benchmark > Management** 
 - Click the green **Upload** button at the top right.