feat: Support optional anonymous authentication (opensearch-project#114)

* Add Anonymous auth for OpenSearch as an option

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* fix permissions

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* Clean up issues

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* resolve comments

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

---------

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

Author: lezzago

.env

@@ -23,6 +23,10 @@ OPENSEARCH_PORT=9200
 OPENSEARCH_PROTOCOL=https
 OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g
 
+# Anonymous Authentication
+# Set to true to allow access to OpenSearch/Dashboards without login
+OPENSEARCH_ANONYMOUS_AUTH=false
+
 # OpenSearch Dashboards Configuration
 OPENSEARCH_DASHBOARDS_VERSION=3.6.0
 OPENSEARCH_DASHBOARDS_HOST=opensearch-dashboards

AGENTS.md

@@ -205,6 +205,10 @@ OPENSEARCH_PASSWORD='My_password_123!@#'
 OPENSEARCH_HOST=opensearch
 OPENSEARCH_PORT=9200
 
+# Anonymous Authentication
+# Set to true to allow access to OpenSearch/Dashboards without login
+OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false
+
 # OpenTelemetry Collector Configuration
 OTEL_COLLECTOR_VERSION=0.143.0
 OTEL_COLLECTOR_HOST=otel-collector
@@ -937,13 +941,31 @@ When modifying OpenSearch credentials:
 
 Data Prepper uses a template (`pipelines.template.yaml`) with placeholders processed at container startup via `command:` in docker-compose.yml. No manual credential edits needed in pipeline configs.
 
+### Anonymous Authentication
+
+Anonymous auth is controlled by `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` in `.env` (default: `false`). When enabled, users can access OpenSearch Dashboards without logging in.
+
+The setting is injected at container startup via `sed` into two templates:
+- `docker-compose/opensearch/opensearch-security/config.template.yml` → OpenSearch security plugin config
+- `docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml` → Dashboards config
+
+Additionally, `savedObjects.permission.enabled` is conditionally set in the Dashboards config at container startup: `false` when anonymous auth is enabled (so anonymous users can access workspaces created by the init script), and `true` (the default) when anonymous auth is disabled. This version of OSD does not support per-workspace permission grants via the API, so without disabling this setting anonymous users get 403 on all workspace-scoped API calls.
+
+The init script sets the `defaultWorkspace` UI setting after creating the Observability Stack workspace, so all users (including anonymous) land directly in the workspace instead of seeing a workspace picker.
+
+Anonymous users can browse data, view, create, and modify saved objects (visualizations, dashboards, saved queries), explore traces and service maps, run queries, and access the REST API without credentials. They cannot delete existing saved objects or perform admin operations.
+
+Modify access is required because Dashboards persists UI settings on every page load via `update` and `bulk` writes to its system indices. Without these permissions the page fails with 403 errors. Since UI settings and saved objects share the same indices, this also allows modification of existing saved objects.
+
+**Important**: Toggling `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` requires `docker compose down -v` (not just `restart`) because OpenSearch applies security configuration to an internal index on first startup. The `-v` flag removes all stored data (traces, logs, saved dashboards) to force reinitialization.
+
 ### Configuration File Locations
 
-- **OpenSearch**: No custom config file - uses environment variables in docker-compose.yml
+- **OpenSearch**: Environment variables in docker-compose.yml + `docker-compose/opensearch/opensearch-security/config.template.yml` (anonymous auth injected at startup)
 - **OpenTelemetry Collector**: `docker-compose/otel-collector/config.yaml`
 - **Data Prepper**: `docker-compose/data-prepper/pipelines.template.yaml` (credentials injected at startup) and `docker-compose/data-prepper/data-prepper-config.yaml`
 - **Prometheus**: `docker-compose/prometheus/prometheus.yml`
-- **OpenSearch Dashboards**: `docker-compose/opensearch-dashboards/opensearch_dashboards.yml`
+- **OpenSearch Dashboards**: `docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml` (credentials, anonymous auth, and `savedObjects.permission.enabled` injected at startup)
 - **Environment Variables**: `.env` file in repository root
 
 ### Index Management
@@ -967,6 +989,7 @@ When adding new services, consider adding health checks if they depend on other
 
 Development configuration includes:
 - OpenSearch security enabled with default admin/admin credentials
+- Anonymous authentication disabled by default (enable via `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true` in `.env`)
 - SSL certificate verification disabled for development
 - CORS enabled for all origins
 - No network isolation

README.md

@@ -310,6 +310,42 @@ The interactive installer prompts "Customize OpenSearch credentials?" — enter
 
 **How it works:** `.env` is the single source of truth for credentials. OpenSearch, Dashboards, and the init script read from `.env` via environment variables. Data Prepper uses a [template](docker-compose/data-prepper/pipelines.template.yaml) with `OPENSEARCH_USER`/`OPENSEARCH_PASSWORD` placeholders that are injected via `sed` at container startup — no manual config edits needed. OpenSearch uses HTTPS with self-signed certificates, so use `-k` flag with curl commands.
 
+### Anonymous Authentication
+
+By default, OpenSearch Dashboards requires login with credentials. You can enable anonymous authentication to allow users to access Dashboards without a login prompt — useful for demos, workshops, or shared development environments.
+
+**To enable anonymous access**, set in `.env`:
+```env
+OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true
+```
+
+Then restart the stack:
+```bash
+docker compose down -v
+docker compose up -d
+```
+
+> **Warning:** The `-v` flag removes all stored data (traces, logs, saved dashboards). This is required because OpenSearch applies security configuration (roles, role mappings) to an internal index on first startup — restarting without `-v` won't update the security settings.
+
+**What anonymous users can do:**
+- Browse and search all data (traces, logs, metrics)
+- View existing dashboards, visualizations, and saved queries
+- Create and modify visualizations, dashboards, saved queries, and index patterns
+- Explore trace analytics and service maps
+- Run PPL and SQL queries
+- Access the OpenSearch REST API without credentials (e.g., `curl -k https://localhost:9200/_cat/indices`)
+
+**What anonymous users cannot do:**
+- Delete existing dashboards, visualizations, or saved objects
+- Write data to OpenSearch indices
+- Perform admin operations (cluster settings, security configuration, user management)
+
+> **Why modify is allowed:** OpenSearch Dashboards requires `update` and `bulk` write permissions on its system indices to persist UI settings (theme, date format, default index) on every page load. Without these permissions the page fails with 403 "Unable to update UI setting" errors. Because UI settings and saved objects share the same system indices, granting the permissions Dashboards needs to function also allows modification of existing saved objects. Deletion is still blocked.
+
+Admin operations still require full credentials. When disabled (the default), all users must authenticate via the login page.
+
+**Toggling back to require login:** Set `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false` in `.env` and restart with `docker compose down -v && docker compose up -d`. Note that the `-v` flag removes all stored data (traces, logs, saved dashboards) — this is required because OpenSearch applies security configuration to an internal index on first startup.
+
 ## Resource Requirements
 
 | Configuration | Memory Usage | Recommended Minimum |

docker-compose.local-opensearch-dashboards.yml

@@ -17,10 +17,15 @@ services:
     pull_policy: always
     command: >
       /bin/bash -c "
+      ANON_AUTH='${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}';
+      if [ \"$$ANON_AUTH\" != 'true' ]; then ANON_AUTH='false'; fi;
+      if [ \"$$ANON_AUTH\" = 'true' ]; then SO_PERM='false'; else SO_PERM='true'; fi;
       cp /tmp/opensearch_dashboards.template.yml /tmp/opensearch_dashboards.yml &&
+      sed -i 's|OPENSEARCH_ANONYMOUS_AUTH_ENABLED|'$$ANON_AUTH'|g' /tmp/opensearch_dashboards.yml &&
+      sed -i 's|SAVED_OBJECTS_PERMISSION_ENABLED|'$$SO_PERM'|g' /tmp/opensearch_dashboards.yml &&
       sed -i 's|OPENSEARCH_HOSTS|${OPENSEARCH_PROTOCOL}://${OPENSEARCH_HOST}:${OPENSEARCH_PORT}|g' /tmp/opensearch_dashboards.yml &&
-      sed -i 's|OPENSEARCH_USER|${OPENSEARCH_USER}|g' /tmp/opensearch_dashboards.yml &&
       sed -i 's|OPENSEARCH_PASSWORD|${OPENSEARCH_PASSWORD}|g' /tmp/opensearch_dashboards.yml &&
+      sed -i 's|OPENSEARCH_USER|${OPENSEARCH_USER}|g' /tmp/opensearch_dashboards.yml &&
       cp /tmp/opensearch_dashboards.yml /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml &&
       cd /usr/share/opensearch-dashboards &&
       exec ./opensearch-dashboards-docker-entrypoint.sh opensearch-dashboards"

docker-compose.local-opensearch.yml

@@ -30,9 +30,20 @@ services:
       # Initial admin password (required for OpenSearch 2.12+)
       - "OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_PASSWORD}"
       - plugins.query.datasources.encryption.masterkey=BTqK4Ytdz67La1kShIKV3Pu9
+    command: >
+      /bin/bash -c "
+      ANON_AUTH='${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}';
+      if [ \"$$ANON_AUTH\" != 'true' ]; then ANON_AUTH='false'; fi;
+      cp /tmp/opensearch-security/config.template.yml /usr/share/opensearch/config/opensearch-security/config.yml &&
+      sed -i 's|OPENSEARCH_ANONYMOUS_AUTH_ENABLED|'$$ANON_AUTH'|g' /usr/share/opensearch/config/opensearch-security/config.yml &&
+      exec ./opensearch-docker-entrypoint.sh"
     volumes:
       # Persist data across container restarts
       - opensearch-data:/usr/share/opensearch/data
+      # Custom security config for anonymous authentication (template processed at startup)
+      - ./docker-compose/opensearch/opensearch-security/config.template.yml:/tmp/opensearch-security/config.template.yml
+      - ./docker-compose/opensearch/opensearch-security/roles.yml:/usr/share/opensearch/config/opensearch-security/roles.yml
+      - ./docker-compose/opensearch/opensearch-security/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
     ports:
       # REST API endpoint
       - "${OPENSEARCH_PORT}:9200"

docker-compose.yml

@@ -181,6 +181,7 @@ services:
       - OPENSEARCH_DASHBOARDS_PROTOCOL=${OPENSEARCH_DASHBOARDS_PROTOCOL}
       - PROMETHEUS_HOST=${PROMETHEUS_HOST}
       - PROMETHEUS_PORT=${PROMETHEUS_PORT}
+      - OPENSEARCH_ANONYMOUS_AUTH_ENABLED=${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}
     volumes:
       - ./docker-compose/opensearch-dashboards/init/init-opensearch-dashboards.py:/init.py
       - ./docker-compose/opensearch-dashboards/saved-queries-traces.yaml:/config/saved-queries-traces.yaml

docker-compose/README.md

@@ -299,6 +299,30 @@ Then access your cloud Dashboards URL directly — no local container needed.
 
   For production environments with valid certificates, enable verification in each of these places.
 
+## Anonymous Authentication
+
+By default, users must log in to access OpenSearch Dashboards. To skip the login page (useful for demos or workshops), enable anonymous authentication in `.env`:
+
+```env
+OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true
+```
+
+Then restart:
+```bash
+docker compose down -v
+docker compose up -d
+```
+
+> **Warning:** The `-v` flag removes all stored data (traces, logs, saved dashboards). This is required because OpenSearch applies security configuration (roles, role mappings) to an internal index on first startup. Without `-v`, the security settings are not reinitialized and the change won't take effect.
+
+Anonymous users can browse all data, view, create, and modify saved objects (visualizations, dashboards, saved queries, index patterns), explore traces and service maps, run queries, and access the OpenSearch REST API without credentials. They cannot delete existing saved objects or perform admin operations — those still require credentials.
+
+> **Note:** Modify access is required because OpenSearch Dashboards persists UI settings (theme, date format, default index) on every page load via `update` and `bulk` writes to its system indices. Without these permissions the page fails with 403 errors. Since UI settings and saved objects share the same indices, this also allows modification of existing saved objects. Deletion is still blocked.
+
+Set `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false` (the default) to require login for all users. Restart with `docker compose down -v && docker compose up -d` to apply. Note that the `-v` flag removes all stored data (traces, logs, saved dashboards) — this is required because OpenSearch applies security configuration to an internal index on first startup.
+
+**Troubleshooting:** If toggling `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` doesn't take effect, make sure you used `docker compose down -v` (not just `docker compose restart` or `docker compose down` without `-v`). The `-v` flag is required to reinitialize OpenSearch's security configuration.
+
 ## Security Warning
 
 ⚠️ **This configuration is for development only!**
@@ -308,13 +332,15 @@ Security considerations:
 - SSL certificate verification is disabled for development ease
 - Permissive CORS settings
 - No network isolation between services
+- Anonymous authentication is disabled by default (enable via `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true` in `.env`)
 
 For production use:
 - Change default passwords
 - Enable proper SSL/TLS with valid certificates
 - Configure proper authentication and authorization
 - Implement network policies
 - Review and harden all security settings
+- Keep anonymous authentication disabled
 
 Never use this configuration in production without proper hardening.
 

docker-compose/opensearch-dashboards/init/init-opensearch-dashboards.py

@@ -15,6 +15,7 @@
 PROMETHEUS_PORT = os.getenv("PROMETHEUS_PORT", "9090")
 _opensearch_protocol = os.getenv("OPENSEARCH_PROTOCOL", "https")
 OPENSEARCH_ENDPOINT = f"{_opensearch_protocol}://{os.getenv('OPENSEARCH_HOST', 'opensearch')}:{os.getenv('OPENSEARCH_PORT', '9200')}"
+ANONYMOUS_AUTH_ENABLED = os.getenv("OPENSEARCH_ANONYMOUS_AUTH_ENABLED", "false").lower() == "true"
 
 def wait_for_dashboards():
     """Wait for OpenSearch Dashboards to be ready"""
@@ -232,7 +233,7 @@ def create_prometheus_datasource(workspace_id):
 
     payload = {
         "name": datasource_name,
-        "allowedRoles": [],
+        "allowedRoles": ["all_access", "opendistro_security_anonymous_role"] if ANONYMOUS_AUTH_ENABLED else ["all_access"],
         "connector": "prometheus",
         "properties": {
             "prometheus.uri": prometheus_endpoint,
@@ -427,6 +428,45 @@ def create_opensearch_datasource(workspace_id):
         return None
 
 
+def set_default_workspace(workspace_id):
+    """Set the default workspace so all users land here on login.
+
+    When workspace.enabled is true, users see a workspace picker on first load.
+    Setting defaultWorkspace directs all users (including anonymous) straight
+    to the Observability Stack workspace instead.
+
+    Returns True on success, False on failure or skip.
+    """
+    if not workspace_id or workspace_id == "default":
+        print("⏭️  Skipping default workspace (using default)")
+        return False
+
+    print(f"⭐ Setting default workspace: {workspace_id}")
+
+    url = f"{BASE_URL}/api/opensearch-dashboards/settings"
+    payload = {"changes": {"defaultWorkspace": workspace_id}}
+
+    try:
+        response = requests.post(
+            url,
+            auth=(USERNAME, PASSWORD),
+            headers={"Content-Type": "application/json", "osd-xsrf": "true"},
+            json=payload,
+            verify=False,
+            timeout=10,
+        )
+
+        if response.status_code == 200:
+            print("✅ Default workspace set")
+            return True
+        else:
+            print(f"⚠️  Failed to set default workspace: {response.status_code} {response.text}")
+            return False
+    except requests.exceptions.RequestException as e:
+        print(f"⚠️  Error setting default workspace: {e}")
+        return False
+
+
 def set_default_index_pattern(workspace_id, pattern_id):
     """Set the default index pattern"""
     print(f"⭐ Setting default index pattern: {pattern_id}")
@@ -1253,6 +1293,9 @@ def main():
     else:
         workspace_id = create_workspace()
 
+    # Direct all users (including anonymous) to this workspace on login
+    set_default_workspace(workspace_id)
+
     # Create index patterns (idempotent - will skip if already exist)
     # Titles must match exactly what the APM plugin expects
     logs_schema_mappings = '{"otelLogs":{"timestamp":"time","traceId":"traceId","spanId":"spanId","serviceName":"resource.attributes.service.name"}}'

docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml

@@ -31,6 +31,13 @@ opensearch.pingTimeout: 3000
 # Allow requests from any origin for development
 opensearch.requestHeadersAllowlist: [authorization, securitytenant]
 
+# Anonymous authentication - skip login page for browse access
+# Anonymous users can view all data and create/modify saved objects (visualizations, dashboards, saved queries)
+# but cannot delete existing saved objects or perform admin operations
+# Modify access is required — Dashboards persists UI settings via update/bulk writes on page load
+# Set OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true in .env to enable
+opensearch_security.auth.anonymous_auth_enabled: OPENSEARCH_ANONYMOUS_AUTH_ENABLED
+
 # Multi-tenancy configuration
 # Disabled for simpler development setup
 opensearch_security.multitenancy.enabled: false
@@ -79,6 +86,14 @@ explore.discoverTraces.enabled: true
 explore.discoverMetrics.enabled: true
 explore.agentTraces.enabled: true
 workspace.enabled: true
+# Saved-object-level permission checks.
+# When anonymous auth is enabled, this must be false so anonymous users can access
+# workspaces created by the init script. Without this, only the workspace owner (admin)
+# can access workspace-scoped API calls — anonymous users get 403.
+# This version of OSD does not support per-workspace permission grants via the API.
+# When anonymous auth is disabled, this is true (default) to preserve workspace permissions.
+# Value is computed at container startup from OPENSEARCH_ANONYMOUS_AUTH_ENABLED.
+savedObjects.permission.enabled: SAVED_OBJECTS_PERMISSION_ENABLED
 data_source.enabled: true
 data_source.ssl.verificationMode: none
 datasetManagement.enabled: true

docker-compose/opensearch/opensearch-security/config.template.yml

@@ -0,0 +1,28 @@
+# OpenSearch Security Plugin Configuration
+# Controls anonymous authentication for OpenSearch Dashboards access without login
+# Set OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true in .env to enable
+# NOTE: Applying partial config.yml is not supported. Must include entire default config.
+# Only anonymous_auth_enabled and basic_internal_auth_domain are active;
+# unused auth domains (kerberos, proxy, JWT, client cert, LDAP) have been removed.
+
+_meta:
+  type: "config"
+  config_version: 2
+config:
+  dynamic:
+    http:
+      anonymous_auth_enabled: OPENSEARCH_ANONYMOUS_AUTH_ENABLED
+      xff:
+        enabled: false
+        internalProxies: "192\\.168\\.0\\.10|192\\.168\\.0\\.11"
+    authc:
+      basic_internal_auth_domain:
+        description: "Authenticate via HTTP Basic against internal users database"
+        http_enabled: true
+        transport_enabled: true
+        order: 4
+        http_authenticator:
+          type: "basic"
+          challenge: true
+        authentication_backend:
+          type: "intern"