Skip to content

RANGER-5679: Document OpenSearch audit setup in docker README and per… #1057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
paras200 wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

RANGER-5679: Document OpenSearch audit setup in docker README and per… #1057

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions dev-support/ranger-docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Use Dockerfiles in this directory to create docker images and run them to build
~~~
export RANGER_DB_TYPE=postgres

# valid values for RANGER_DB_TYPE: mysql/postgres/oracle
# valid values for RANGER_DB_TYPE: mysql/postgres/oracle/sqlserver
~~~

### Apache Ranger Build
Expand Down Expand Up @@ -79,7 +79,7 @@ cd dev-support/ranger-docker
# To enable file based sync source for usersync do:
# export ENABLE_FILE_SYNC_SOURCE=true

# valid values for RANGER_DB_TYPE: mysql/postgres/oracle
# valid values for RANGER_DB_TYPE: mysql/postgres/oracle/sqlserver

docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f docker-compose.ranger-pdp.yml -f docker-compose.ranger-kms.yml up -d

Expand Down Expand Up @@ -119,6 +119,9 @@ used by the Solr audit path.

##### Setup

With the default `RANGER_DB_TYPE=postgres`, OpenSearch auditing is preconfigured and runs
out of the box — the commands below need no `install.properties` changes.

~~~
# Prerequisites: build the audit-dispatcher tarball and download archives
mvn clean package -DskipTests -pl distro -am
Expand Down Expand Up @@ -149,11 +152,12 @@ docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-opensearch.
-f docker-compose.ranger-audit-dispatcher-opensearch.yml up -d ranger-audit-ingestor ranger-audit-dispatcher-opensearch
~~~

For **fresh Ranger installs** using OpenSearch for audits, set `audit_store=opensearch` in
`scripts/admin/ranger-admin-install-postgres.properties` and configure the `audit_opensearch_*` properties.
To use OpenSearch with **mysql, oracle or sqlserver** instead, enable the OpenSearch block in the matching
`scripts/admin/ranger-admin-install-${RANGER_DB_TYPE}.properties` (uncomment the `audit_store=opensearch`
and `audit_opensearch_*` lines and comment out the Solr block) before running the setup commands.

For **existing Solr-based installs**, switching stores requires updating `audit_store=opensearch`
in install.properties, configuring the `audit_opensearch_*` properties, and restarting Ranger Admin.
For **existing Solr-based installs**, switch stores by setting `audit_store=opensearch` (and the
`audit_opensearch_*` properties) in install.properties and restarting Ranger Admin.
Similarly, check the `depends` section of the `docker-compose.ranger-service.yaml` file and add docker-compose files for these services when trying to bring up the `service` container.

#### Bring up all containers
Expand All @@ -180,6 +184,9 @@ docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-kafka.yml -
# Solr dispatcher
docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-audit-dispatcher-solr.yml up -d

# OpenSearch dispatcher (requires the ranger-opensearch container; see "OpenSearch audit flow" above)
docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-opensearch.yml -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-audit-dispatcher-opensearch.yml up -d

# HDFS dispatcher
docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-hadoop.yml -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-audit-dispatcher-hdfs.yml up -d
~~~
9 changes: 9 additions & 0 deletions dev-support/ranger-docker/scripts/admin/ranger-admin-install-mysql.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,15 @@ audit_elasticsearch_password=elasticsearch
audit_elasticsearch_index=ranger_audits
audit_elasticsearch_bootstrap_enabled=true

# audit_store=opensearch
# audit_opensearch_urls=ranger-opensearch
# audit_opensearch_port=9200
# audit_opensearch_protocol=http
# audit_opensearch_user=admin
# audit_opensearch_password=admin
# audit_opensearch_index=ranger_audits
# audit_opensearch_bootstrap_enabled=true

policymgr_external_url=http://ranger-admin:6080
policymgr_http_enabled=true

Expand Down
9 changes: 9 additions & 0 deletions dev-support/ranger-docker/scripts/admin/ranger-admin-install-oracle.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,15 @@ audit_elasticsearch_password=elasticsearch
audit_elasticsearch_index=ranger_audits
audit_elasticsearch_bootstrap_enabled=true

# audit_store=opensearch
# audit_opensearch_urls=ranger-opensearch
# audit_opensearch_port=9200
# audit_opensearch_protocol=http
# audit_opensearch_user=admin
# audit_opensearch_password=admin
# audit_opensearch_index=ranger_audits
# audit_opensearch_bootstrap_enabled=true

policymgr_external_url=http://ranger-admin:6080
policymgr_http_enabled=true

Expand Down
9 changes: 9 additions & 0 deletions dev-support/ranger-docker/scripts/admin/ranger-admin-install-sqlserver.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,15 @@ audit_elasticsearch_password=elasticsearch
audit_elasticsearch_index=ranger_audits
audit_elasticsearch_bootstrap_enabled=true

# audit_store=opensearch
# audit_opensearch_urls=ranger-opensearch
# audit_opensearch_port=9200
# audit_opensearch_protocol=http
# audit_opensearch_user=admin
# audit_opensearch_password=admin
# audit_opensearch_index=ranger_audits
# audit_opensearch_bootstrap_enabled=true

policymgr_external_url=http://ranger-admin:6080
policymgr_http_enabled=true

Expand Down