diff --git a/docs/faq.rst b/docs/faq.rst index ceed0a2e91..7df353d73a 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -22,13 +22,18 @@ you may use the Docker image to run pipelines as a single command: .. code-block:: bash docker run --rm \ - -v "$(pwd)":/codedrop \ ghcr.io/aboutcode-org/scancode.io:latest \ - sh -c "run scan_codebase /codedrop" \ + -v "$(pwd)":/codedrop \ + run scan_codebase /codedrop \ > results.json Refer to the :ref:`cli_run` section for more about this approach. +.. tip:: + Prefer a one-liner? Use this to scan your current directory:: + + curl -sSL https://raw.githubusercontent.com/aboutcode-org/scancode.io/main/etc/scripts/run-scan.sh | bash + .. _faq_which_pipeline: Which pipeline should I use? diff --git a/docs/index.rst b/docs/index.rst index 85d5c06ba2..8a32c9e26c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,19 +1,22 @@ -ScanCode.io documentation +ScanCode.io Documentation ========================= -Welcome to the very start of your ScanCode.io journey! In this documentation -you’ll find information on: +Welcome! This is your starting point for everything ScanCode.io. -- An overview of ScanCode.io and ScanPipe -- Installation instructions -- Tutorials to get you started -- Reference documentation about the ScanPipe concepts, Pipelines, Pipes and more -- How to make technical contributions to the project and the community +In this documentation, you’ll find: + +- A **QuickStart guide** to run your first scan +- An **overview** of what ScanCode.io is and what it can do +- **Installation instructions** for full access +- Step-by-step **tutorials** to dive deeper +- In-depth **reference docs** on concepts, pipelines, configuration, and APIs +- Guides on how to **contribute** to the project and community .. toctree:: :maxdepth: 2 :caption: Getting Started + quickstart introduction installation user-interface @@ -23,7 +26,7 @@ you’ll find information on: .. toctree:: :maxdepth: 2 - :caption: Tutorial + :caption: Tutorials tutorial_web_ui_analyze_docker_image tutorial_web_ui_review_scan_results @@ -37,7 +40,7 @@ you’ll find information on: .. toctree:: :maxdepth: 2 - :caption: Reference Documentation + :caption: Reference scanpipe-concepts built-in-pipelines diff --git a/docs/installation.rst b/docs/installation.rst index 83cc1ad197..2cc29642e6 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -221,8 +221,8 @@ And visit the web UI at: http://localhost/project/ .. _local_development_installation: -Local development installation ------------------------------- +Local development +----------------- Supported Platforms ^^^^^^^^^^^^^^^^^^^ @@ -247,7 +247,7 @@ Before you install ScanCode.io, make sure you have the following prerequisites: * **Python: versions 3.10 to 3.13** found at https://www.python.org/downloads/ * **Git**: most recent release available at https://git-scm.com/ - * **PostgreSQL**: release 11 or later found at https://www.postgresql.org/ or + * **PostgreSQL**: release 13 or later found at https://www.postgresql.org/ or https://postgresapp.com/ on macOS .. _system_dependencies: diff --git a/docs/introduction.rst b/docs/introduction.rst index 051903f5d3..618b8ec449 100644 --- a/docs/introduction.rst +++ b/docs/introduction.rst @@ -105,8 +105,3 @@ images, containers, root filesystems, and virtual machine images. As a common practice, ScanCode.io releases usually follow ScanCode Toolkit releases to ensure the latest improvements of the scanning engines are included in the latest release of ScanCode.io. - - -.. Some of this documentation is borrowed from the metaflow documentation and is also - under Apache-2.0 -.. Copyright (c) Netflix diff --git a/docs/quickstart.rst b/docs/quickstart.rst new file mode 100644 index 0000000000..11189c1e31 --- /dev/null +++ b/docs/quickstart.rst @@ -0,0 +1,107 @@ +.. _quickstart: + +QuickStart +========== + +Run a Scan (no installation required!) +-------------------------------------- + +The **fastest way** to get started and **scan a codebase** — +**no installation needed** — is by using the latest +**ScanCode.io Docker image**. + +.. warning:: + **Docker must be installed on your system**. + Visit the `Docker documentation `_ to install + it for your platform. + +To run the :ref:`pipeline_scan_codebase` pipeline on a **local directory** +with a **single command**: + +.. code-block:: bash + + docker run --rm \ + -v "$(pwd)":/codedrop \ + ghcr.io/aboutcode-org/scancode.io:latest \ + run scan_codebase /codedrop \ + > results.json + +Let's unpack what each part of the command does: + +- ``docker run --rm`` + Runs a temporary Docker container that is automatically removed after it finishes. + +- ``-v "$(pwd)":/codedrop`` + Mounts your current directory into the container at ``/codedrop`` so it can be + scanned. + +- ``ghcr.io/aboutcode-org/scancode.io:latest`` + Uses the latest ScanCode.io image from GitHub Container Registry. + +- ``run scan_codebase /codedrop`` + Runs the ``scan_codebase`` pipeline inside the container, using the mounted directory + as the input source. + +- ``> results.json`` + Saves the scan output to a ``results.json`` file on your machine. + +The result? A **full scan of your local directory — no setup, one command!** + +See the :ref:`RUN command ` section for more details on this command. + +.. note:: + Not sure which pipeline to use? Check out :ref:`faq_which_pipeline`. + +Next Step: Local Installation +----------------------------- + +Install ScanCode.io, to **unlock all features**: + +- **User Interface:** Explore dashboards, codebase data, charts, and scan results. + See :ref:`user_interface`. +- **Project Management:** Create, filter, and monitor projects. +- **REST API:** Automate your scans with the :ref:`rest_api`. +- **CLI:** Use the :ref:`command_line_interface` to work from the terminal. +- **Webhooks:** Get real-time updates via custom integrations. See :ref:`webhooks`. +- **Slack Notifications:** Send project updates to Slack. Follow setup in + :ref:`webhooks_slack_notifications`. + +See the :ref:`installation` chapter for the full list of installation options. + +Integrate with Your Workflows +----------------------------- + +ScanCode.io can be part of your CI/CD workflow. + +GitHub Actions +^^^^^^^^^^^^^^ + +Use the official `scancode-action `_ +to integrate **ScanCode.io into your GitHub workflows** with ease. + +This action lets you: + +- **Run pipelines** +- **Check for compliance issues** +- **Detect vulnerabilities** +- **Generate SBOMs and scan results** + +Example usage: + +.. code-block:: yaml + + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + path: scancode-inputs + - uses: aboutcode-org/scancode-action@main + with: + pipelines: "scan_codebase" + output-formats: "json xlsx spdx cyclonedx" + +Full details available at: +https://github.com/aboutcode-org/scancode-action + +.. tip:: + Learn more about automation options in the :ref:`automation` section. diff --git a/etc/scripts/run-scan.sh b/etc/scripts/run-scan.sh new file mode 100755 index 0000000000..75187f3d21 --- /dev/null +++ b/etc/scripts/run-scan.sh @@ -0,0 +1,59 @@ +#!/bin/bash + +# SPDX-License-Identifier: Apache-2.0 +# +# http://nexb.com and https://github.com/aboutcode-org/scancode.io +# The ScanCode.io software is licensed under the Apache License version 2.0. +# Data generated with ScanCode.io is provided as-is without warranties. +# ScanCode is a trademark of nexB Inc. + +# Usage: +# Default scan of current directory using 'scan_codebase': +# ./run-scan.sh +# +# Run pipelines on current directory: +# ./run-scan.sh scan_codebase find_vulnerabilities +# +# Run pipelines on a specific directory: +# ./run-scan.sh scan_codebase /path/to/scan/dir +# +# Run multiple pipelines on a specific directory (directory must come last): +# ./run-scan.sh scan_codebase find_vulnerabilities /my/codebase +# +# Scan a directory using default pipeline: +# ./run-scan.sh /path/to/codebase + +set -e + +readonly SCIO_DOCKER_IMAGE="ghcr.io/aboutcode-org/scancode.io:latest" +readonly RESULTS_LOCATION="results.json" + +# Default values +SCAN_DIR="$(pwd)" +PIPELINES=() + +# Check if last argument is a directory +LAST_ARG="${!#}" +if [ -d "$LAST_ARG" ]; then + SCAN_DIR="$LAST_ARG" + PIPELINES=("${@:1:$#-1}") # All args except last +else + PIPELINES=("$@") +fi + +# Default pipeline if none specified +if [ "${#PIPELINES[@]}" -eq 0 ]; then + PIPELINES=("scan_codebase") +fi + +# Run the scan +docker run --rm \ + -v "$SCAN_DIR":/codebase \ + "$SCIO_DOCKER_IMAGE" \ + run "${PIPELINES[@]}" /codebase \ + > "$RESULTS_LOCATION" + +ABS_RESULTS_PATH="$(pwd)/$RESULTS_LOCATION" + +echo "✅ Scan complete using pipeline(s): ${PIPELINES[*]}" +echo "💾 Results saved to: $ABS_RESULTS_PATH" diff --git a/scanpipe/migrations/0031_scancode_toolkit_v32_data_updates.py b/scanpipe/migrations/0031_scancode_toolkit_v32_data_updates.py index 4982ce6f7a..3d05db73c2 100644 --- a/scanpipe/migrations/0031_scancode_toolkit_v32_data_updates.py +++ b/scanpipe/migrations/0031_scancode_toolkit_v32_data_updates.py @@ -25,7 +25,7 @@ def compute_package_declared_license_expression_spdx(apps, schema_editor): ).only("declared_license_expression") object_count = queryset.count() - logger.info(f"Compute declared_license_expression_spdx for {object_count:,} packages.") + logger.debug(f"Compute declared_license_expression_spdx for {object_count:,} packages.") chunk_size = 2000 iterator = queryset.iterator(chunk_size=chunk_size) @@ -37,9 +37,9 @@ def compute_package_declared_license_expression_spdx(apps, schema_editor): unsaved_objects.append(package) if not (index % chunk_size) and unsaved_objects: - logger.info(f" {index:,} / {object_count:,} computed") + logger.debug(f" {index:,} / {object_count:,} computed") - logger.info("Updating DB objects...") + logger.debug("Updating DB objects...") DiscoveredPackage.objects.bulk_update( objs=unsaved_objects, fields=["declared_license_expression_spdx"], @@ -65,7 +65,7 @@ def compute_resource_detected_license_expression(apps, schema_editor): ) object_count = queryset.count() - logger.info(f"Compute detected_license_expression for {object_count:,} resources.") + logger.debug(f"Compute detected_license_expression for {object_count:,} resources.") chunk_size = 2000 iterator = queryset.iterator(chunk_size=chunk_size) @@ -92,9 +92,9 @@ def compute_resource_detected_license_expression(apps, schema_editor): unsaved_objects.append(resource) if not (index % chunk_size) and unsaved_objects: - logger.info(f" {index:,} / {object_count:,} computed") + logger.debug(f" {index:,} / {object_count:,} computed") - logger.info("Updating DB objects...") + logger.debug("Updating DB objects...") CodebaseResource.objects.bulk_update( objs=unsaved_objects, fields=[ @@ -168,7 +168,7 @@ def compute_resource_license_detections(apps, schema_editor): queryset = CodebaseResource.objects.filter(~Q(licenses=[])).only("licenses") object_count = queryset.count() - logger.info(f"Compute license_detections for {object_count:,} resources.") + logger.debug(f"Compute license_detections for {object_count:,} resources.") chunk_size = 2000 iterator = queryset.iterator(chunk_size=chunk_size) @@ -180,9 +180,9 @@ def compute_resource_license_detections(apps, schema_editor): unsaved_objects.append(resource) if not (index % chunk_size): - logger.info(f" {index:,} / {object_count:,} computed") + logger.debug(f" {index:,} / {object_count:,} computed") - logger.info("Updating DB objects...") + logger.debug("Updating DB objects...") # Keeping the batch_size small as the `license_detections` content is often large, # and it may raise `django.db.utils.OperationalError: out of memory` CodebaseResource.objects.bulk_update( diff --git a/scanpipe/migrations/0055_discoveredpackage_datafile_paths.py b/scanpipe/migrations/0055_discoveredpackage_datafile_paths.py index c2fe264042..e4d55181e5 100644 --- a/scanpipe/migrations/0055_discoveredpackage_datafile_paths.py +++ b/scanpipe/migrations/0055_discoveredpackage_datafile_paths.py @@ -18,7 +18,7 @@ def update_package_datasource_ids(apps, schema_editor): object_count = queryset.count() if object_count: - logger.info(f"Compute datasource_ids for {object_count:,} packages.") + logger.debug(f"Compute datasource_ids for {object_count:,} packages.") chunk_size = 2000 iterator = queryset.iterator(chunk_size=chunk_size) @@ -30,7 +30,7 @@ def update_package_datasource_ids(apps, schema_editor): unsaved_objects.append(package) if not (index % chunk_size) and unsaved_objects: - logger.info(f" {index:,} / {object_count:,} computed") + logger.debug(f" {index:,} / {object_count:,} computed") DiscoveredPackage.objects.bulk_update( objs=unsaved_objects,