Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,18 @@ you may use the Docker image to run pipelines as a single command:
.. code-block:: bash

docker run --rm \
-v "$(pwd)":/codedrop \
ghcr.io/aboutcode-org/scancode.io:latest \
sh -c "run scan_codebase /codedrop" \
-v "$(pwd)":/codedrop \
run scan_codebase /codedrop \
> results.json

Refer to the :ref:`cli_run` section for more about this approach.

.. tip::
Prefer a one-liner? Use this to scan your current directory::

curl -sSL https://raw.githubusercontent.com/aboutcode-org/scancode.io/main/etc/scripts/run-scan.sh | bash

.. _faq_which_pipeline:

Which pipeline should I use?
Expand Down
23 changes: 13 additions & 10 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
ScanCode.io documentation
ScanCode.io Documentation
=========================

Welcome to the very start of your ScanCode.io journey! In this documentation
you’ll find information on:
Welcome! This is your starting point for everything ScanCode.io.

- An overview of ScanCode.io and ScanPipe
- Installation instructions
- Tutorials to get you started
- Reference documentation about the ScanPipe concepts, Pipelines, Pipes and more
- How to make technical contributions to the project and the community
In this documentation, you’ll find:

- A **QuickStart guide** to run your first scan
- An **overview** of what ScanCode.io is and what it can do
- **Installation instructions** for full access
- Step-by-step **tutorials** to dive deeper
- In-depth **reference docs** on concepts, pipelines, configuration, and APIs
- Guides on how to **contribute** to the project and community

.. toctree::
:maxdepth: 2
:caption: Getting Started

quickstart
introduction
installation
user-interface
Expand All @@ -23,7 +26,7 @@ you’ll find information on:

.. toctree::
:maxdepth: 2
:caption: Tutorial
:caption: Tutorials

tutorial_web_ui_analyze_docker_image
tutorial_web_ui_review_scan_results
Expand All @@ -37,7 +40,7 @@ you’ll find information on:

.. toctree::
:maxdepth: 2
:caption: Reference Documentation
:caption: Reference

scanpipe-concepts
built-in-pipelines
Expand Down
6 changes: 3 additions & 3 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -221,8 +221,8 @@ And visit the web UI at: http://localhost/project/

.. _local_development_installation:

Local development installation
------------------------------
Local development
-----------------

Supported Platforms
^^^^^^^^^^^^^^^^^^^
Expand All @@ -247,7 +247,7 @@ Before you install ScanCode.io, make sure you have the following prerequisites:

* **Python: versions 3.10 to 3.13** found at https://www.python.org/downloads/
* **Git**: most recent release available at https://git-scm.com/
* **PostgreSQL**: release 11 or later found at https://www.postgresql.org/ or
* **PostgreSQL**: release 13 or later found at https://www.postgresql.org/ or
https://postgresapp.com/ on macOS

.. _system_dependencies:
Expand Down
5 changes: 0 additions & 5 deletions docs/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,3 @@ images, containers, root filesystems, and virtual machine images.
As a common practice, ScanCode.io releases usually follow ScanCode Toolkit releases
to ensure the latest improvements of the scanning engines are included in the
latest release of ScanCode.io.


.. Some of this documentation is borrowed from the metaflow documentation and is also
under Apache-2.0
.. Copyright (c) Netflix
107 changes: 107 additions & 0 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
.. _quickstart:

QuickStart
==========

Run a Scan (no installation required!)
--------------------------------------

The **fastest way** to get started and **scan a codebase** —
**no installation needed** — is by using the latest
**ScanCode.io Docker image**.

.. warning::
**Docker must be installed on your system**.
Visit the `Docker documentation <https://docs.docker.com/get-docker/>`_ to install
it for your platform.

To run the :ref:`pipeline_scan_codebase` pipeline on a **local directory**
with a **single command**:

.. code-block:: bash

docker run --rm \
-v "$(pwd)":/codedrop \
ghcr.io/aboutcode-org/scancode.io:latest \
run scan_codebase /codedrop \
> results.json

Let's unpack what each part of the command does:

- ``docker run --rm``
Runs a temporary Docker container that is automatically removed after it finishes.

- ``-v "$(pwd)":/codedrop``
Mounts your current directory into the container at ``/codedrop`` so it can be
scanned.

- ``ghcr.io/aboutcode-org/scancode.io:latest``
Uses the latest ScanCode.io image from GitHub Container Registry.

- ``run scan_codebase /codedrop``
Runs the ``scan_codebase`` pipeline inside the container, using the mounted directory
as the input source.

- ``> results.json``
Saves the scan output to a ``results.json`` file on your machine.

The result? A **full scan of your local directory — no setup, one command!**

See the :ref:`RUN command <cli_run>` section for more details on this command.

.. note::
Not sure which pipeline to use? Check out :ref:`faq_which_pipeline`.

Next Step: Local Installation
-----------------------------

Install ScanCode.io, to **unlock all features**:

- **User Interface:** Explore dashboards, codebase data, charts, and scan results.
See :ref:`user_interface`.
- **Project Management:** Create, filter, and monitor projects.
- **REST API:** Automate your scans with the :ref:`rest_api`.
- **CLI:** Use the :ref:`command_line_interface` to work from the terminal.
- **Webhooks:** Get real-time updates via custom integrations. See :ref:`webhooks`.
- **Slack Notifications:** Send project updates to Slack. Follow setup in
:ref:`webhooks_slack_notifications`.

See the :ref:`installation` chapter for the full list of installation options.

Integrate with Your Workflows
-----------------------------

ScanCode.io can be part of your CI/CD workflow.

GitHub Actions
^^^^^^^^^^^^^^

Use the official `scancode-action <https://github.com/aboutcode-org/scancode-action>`_
to integrate **ScanCode.io into your GitHub workflows** with ease.

This action lets you:

- **Run pipelines**
- **Check for compliance issues**
- **Detect vulnerabilities**
- **Generate SBOMs and scan results**

Example usage:

.. code-block:: yaml

runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: scancode-inputs
- uses: aboutcode-org/scancode-action@main
with:
pipelines: "scan_codebase"
output-formats: "json xlsx spdx cyclonedx"

Full details available at:
https://github.com/aboutcode-org/scancode-action

.. tip::
Learn more about automation options in the :ref:`automation` section.
59 changes: 59 additions & 0 deletions etc/scripts/run-scan.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/bash

# SPDX-License-Identifier: Apache-2.0
#
# http://nexb.com and https://github.com/aboutcode-org/scancode.io
# The ScanCode.io software is licensed under the Apache License version 2.0.
# Data generated with ScanCode.io is provided as-is without warranties.
# ScanCode is a trademark of nexB Inc.

# Usage:
# Default scan of current directory using 'scan_codebase':
# ./run-scan.sh
#
# Run pipelines on current directory:
# ./run-scan.sh scan_codebase find_vulnerabilities
#
# Run pipelines on a specific directory:
# ./run-scan.sh scan_codebase /path/to/scan/dir
#
# Run multiple pipelines on a specific directory (directory must come last):
# ./run-scan.sh scan_codebase find_vulnerabilities /my/codebase
#
# Scan a directory using default pipeline:
# ./run-scan.sh /path/to/codebase

set -e

readonly SCIO_DOCKER_IMAGE="ghcr.io/aboutcode-org/scancode.io:latest"
readonly RESULTS_LOCATION="results.json"

# Default values
SCAN_DIR="$(pwd)"
PIPELINES=()

# Check if last argument is a directory
LAST_ARG="${!#}"
if [ -d "$LAST_ARG" ]; then
SCAN_DIR="$LAST_ARG"
PIPELINES=("${@:1:$#-1}") # All args except last
else
PIPELINES=("$@")
fi

# Default pipeline if none specified
if [ "${#PIPELINES[@]}" -eq 0 ]; then
PIPELINES=("scan_codebase")
fi

# Run the scan
docker run --rm \
-v "$SCAN_DIR":/codebase \
"$SCIO_DOCKER_IMAGE" \
run "${PIPELINES[@]}" /codebase \
> "$RESULTS_LOCATION"

ABS_RESULTS_PATH="$(pwd)/$RESULTS_LOCATION"

echo "✅ Scan complete using pipeline(s): ${PIPELINES[*]}"
echo "💾 Results saved to: $ABS_RESULTS_PATH"
18 changes: 9 additions & 9 deletions scanpipe/migrations/0031_scancode_toolkit_v32_data_updates.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def compute_package_declared_license_expression_spdx(apps, schema_editor):
).only("declared_license_expression")

object_count = queryset.count()
logger.info(f"Compute declared_license_expression_spdx for {object_count:,} packages.")
logger.debug(f"Compute declared_license_expression_spdx for {object_count:,} packages.")

chunk_size = 2000
iterator = queryset.iterator(chunk_size=chunk_size)
Expand All @@ -37,9 +37,9 @@ def compute_package_declared_license_expression_spdx(apps, schema_editor):
unsaved_objects.append(package)

if not (index % chunk_size) and unsaved_objects:
logger.info(f" {index:,} / {object_count:,} computed")
logger.debug(f" {index:,} / {object_count:,} computed")

logger.info("Updating DB objects...")
logger.debug("Updating DB objects...")
DiscoveredPackage.objects.bulk_update(
objs=unsaved_objects,
fields=["declared_license_expression_spdx"],
Expand All @@ -65,7 +65,7 @@ def compute_resource_detected_license_expression(apps, schema_editor):
)

object_count = queryset.count()
logger.info(f"Compute detected_license_expression for {object_count:,} resources.")
logger.debug(f"Compute detected_license_expression for {object_count:,} resources.")

chunk_size = 2000
iterator = queryset.iterator(chunk_size=chunk_size)
Expand All @@ -92,9 +92,9 @@ def compute_resource_detected_license_expression(apps, schema_editor):
unsaved_objects.append(resource)

if not (index % chunk_size) and unsaved_objects:
logger.info(f" {index:,} / {object_count:,} computed")
logger.debug(f" {index:,} / {object_count:,} computed")

logger.info("Updating DB objects...")
logger.debug("Updating DB objects...")
CodebaseResource.objects.bulk_update(
objs=unsaved_objects,
fields=[
Expand Down Expand Up @@ -168,7 +168,7 @@ def compute_resource_license_detections(apps, schema_editor):
queryset = CodebaseResource.objects.filter(~Q(licenses=[])).only("licenses")

object_count = queryset.count()
logger.info(f"Compute license_detections for {object_count:,} resources.")
logger.debug(f"Compute license_detections for {object_count:,} resources.")

chunk_size = 2000
iterator = queryset.iterator(chunk_size=chunk_size)
Expand All @@ -180,9 +180,9 @@ def compute_resource_license_detections(apps, schema_editor):
unsaved_objects.append(resource)

if not (index % chunk_size):
logger.info(f" {index:,} / {object_count:,} computed")
logger.debug(f" {index:,} / {object_count:,} computed")

logger.info("Updating DB objects...")
logger.debug("Updating DB objects...")
# Keeping the batch_size small as the `license_detections` content is often large,
# and it may raise `django.db.utils.OperationalError: out of memory`
CodebaseResource.objects.bulk_update(
Expand Down
4 changes: 2 additions & 2 deletions scanpipe/migrations/0055_discoveredpackage_datafile_paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ def update_package_datasource_ids(apps, schema_editor):

object_count = queryset.count()
if object_count:
logger.info(f"Compute datasource_ids for {object_count:,} packages.")
logger.debug(f"Compute datasource_ids for {object_count:,} packages.")

chunk_size = 2000
iterator = queryset.iterator(chunk_size=chunk_size)
Expand All @@ -30,7 +30,7 @@ def update_package_datasource_ids(apps, schema_editor):
unsaved_objects.append(package)

if not (index % chunk_size) and unsaved_objects:
logger.info(f" {index:,} / {object_count:,} computed")
logger.debug(f" {index:,} / {object_count:,} computed")

DiscoveredPackage.objects.bulk_update(
objs=unsaved_objects,
Expand Down