docs(python): improve Python SDK documentation (#931)

* main readme

* cli readme

* after self-review

Author: Iskander14yo

python/README.md

@@ -1,108 +1,135 @@
 # GraphAr Python SDK
 
-GraphAr Python SDK provides Python bindings for the GraphAr C++ library, allowing users to work with GraphAr formatted graph data in Python environments. It includes both a high-level API for data manipulation and a command-line interface for common operations.
+The GraphAr Python SDK provides Python bindings for the GraphAr C++ library.
+It lets Python applications read GraphAr metadata, use the high-level graph APIs,
+and run the bundled `graphar` command-line tool.
 
-## Installation
+This package is separate from the PySpark package in [`../pyspark`](../pyspark).
 
-### Prerequisites
+## Requirements
 
 - Python >= 3.9
-- pip (latest version recommended)
-- CMake >= 3.15 (for building from source)
-- Apache Arrow >= 12.0 (for building from source)
+- pip
+- CMake >= 3.15, Apache Arrow >= 12.0, and a C++ toolchain when building from source
 
-### Install from Pypi
-Install the latest released version from PyPI:
+## Install
+
+### From PyPI
 
 ```bash
 pip install -U graphar
 ```
 
-### Install from Source
-
-Clone the repository and install the Python package:
+Verify the installation:
 
 ```bash
-git clone https://github.com/apache/incubator-graphar.git
-cd incubator-graphar
-pip install ./python
+python -c "import graphar; print(graphar.GraphInfo)"
+graphar --help
 ```
 
-For verbose output during installation:
+### From Source
+
+Clone the repository, then from its root:
 
 ```bash
-pip install -v ./python
+pip install ./python
+
+# for local development
+pip install -e ./python
 ```
 
-### Using Docker (Recommended)
+### Docker
 
-The easiest way to get started is by using our pre-configured Docker environment:
+The project also publishes a development image:
 
 ```bash
 docker run -it ghcr.io/apache/graphar-dev
 ```
 
 ## Quick Start
 
-### Importing the Package
-
-After installation, you can import the GraphAr Python SDK in your Python scripts:
+Load graph metadata from a GraphAr YAML file:
 
 ```python
 import graphar
+
+graph_info = graphar.GraphInfo.load("path/to/graph.graph.yml")
+
+print(graph_info.get_name())
+print(graph_info.get_vertex_info("person").get_type())
+print(graph_info.get_edge_info("person", "knows", "person").get_edge_type())
 ```
 
-### Basic Usage
+Replace `path/to/graph.graph.yml` with the path to a GraphAr graph metadata file.
 
-Loading graph information:
+## Modules
 
-```python
-import graphar
+The Python SDK exposes the core GraphAr functionality through these modules:
+
+- [`graphar.graph_info`](src/graphar/graph_info.py): graph, vertex, edge, property, and metadata APIs.
+- [`graphar.high_level`](src/graphar/high_level.py): high-level vertex and edge collection APIs.
+- [`graphar.types`](src/graphar/types.py): GraphAr enum types used by metadata and high-level APIs.
+
+## Examples
+
+Example scripts are available in [`python/example`](example):
 
-# Load graph info from a YAML file
-graph_info = graphar.graph_info.GraphInfo.load("path/to/graph.yaml")
+- [`graph_info_example.py`](example/graph_info_example.py) shows how to load graph metadata and inspect vertex and edge information.
+- [`high_level_example.py`](example/high_level_example.py) shows how to use the high-level vertex and edge collection APIs.
 
-# Access vertex information
-vertex_info = graph_info.get_vertex_info("person")
-print(f"Vertex type: {vertex_info.get_type()}")
+The examples expect `GAR_TEST_DATA` to point to a directory that contains the
+`ldbc_sample/parquet/ldbc_sample.graph.yml` test graph:
 
-# Access edge information
-edge_info = graph_info.get_edge_info("person", "knows", "person")
-print(f"Edge type: {edge_info.get_edge_type()}")
+```bash
+bash dev/download_test_data.sh
+export GAR_TEST_DATA=/tmp/graphar-testing
+python python/example/graph_info_example.py
+python python/example/high_level_example.py
 ```
 
 ## Command-Line Interface
 
-GraphAr Python SDK also provides a command-line interface for common operations such as checking metadata, showing graph information, and importing data.
+The package installs a `graphar` command-line tool:
+
+```bash
+graphar --help
+graphar show --path path/to/graph.graph.yml
+graphar check --path path/to/graph.graph.yml
+```
 
-For detailed information about the CLI functionality, please see [CLI Documentation](src/cli/README.md).
+See [`python/src/cli/README.md`](src/cli/README.md) for more CLI examples.
 
 ## API Documentation
 
-### build docs
+Build the Python API documentation from the `python` directory:
+
 ```bash
 make install_docs
 make docs
 ```
 
-The Python SDK exposes the core GraphAr functionality through several modules:
+The generated documentation is written to `python/docs`.
 
-- `graphar.graph_info`: Main API for working with graph, vertex, and edge information
-- `graphar.high_level`: High-level API for data reading and writing
+## Development
 
-## Examples
-> [!NOTE]
-> under development.
+Install test dependencies from the repository root:
 
-You can find various examples in the [examples directory](../cpp/examples/) which demonstrate usage of the underlying C++ library. These concepts translate directly to the Python SDK.
+```bash
+pip install -e "./python[test]"
+```
 
-## Development
+Run the Python tests from the `python` directory:
+
+```bash
+pytest
+```
+
+Some tests require `GAR_TEST_DATA`; use [`dev/download_test_data.sh`](../dev/download_test_data.sh)
+if the test data is not available locally.
 
-To contribute to the Python SDK, please follow the guidelines in the main [CONTRIBUTING.md](../CONTRIBUTING.md) file.
+For general contribution guidelines, see [`../CONTRIBUTING.md`](../CONTRIBUTING.md).
 
 ## License
 
-**GraphAr** is distributed under [Apache License
-2.0](https://github.com/apache/incubator-graphar/blob/main/LICENSE).
-Please note that third-party libraries may not have the same license as
-GraphAr.
+GraphAr is distributed under the Apache License 2.0. See [`../LICENSE`](../LICENSE)
+and [`../NOTICE`](../NOTICE) for details.

python/src/cli/README.md

@@ -1,96 +1,99 @@
 # GraphAr Python CLI
 
-GraphAr python cli uses [pybind11][] and [scikit-build-core][] to bind C++ code into Python and build command line tools through Python. Command line tools developed using [typer][].
+The GraphAr Python package installs a `graphar` command-line tool for inspecting
+GraphAr metadata and importing data into GraphAr format.
 
-[pybind11]: https://pybind11.readthedocs.io
-[scikit-build-core]: https://scikit-build-core.readthedocs.io
-[typer]: https://typer.tiangolo.com/
-
-## Requirements
-
-- Linux (work fine on Ubuntu 22.04)
-- Cmake >= 3.15
-- Arrow >= 12.0
-- Python >= 3.7
-- pip == latest
+The CLI is implemented with [Typer][] and uses the same Python bindings as the
+[`graphar` Python package](../../README.md).
 
+[Typer]: https://typer.tiangolo.com/
 
-The best testing environment is `ghcr.io/apache/graphar-dev` Docker environment.
+## Requirements
 
-And using Python in conda or venv is a good choice. 
+- Python >= 3.9
+- pip
+- CMake >= 3.15, Apache Arrow >= 12.0, and a C++ toolchain when building from source
 
 ## Installation
 
-### Install from Pypi
 Install the latest released version from PyPI:
 
 ```bash
 pip install -U graphar
 ```
 
-### Install from Source
+Or install from the repository root:
 
-- Clone this repository
-- `pip install ./python` or set verbose level `pip install -v ./python`
+```bash
+pip install ./python
+```
 
-## Usage
+Verify the CLI is available:
 
 ```bash
 graphar --help
-
-# check the metadata, verify whether the vertex edge information and attribute information of the graph are valid
-graphar check -p ../testing/neo4j/MovieGraph.graph.yml
-
-# show the vertex
-graphar show -p ../testing/neo4j/MovieGraph.graph.yml -v Person
-
-# show the edge
-graphar show -p ../testing/neo4j/MovieGraph.graph.yml -es Person -e ACTED_IN -ed Movie
-
-# import graph data by using a config file
-graphar import -c ../testing/neo4j/data/import.mini.yml
 ```
 
-## Import config file
+## Usage
 
-The config file supports `yaml` data type. We provide two reference templates for it: full and mini.
+Replace the paths below with paths to your GraphAr metadata or import config
+files.
 
-The full version of the configuration file contains all configurable fields, and additional fields will be automatically ignored.
+```bash
+# Show all graph metadata.
+graphar show --path path/to/graph.graph.yml
 
-The mini version of the configuration file is a simplified version of the full configuration file, retaining the same functionality. It shows the essential parts of the configuration information. 
+# Validate graph metadata.
+graphar check --path path/to/graph.graph.yml
 
-For the full configuration file, if all fields can be set to their default values, you can simplify it to the mini version. However, it cannot be further reduced beyond the mini version.
+# Show one vertex type.
+graphar show --path path/to/graph.graph.yml --vertex Person
 
-In the full `yaml` config file, we provide brief comments on the fields, which can be used as a reference.
+# Show one edge type.
+graphar show \
+  --path path/to/graph.graph.yml \
+  --edge-src Person \
+  --edge ACTED_IN \
+  --edge-dst Movie
 
-**Example**
+# Import data with a config file.
+graphar import --config path/to/import.yml
+```
 
-To import the movie graph data from the `testing` directory, you first need to prepare data files. Supported file types include `csv`, `json`(as well as`jsonline`, but should have the `.json` extension), `parquet`, and `orc` files. Please ensure the correct file extensions are set in advance, or specify the `file_type` field in the source section of the configuration. The `file_type` field will ignore the file extension.
+Short options are also available:
 
-Next, write a configuration file following the provided sample. Any empty fields in the `graphar` configuration will be filled with default values. In the `import_schema`, empty fields will use the global configuration values from `graphar`. If fields in `import_schema` are not empty, they will override the values from `graphar`.
+```bash
+graphar show -p path/to/graph.graph.yml -v Person
+graphar show -p path/to/graph.graph.yml -es Person -e ACTED_IN -ed Movie
+graphar import -c path/to/import.yml
+```
 
-A few important notes:
+## Import Config
 
-1. The sources list specifies configuration for the data source files. For `csv` files, you can set the `delimiter`. The format of the `json` file should be given in the format of `jsonline`.
+The import command reads a YAML config file. A config describes source files,
+GraphAr output settings, and how source columns map to vertex or edge
+properties.
 
-2. The columns dictionary maps column names in the data source to node or edge properties. Keys represent column names in the data source, and values represent property names.
+Supported source file types are `csv`, `json`, `parquet`, and `orc`. JSON input
+uses JSON Lines format and should use the `.json` extension. You can override
+extension-based detection by setting `file_type` in the source config.
 
-3. Currently, edge properties cannot have the same names as the edge endpoints' properties; doing so will raise an exception.
+Important fields:
 
-4. The following table lists the default fields, more of which are included in the full configuration.
+1. `sources` describes the input files. CSV sources can set a `delimiter`.
+2. `columns` maps source column names to GraphAr property names.
+3. Edge property names must not duplicate endpoint property names.
+4. Empty fields in `import_schema` use values from the top-level `graphar`
+   config. Explicit `import_schema` values override the top-level defaults.
 
+Common defaults:
 
 | Field                             | Default value        |
-| -----------                       | -----------          |
-|  `graphar.vertex_chunk_size`      | `100`                |
-|  `graphar.edge_chunk_size`        | `1024`               |
-|  `graphar.file_type`              | `parquet`            |
-|  `graphar.adj_list_type`          | `ordered_by_source`  |
-|  `graphar.validate_level`         | `weak`               |
-|  `graphar.version`                | `gar/v1`             |
-|  `property.nullable`              | `true`               |
-
-
-
-
-Wish you a happy use！
+| --------------------------------- | -------------------- |
+| `graphar.vertex_chunk_size`       | `100`                |
+| `graphar.edge_chunk_size`         | `1024`               |
+| `graphar.file_type`               | `parquet`            |
+| `graphar.adj_list_type`           | `ordered_by_source`  |
+| `graphar.validate_level`          | `weak`               |
+| `graphar.version`                 | `gar/v1`             |
+| `property.nullable`               | `true`               |