Skip to content

Commit d8cfbc0

Browse files
committed
Merge branch 'development/9.2' into tmp/octopus/w/9.2/feature/CLDSRV-813/optional-attributes-response
2 parents 1d3ef8a + f5e7401 commit d8cfbc0

File tree

2 files changed

+166
-0
lines changed

2 files changed

+166
-0
lines changed

.dockerignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@ localMetadata/*
77
.tox
88
coverage
99
.DS_Store
10+
CLAUDE.md

CLAUDE.md

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with
4+
code in this repository.
5+
6+
## Project Overview
7+
8+
CloudServer is Scality's open-source, AWS S3-compatible object storage server.
9+
It provides a single S3 API interface to access multiple storage backends
10+
(Scality RING, file, memory, AWS S3, Azure, GCP, etc.). Part of the
11+
S3C/Federation and Zenko stacks.
12+
13+
- **Node.js >= 22** required
14+
- **Package Manager**: Yarn with `--frozen-lockfile`
15+
16+
## Common Commands
17+
18+
### Development
19+
20+
```bash
21+
yarn install --frozen-lockfile # Install dependencies
22+
yarn start # Start S3 + metadata + data servers
23+
yarn dev # Development mode with auto-restart (nodemon)
24+
yarn mem_backend # Start with in-memory backend (S3BACKEND=mem)
25+
yarn start_mongo # Start with MongoDB metadata backend
26+
```
27+
28+
### Testing
29+
30+
```bash
31+
# Unit tests
32+
yarn test # Run unit tests (S3BACKEND=mem)
33+
yarn cover test # Run unit tests with coverage
34+
35+
# Functional tests (require running server)
36+
yarn ft_awssdk # AWS SDK functional tests
37+
yarn ft_awssdk_buckets # Bucket-specific AWS SDK tests
38+
yarn ft_awssdk_versioning # Versioning tests
39+
yarn ft_s3cmd # S3cmd CLI tests
40+
yarn ft_healthchecks # Health check tests
41+
yarn ft_test # Run all functional tests
42+
43+
# Multi-backend tests
44+
yarn multiple_backend_test # Multi-backend unit tests
45+
yarn ft_awssdk_external_backends # External backend functional tests
46+
```
47+
48+
### Linting
49+
50+
```bash
51+
yarn lint # ESLint on all JS files
52+
yarn lint_md # Markdown linting
53+
```
54+
55+
## Architecture
56+
57+
### Entry Points
58+
59+
- `index.js` → Main S3 API server (port 8000)
60+
- `mdserver.js` → Metadata server (port 9990)
61+
- `dataserver.js` → Data storage server (port 9991)
62+
- `pfsserver.js` → Passthrough filesystem server (port 9992)
63+
- `managementAgent.js` → WebSocket management agent (port 8010)
64+
65+
### Core Components
66+
67+
```
68+
lib/
69+
├── server.js # S3Server class - main HTTP server with cluster support
70+
├── Config.js # Configuration management (env vars, config.json)
71+
├── api/ # S3 API implementations (80+ operations)
72+
│ ├── api.js # Main request router and auth handler
73+
│ └── apiUtils/ # Shared utilities (auth, bucket ops, object ops, quotas)
74+
├── auth/ # Authentication (vault.js for IAM, in_memory/ for local)
75+
├── data/ # Data backend abstraction (file, memory, multiple)
76+
├── metadata/ # Metadata backend abstraction
77+
├── routes/ # Special routes (Backbeat, metadata service, Veeam)
78+
├── kms/ # Key management (file, memory, AWS, KMIP)
79+
├── management/ # Management API and configuration
80+
└── utilities/ # Logging, health checks, XML parsing
81+
```
82+
83+
### Backend Configuration
84+
85+
Set via environment variables. Each backend type abstracts multiple
86+
implementations:
87+
88+
### Data Backends (`S3DATA`)
89+
90+
| Backend | Port | Description |
91+
|---------|------|-------------|
92+
| `file` | 9991 | Local filesystem via `dataserver.js` |
93+
| `multiple` | - | Multi-backend gateway (AWS S3, Azure, GCP, Sproxyd) |
94+
| `mem` | - | In-memory (testing only) |
95+
96+
`scality` is an alias for `multiple`. With `multiple`, objects route to
97+
backends defined in `locationConfig.json` based on location constraints.
98+
99+
### Metadata Backends (`S3METADATA`)
100+
101+
| Backend | Port | Description |
102+
|---------|------|-------------|
103+
| `file` (default) | 9990 | Local LevelDB via `mdserver.js` |
104+
| `scality` | 9000 | External bucketd service (production Scality RING) |
105+
| `mongodb` | 27017+ | MongoDB replica set |
106+
| `mem` | - | In-memory (testing only) |
107+
108+
**file vs scality**: The `file` backend runs a self-contained metadata server
109+
(`mdserver.js`) for development. The `scality` backend connects to external
110+
**bucketd** which uses Raft-based distributed metadata (repd) for production
111+
HA deployments.
112+
113+
### Auth Backends (`S3VAULT`)
114+
115+
| Backend | Port | Description |
116+
|---------|------|-------------|
117+
| `mem` (default) | - | In-memory accounts from `conf/authdata.json` |
118+
| `vault` | 8500 | External Vault IAM service (vaultd) |
119+
120+
### KMS Backends (`S3KMS`)
121+
122+
| Backend | Description |
123+
|---------|-------------|
124+
| `file` (default) | Local file-based key storage |
125+
| `mem` | In-memory (testing only) |
126+
| `kmip` | External KMIP server |
127+
| `aws` | AWS KMS |
128+
| `scality` | Scality KMS |
129+
130+
### Path Configuration
131+
132+
- `S3DATAPATH`: Data storage directory (default: `./localData`)
133+
- `S3METADATAPATH`: Metadata storage directory (default: `./localMetadata`)
134+
135+
### Key Files
136+
137+
- `config.json` - Default configuration
138+
- `constants.js` - Global constants (splitters, limits, service accounts)
139+
- `locationConfig.json` - Location constraints for multi-backend
140+
141+
## Testing Conventions
142+
143+
- **Framework**: Mocha with Sinon for mocking
144+
- **Timeout**: 40000ms global
145+
- **Coverage**: NYC (Istanbul), 80% patch target
146+
- **Reporters**: mocha-multi-reporters (spec + junit XML)
147+
148+
Run a single test file:
149+
150+
```bash
151+
S3BACKEND=mem yarn mocha tests/unit/path/to/test.js --exit
152+
```
153+
154+
Run tests matching a pattern:
155+
156+
```bash
157+
S3BACKEND=mem yarn mocha --grep "pattern" tests/unit --recursive --exit
158+
```
159+
160+
## Code Style
161+
162+
- ESLint with `@scality/eslint-config-scality`
163+
- `mocha/no-exclusive-tests: error` - Never commit `.only()` tests
164+
- Many formatting rules disabled for legacy compatibility
165+
- ECMAScript 2021, CommonJS modules (`sourceType: "script"`)

0 commit comments

Comments
 (0)