Skip to content

Commit 7435fd3

Browse files
docs(Echo): Expand README with comprehensive getting started guide
Expand the Echo README from a concise (~150 line) overview to a full developer onboarding guide with installation instructions, usage examples, and contribution pathways. Key additions include: - New "Getting Started" section with Cargo.toml installation snippet and three-step usage walkthrough (initialize scheduler, submit tasks, graceful shutdown) - Detailed "Key Features" list covering work-stealing, task prioritization, builder API, and graceful shutdown - "Core Architecture Principles" table mapping Performance, Structured Concurrency, Decoupling, Resilience, and Composability to their key components - "Echo in the Land Ecosystem" section with Mermaid diagram showing integration with Mountain and ActionEffect pattern - Contribution call-to-action with four-tier task matrix (Quick Wins, Architectural, Expert Tuning, Advanced Logic) - Expanded license and changelog reference sections This addresses the developer friction identified in recent README iterations by providing the concrete setup and usage code that was trimmed in the initial benefit-focused rewrite.
1 parent 0f3840b commit 7435fd3

File tree

1 file changed

+168
-20
lines changed

1 file changed

+168
-20
lines changed

README.md

Lines changed: 168 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -37,36 +37,94 @@
3737
</tr>
3838
</table>
3939

40-
4140
---
4241

4342
# **Echo**&#x2001;📣
4443

45-
> **VS Code's background tasks (file indexing, symbol scanning, git blame) run in a single-threaded Node.js process. Heavy indexing freezes everything on that thread. The only escape is spawning more processes, adding memory and IPC overhead.**
44+
A Resilient, High-Performance Task Scheduler for Rust
45+
46+
> **VS Code's background tasks (file indexing, symbol scanning, git blame) run
47+
> in a single-threaded Node.js process. Heavy indexing freezes everything on
48+
> that thread. The only escape is spawning more processes, adding memory and IPC
49+
> overhead.**
4650
47-
_"Indexing, search, and builds run on every CPU core in parallel. The editor stays responsive."_
51+
_"Indexing, search, and builds run on every CPU core in parallel. The editor
52+
stays responsive."_
4853

4954
[![License: CC0-1.0](https://img.shields.io/badge/License-CC0_1.0-lightgrey.svg)](https://github.com/CodeEditorLand/Echo/tree/Current/LICENSE)
5055
[<img src="https://editor.land/Image/Rust.svg" width="14" alt="Rust" />](https://www.rust-lang.org/)&#x2001;[![Crates.io](https://img.shields.io/crates/v/Echo.svg)](https://crates.io/crates/Echo)
56+
[<img src="https://editor.land/Image/Rust.svg" width="14" alt="Rust" />](https://www.rust-lang.org/)&#x2001;[![Rust Version](https://img.shields.io/badge/Rust-1.75+-orange.svg)](https://www.rust-lang.org/)
5157
[![Tokio Version](https://img.shields.io/badge/Tokio-v1-blue.svg)](https://tokio.rs/)
5258
[![Crossbeam Version](https://img.shields.io/badge/Crossbeam-blueviolet.svg)](https://github.com/crossbeam-rs/crossbeam)
5359

54-
Echo is a lock-free work-stealing scheduler built on Rust's `crossbeam-deque`. Every task runs in a supervised worker pool spanning all CPU cores. Tasks are work-stolen (idle threads pick up tasks from busy threads' queues), supervised (every task has a parent scope with controlled crash propagation), and gracefully shut down (no task can outlive its scope).
55-
5660
📖 **[Rust API Documentation](https://Rust.Documentation.Editor.Land/Echo/)**
5761

62+
Welcome to **Echo**! This crate provides a powerful, structured concurrency
63+
runtime for Rust applications, built on a high-performance **work-stealing
64+
scheduler**. It is designed to be the core execution engine for application
65+
backends like `Mountain`, integrating seamlessly with declarative systems like
66+
the `ActionEffect` pattern. **Echo** moves beyond simple task spawning
67+
(`tokio::spawn`) to provide a robust framework for managing, prioritizing, and
68+
executing complex asynchronous workflows with resilience and efficiency.
69+
70+
**Echo** is engineered to:
71+
72+
1. **Provide High-Performance Concurrency:** Utilizes a lock-free,
73+
work-stealing queue (`crossbeam-deque`) to ensure all worker threads remain
74+
busy, maximizing CPU utilization and application throughput.
75+
2. **Enable Structured Task Management:** Offers a clean API for submitting
76+
tasks with different priorities, allowing critical, UI-blocking operations
77+
to pre-empt background work.
78+
3. **Integrate Natively with Effect Systems:** Designed from the ground up to
79+
be the execution backend for systems like the `ActionEffect` pattern,
80+
providing a bridge between declarative task definitions and their concrete
81+
execution.
82+
83+
---
84+
85+
## Key Features&#x2001;🔐
86+
87+
- **Work-Stealing Scheduler:** Implements a modern, priority-aware work-stealing
88+
algorithm to efficiently distribute tasks across a pool of worker threads.
89+
- **Task Prioritization:** Supports submitting tasks with `High`, `Normal`, or
90+
`Low` priority, ensuring that latency-sensitive operations are handled
91+
immediately.
92+
- **Fluent Builder API:** A clean `SchedulerBuilder` allows for easy
93+
configuration of the worker pool size.
94+
- **Graceful Shutdown:** Provides a `Stop()` method to ensure all worker threads
95+
complete their current tasks and exit cleanly, preventing orphaned threads.
96+
- **Decoupled Architecture:** A generic `Queue` module provides the core
97+
work-stealing logic, which is consumed by the application-specific
98+
`Scheduler`.
99+
100+
---
101+
102+
## Core Architecture Principles&#x2001;🏗️
103+
104+
| Principle | Description | Key Components Involved |
105+
| :------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------ |
106+
| **Performance** | Use lock-free data structures (`crossbeam-deque`) and a high-performance work-stealing algorithm to achieve maximum throughput and low-latency task execution. | `Queue::StealingQueue`, `Scheduler::Worker` |
107+
| **Structured Concurrency** | Manage all asynchronous operations within a supervised pool of workers, providing graceful startup and shutdown, unlike fire-and-forget `tokio::spawn`. | `Scheduler::Scheduler`, `Scheduler::SchedulerBuilder` |
108+
| **Decoupling** | Separate the generic **Queueing Logic** from the application-specific **Scheduler Implementation**. The scheduler uses the queue to run its tasks. | `Queue::StealingQueue<TTask>`, `Scheduler::Scheduler`, `Task::Task` |
109+
| **Resilience** | The scheduler's design is inherently resilient; the failure of one task (if it panics) is contained within its `tokio` task and does not crash the worker pool. | `Scheduler::Worker::Run` |
110+
| **Composability** | Provide a simple `Submit` API that accepts any `Future<Output = ()>`, making it easy to integrate with any asynchronous Rust code. | `Task::Task`, `Scheduler::Scheduler::Submit` |
111+
58112
---
59113

60-
## What It Does&#x2001;🔐
114+
## Deep Dive & Component Breakdown&#x2001;🔬
61115

62-
- **Work-stealing across all cores.** Idle threads pick up tasks from busy threads' queues automatically.
63-
- **Supervised task scopes.** Every task has a parent scope. Crash propagation is controlled.
64-
- **Guaranteed teardown.** No task can outlive its scope. Shutdown is deterministic.
65-
- **UI never blocks.** Heavy indexing, symbol analysis, and test runners all go to Echo's thread pool.
116+
To understand how `Echo`'s internal components interact to provide these
117+
services, please refer to the detailed technical breakdown in
118+
[`Documentation/GitHub/DeepDive.md`](https://github.com/CodeEditorLand/Echo/tree/Current/Documentation/GitHub/DeepDive.md).
119+
This document explains the roles of the `Task`, `StealingQueue`, `Worker`, and
120+
`Scheduler` in detail.
66121

67122
---
68123

69-
## In the Ecosystem&#x2001;📣 + 🏞️
124+
## `Echo` in the Land Ecosystem&#x2001;📣 + 🏞️
125+
126+
This diagram illustrates `Echo`'s role as the core execution engine within the
127+
`Mountain` backend.
70128

71129
```mermaid
72130
graph LR
@@ -103,7 +161,10 @@ graph LR
103161

104162
---
105163

106-
## Project Structure&#x2001;🗺️
164+
## Project Structure Overview&#x2001;🗺️
165+
166+
The `Echo` repository is organized into a few core modules with a clear
167+
separation of concerns:
107168

108169
```
109170
Echo/
@@ -116,18 +177,87 @@ Echo/
116177

117178
---
118179

119-
## Development&#x2001;🛠️
180+
## Getting Started&#x2001;🚀
181+
182+
### Installation&#x2001;📥
183+
184+
To add `Echo` to your project, add the following to your `Cargo.toml`:
185+
186+
```toml
187+
[dependencies]
188+
Echo = { git = "https://github.com/CodeEditorLand/Echo.git", branch = "Current" }
189+
```
190+
191+
**Key Dependencies:**
120192

121-
Echo is a component of the Land workspace. Follow the
122-
[Land Repository](https://github.com/CodeEditorLand/Land) instructions to
123-
build and run.
193+
- `tokio = { version = "*", features = ["full"] }`
194+
- `crossbeam-deque = "*"`
195+
- `rand = "*"`
196+
- `log = "*"`
197+
- `num_cpus = "*"`
198+
199+
### Usage&#x2001;🚀
200+
201+
1. **Initialize the Scheduler:** Create and start the scheduler when your
202+
application starts. It is typically wrapped in an `Arc` to be shared safely
203+
across your application.
204+
205+
```rust
206+
use std::sync::Arc;
207+
use Echo::Scheduler::SchedulerBuilder;
208+
use Echo::Task::Priority;
209+
210+
let Scheduler = Arc::new(SchedulerBuilder::Create().WithWorkerCount(8).Build());
211+
```
212+
213+
2. **Submit Tasks:** Use the `Scheduler` instance to submit asynchronous work
214+
from anywhere in your application.
215+
216+
```rust
217+
let MyTask = async {
218+
println!("This is running on an Echo worker thread!");
219+
// ... perform some work ...
220+
};
221+
222+
// Submit the task with a desired priority
223+
Scheduler.Submit(MyTask, Priority::Normal);
224+
225+
// Another example with high priority
226+
Scheduler.Submit(async { /* critical work */ }, Priority::High);
227+
```
228+
229+
3. **Graceful Shutdown:** Before your application exits, ensure a clean
230+
shutdown of all worker threads.
231+
232+
```rust
233+
// Note: Arc::try_unwrap requires the Arc to have only one strong reference.
234+
if let Ok(mut Scheduler) = Arc::try_unwrap(Scheduler) {
235+
Scheduler.Stop().await;
236+
}
237+
```
124238

125239
---
126240

127-
## License&#x2001;⚖️
241+
## Help Us Boost Performance: A Call for Contributions!&#x2001;🫱🏻‍🫲🏿
242+
243+
`Echo` is built on a high-performance foundation, but there's always room to
244+
push the boundaries of speed and efficiency. We maintain a detailed roadmap of
245+
features and performance optimizations, with tasks suitable for all skill
246+
levels.
247+
248+
| Contribution Level | Example Tasks |
249+
| :----------------- | :---------------------------------------------------------- |
250+
| **Quick Wins** | Implement faster random number generation for stealing. |
251+
| **Architectural** | Add a true sleep/notification system for idle workers. |
252+
| **Expert Tuning** | Build a `criterion` benchmark suite; implement CPU pinning. |
253+
| **Advanced Logic** | Introduce an anti-starvation mechanism for tasks. |
128254

129-
CC0 1.0 Universal. Public domain. No restrictions.
130-
[LICENSE](https://github.com/CodeEditorLand/Echo/tree/Current/LICENSE)
255+
**Interested in tackling one of these challenges?** 👉🏻
256+
257+
- **[Check out our full TODO](https://github.com/CodeEditorLand/Echo/tree/Current/Documentation/GitHub/Todo.md)**
258+
for challenges!
259+
- **[Follow our Contribution Guide](https://github.com/CodeEditorLand/Echo/tree/Current/)**
260+
to get started!
131261

132262
---
133263

@@ -139,8 +269,26 @@ CC0 1.0 Universal. Public domain. No restrictions.
139269
- [Mountain](https://github.com/CodeEditorLand/Mountain)
140270
- [Common](https://github.com/CodeEditorLand/Common)
141271

272+
---
273+
274+
## License&#x2001;⚖️
275+
276+
This project is released into the public domain under the **Creative Commons CC0
277+
Universal** license. You are free to use, modify, distribute, and build upon
278+
this work for any purpose, without any restrictions. For the full legal text,
279+
see the [`LICENSE`](https://github.com/CodeEditorLand/Echo/tree/Current/) file.
280+
281+
---
282+
283+
## Changelog&#x2001;📜
284+
285+
Stay updated with our progress! See
286+
[`CHANGELOG.md`](https://github.com/CodeEditorLand/Echo/tree/Current/) for a
287+
history of changes specific to **Echo**.
288+
289+
---
142290

143-
## Funding & Acknowledgements🙏🏻
291+
## Funding \& Acknowledgements&#x2001;🙏🏻
144292

145293
**Echo** is a core element of the **Land** ecosystem. This project is funded
146294
through [NGI0 Commons Fund](https://NLnet.NL/commonsfund), a fund established by

0 commit comments

Comments
 (0)