Ticket Contents

Description

This project focuses on building a production-grade decentralized collaboration layer for spreadsheet applications by deeply integrating EtherCalc (or a modernized SocialCalc UI) with py-libp2p.

Today, EtherCalc provides real-time collaborative editing using a centralized server model (Node.js + WebSockets). While functional, this architecture introduces:

• Central points of failure
• Scalability bottlenecks
• Lack of offline-first capability
• Limited applicability in low-connectivity civic environments

This project re-architects the synchronization layer, not the UI, by:

• Retaining EtherCalc/SocialCalc UI for editing experience
• Replacing centralized sync with libp2p-based decentralized messaging
• Enabling peer-to-peer real-time spreadsheet collaboration

A key design principle is:

Do not rebuild the spreadsheet UI — reuse EtherCalc/SocialCalc and focus entirely on decentralized systems innovation.

Goals & Mid-Point Milestone

Goals

• [Perform deep analysis of EtherCalc/SocialCalc internal data model and event system]

• [Design a protocol to encode spreadsheet operations into libp2p messages]

• [Implement a py-libp2p-based pub-sub layer (GossipSub) for real-time updates]

• [Build an adapter layer bridging UI events ↔ libp2p network]

• [Implement conflict resolution using CRDTs tailored for spreadsheet structures]

• [Enable peer discovery using libp2p (mDNS for LAN, DHT for WAN)]

• [Implement state synchronization for late-joining peers]

• [Add persistence layer (local-first with optional IPFS integration)]

• [Benchmark performance against centralized EtherCalc deployment]

• [Document architecture, protocol design, and developer workflows]

• [Goals Achieved By Mid-point Milestone]

• [Complete analysis of EtherCalc/SocialCalc internal data structures and event flow]

• [Define and document the spreadsheet operation encoding protocol for P2P messaging]

• [Set up a functional py-libp2p node with basic connectivity between peers]

• [Implement GossipSub-based pub-sub communication for broadcasting updates]

• [Build initial adapter to capture UI events and convert them into libp2p messages]

• [Enable basic peer discovery using mDNS (local network)]

• [Achieve real-time synchronization of simple cell edits across 2–3 peers]

• [Implement message handling pipeline (send → receive → apply updates in UI)]

• [Ensure basic consistency (last-write-wins or simple conflict handling)]

• [Create a working demo showcasing multi-peer collaborative editing]

• [Document setup process, architecture, and initial learnings]

Setup/Installation

Setup/Installation

This project involves integrating a spreadsheet UI (EtherCalc or SocialCalc-based UI) with a decentralized networking layer using py-libp2p. The setup is divided into three main components: UI, P2P node, and adapter layer.

Prerequisites

• Python 3.10+
• Node.js (v16 or above) and npm
• Git
• Basic familiarity with Python async programming and JavaScript

1. Clone the Repository

git clone <project-repo-url>
cd <project-directory>

2. Setup Spreadsheet UI

Option A: EtherCalc

git clone https://github.com/ether/ethercalc
cd ethercalc
npm install
npm start

• Runs on: http://localhost:8000

Option B: SocialCalc UI (if used)

• Clone or use a maintained fork of SocialCalc
• Serve using a simple HTTP server:

npx serve .

3. Setup py-libp2p Node

Install dependencies:

pip install py-libp2p asyncio trio multiaddr

Run the P2P node:

python p2p_node.py

This will:

• Initialize a libp2p peer
• Join a pub-sub topic (e.g., ethercalc-updates)
• Enable peer discovery (mDNS initially)

4. Start Adapter Layer

The adapter connects the UI with the P2P network.

python adapter.py

Responsibilities of the contributor:

• Capture spreadsheet events (cell edits, updates)
• Encode them into libp2p-compatible messages
• Broadcast updates via pub-sub
• Apply incoming updates to the UI

5. Running Multiple Peers (Local Testing)

To simulate multiple peers:

• Open multiple terminal instances
• Run separate p2p_node.py processes (different ports)
• Open UI in multiple browser tabs/devices

Peers should automatically discover each other via mDNS and sync updates.

6. Optional: Advanced Setup (Later Phases)

• Enable DHT-based peer discovery for non-local networks
• Add IPFS for persistence:

pip install ipfshttpclient

• Dockerize services for easier deployment

7. Verification

• Edit a cell in one peer
• Confirm update appears in other peers in real time
• Restart a peer and verify state resynchronization

Notes

• Initial setup focuses on local network (LAN) testing
• Production deployment considerations (security, scaling) will be added later
• Detailed scripts, configs, and troubleshooting guide will be included in the repository

Expected Outcome

Expected Outcome

The final outcome of this project will be a fully functional decentralized collaborative spreadsheet system that combines the familiar user experience of EtherCalc (or a SocialCalc-based UI) with the resilience and scalability of py-libp2p.

Functional Behavior

• Users can open the spreadsheet UI (EtherCalc/SocialCalc) in their browser and start editing immediately, just like a traditional collaborative spreadsheet.

• Multiple users (peers) can edit the same spreadsheet simultaneously in real time, without connecting to any central server.

• All updates (cell edits, formulas, formatting changes) are propagated through a peer-to-peer pub-sub network.

• The system supports offline-first workflows:

  • If a peer temporarily disconnects, it can continue making edits locally.
  • Once reconnected, changes are synchronized automatically with other peers.

• New peers joining the network can reconstruct the current spreadsheet state by syncing from existing peers.

System Characteristics

1. Decentralization

• No central backend server is required for coordination or data storage.
• Each peer acts as both a client and a server in the network.
• The system remains operational even if multiple peers drop out.

2. Real-Time Synchronization

• Spreadsheet updates propagate across peers with minimal latency (near real-time on LAN).
• Changes appear seamlessly across all connected instances.

3. Consistency & Conflict Handling

• The system ensures eventual consistency across all peers.
• Concurrent edits are resolved using deterministic conflict resolution (e.g., CRDT or last-write-wins baseline).
• No data corruption or divergence in spreadsheet state.

4. Peer Discovery & Networking

• Peers automatically discover each other:

  • Locally via mDNS
  • Across networks via DHT (in advanced stages)

• Network topology is dynamic, handling peer joins and leaves gracefully.

5. Persistence & Recovery

• Spreadsheet state is preserved locally on each peer.

• On restart, a peer can:

  • Reload its last known state
  • Sync missing updates from the network

• Optional support for decentralized storage (e.g., IPFS) for durable snapshots.

Technical Deliverables

• A working integration of EtherCalc/SocialCalc UI with a libp2p-based synchronization layer

• A reusable adapter/middleware layer that bridges UI events and P2P messaging

• A defined protocol for spreadsheet operation encoding and propagation

• A modular py-libp2p node implementation supporting pub-sub, discovery, and sync

• Documentation covering:

  • Architecture
  • Setup and usage
  • Protocol design

Demonstration Scenario

A complete demo will showcase:

• 3–5 peers editing the same spreadsheet in real time
• One peer going offline and continuing edits
• Reconnection and automatic synchronization
• A new peer joining and reconstructing the full spreadsheet state
• No reliance on any centralized coordination server

End State Vision

By the end of the project, the system will behave like a Google Sheets–like experience powered entirely by peer-to-peer infrastructure, making it suitable for:

• Low-connectivity or rural environments
• Disaster response coordination
• Privacy-preserving collaboration
• Open, resilient civic-tech applications

Acceptance Criteria

Acceptance Criteria

• The system uses EtherCalc or a SocialCalc-based UI without requiring a custom spreadsheet frontend, preserving the core user experience.

• A py-libp2p node is successfully integrated as the primary networking layer for all synchronization.

• At least 3–5 peers can connect and collaboratively edit the same spreadsheet without any central server.

• Spreadsheet operations (cell edits, formulas, structural changes) are:

  • Captured from the UI
  • Encoded into messages
  • Broadcast via libp2p pub-sub
  • Applied correctly on all peers

• Updates propagate across peers with near real-time latency (acceptable delay: ~1–2 seconds on local networks).

• The system guarantees eventual consistency, such that:

  • All peers converge to the same spreadsheet state
  • No permanent divergence occurs

• Basic conflict resolution is implemented:

  • Concurrent edits do not corrupt data
  • Deterministic resolution (e.g., last-write-wins or CRDT-based approach)

• Peer discovery works reliably:

  • mDNS for local network discovery
  • DHT-based discovery (at least partially functional or demonstrated)

• Late-joining peers can:

  • Discover existing peers
  • Sync and reconstruct the latest spreadsheet state

• The system supports peer churn:

  • Peers can join/leave without breaking synchronization
  • Remaining peers continue functioning correctly

• Basic offline handling is demonstrated:

  • A peer can disconnect, make edits locally (optional baseline)
  • Upon reconnection, state synchronizes correctly

• A working adapter layer exists that:

  • Bridges UI events ↔ libp2p messages
  • Handles message serialization, deserialization, and application

• A working demo is provided that clearly shows:

  • Multi-peer collaboration
  • Network resilience (disconnect/reconnect)
  • State synchronization

• Documentation is complete, including:

  • Setup and installation guide
  • Architecture overview
  • Protocol/message format
  • Instructions to reproduce the demo

• Codebase is:

  • Modular and well-structured
  • Open-source and reproducible
  • Easy for future contributors to extend

Implementation Details

Implementation Details

This project implements a decentralized real-time synchronization layer for spreadsheet collaboration by integrating EtherCalc (or a SocialCalc-based UI) with py-libp2p. The focus is on replacing the centralized backend with a peer-to-peer (P2P) architecture, while keeping the UI layer largely unchanged.

1. System Architecture

The system is divided into three major components:

(a) UI Layer (EtherCalc / SocialCalc)

• Runs in the browser (JavaScript-based)

• Responsible for:

  • Rendering spreadsheet grid
  • Handling user inputs (cell edits, formulas, formatting)

• Minimal modifications:

  • Hook into internal event system (e.g., cell update events)
  • Emit structured operation logs instead of relying solely on WebSockets

(b) Adapter Layer (Core Contribution)

• Acts as a bridge between UI and P2P network

• Can be implemented as:

  • Node.js middleware OR
  • Python service with WebSocket/HTTP bridge

Responsibilities:

• Capture UI events (cell edits, sheet updates)

• Convert events into structured operations

• Serialize operations into message format

• Publish messages to libp2p network

• Receive incoming messages and apply updates to UI

• Handle:

  • Message ordering
  • Deduplication (using operation IDs)
  • Retry/buffering

(c) P2P Networking Layer (py-libp2p)

• Built using py-libp2p

Key Components:

• GossipSub (Pub-Sub):

  • Topic: spreadsheet-updates
  • Broadcast all operations to peers

• Kademlia DHT:

  • Peer discovery across networks
  • Optional: locating peers holding specific document states

• mDNS:

  • Local peer discovery (LAN environments)

• Streams:

  • Used for:

    • State sync
    • Snapshot transfer

2. Operation-Based Synchronization

Instead of syncing full spreadsheet state, the system uses operation-based updates.

Operation Format (Example)

{
  "op_id": "uuid",
  "peer_id": "peer_public_key",
  "timestamp": 1710000000,
  "sheet_id": "sheet-1",
  "operation": {
    "type": "SET_CELL",
    "cell": "A1",
    "value": "42"
  }
}

Supported Operations

• SET_CELL (value updates)
• SET_FORMULA
• INSERT_ROW / DELETE_ROW
• INSERT_COLUMN / DELETE_COLUMN
• FORMAT_CELL

3. Conflict Resolution Strategy

To ensure consistency across peers:

Phase 1 (Baseline)

• Last-Write-Wins (LWW) using timestamps

Phase 2 (Advanced)

• CRDT-based approach:

  • Cell-level CRDT registers
  • Operation-based CRDT for structural changes

Guarantees

• Deterministic resolution
• Eventual consistency
• No divergence across peers

4. State Synchronization (Late Joiners)

When a new peer joins:

1. Discover peers via mDNS/DHT

2. Request latest snapshot via libp2p stream

3. Receive:

   • Full spreadsheet snapshot
   • Recent operation log

4. Apply snapshot → replay operations

5. Persistence Layer

Local Persistence

• Store:

  • Latest spreadsheet snapshot
  • Operation logs

• Technologies:

  • Browser: IndexedDB
  • Python: local file storage

Optional (Stretch)

• IPFS integration:

  • Store snapshots as content-addressed data
  • Share snapshot hashes via libp2p

6. Networking Flow

Please visit https://many-rope-89706694.figma.site/

7. Technologies Used

• Frontend/UI:

  • EtherCalc (Node.js, JavaScript) OR SocialCalc

• Backend / Adapter:

  • Python (asyncio)
  • Optional Node.js bridge

• Networking:

  • py-libp2p
  • GossipSub (pub-sub)
  • Kademlia DHT
  • mDNS

• Data Handling:

  • JSON / MessagePack for serialization

• Storage (optional):

  • IndexedDB (browser)
  • IPFS (decentralized storage)

8. Testing & Evaluation

• Simulate multiple peers locally (multiple processes)

• Test scenarios:

  • Concurrent edits
  • Peer disconnection/reconnection
  • Late join synchronization

• Metrics:

  • Latency (update propagation time)
  • Consistency (state convergence)
  • Network overhead

9. Extensibility

The architecture is designed to be reusable for:

• Other collaborative tools (documents, whiteboards)
• Civic-tech data collection platforms
• Offline-first distributed applications

Optional UI Enhancements (Non-Invasive)

• Peer presence indicator (e.g., "3 users connected")
• Sync status:
• Connected
• Syncing
• Offline
• Visual indicator for incoming updates (cell highlight)

Product Name

P2PCalc: Decentralized Spreadsheet over py-libp2p

Organisation Name

NSUT x SEETA x AIC

Domain

Financial Inclusion

Tech Skills Needed

Artificial Intelligence, Ionic, DevOps, Chart.js, JavaScript, Machine Learning, TypeScript, Docker, MongoDB, Node.js, Solidity, Mobile, AWS

Mentor(s)

@seetadev , @johannamoran, @aspiringsecurity, @yashksaini-coder , @prithagupta, @acul71

Category

Database, API, AI