Skip to main content

Deployment Topology

This page describes the network layout, port assignments, and directory structure of a running system.

Network Layout

All components run on localhost for development and demo purposes. In production, each component would run on a separate host. The architecture doesn't change — only the RMI URLs would reference different IPs.

Port Assignments

ComponentPortRMI BindingTLS
AuthService1098//localhost:1098/AuthServicemTLS (client auth required)
ReplicaNode 01099//localhost:1099/ReplicaNode0mTLS (client auth required)
ReplicaNode 11100//localhost:1100/ReplicaNode1mTLS (client auth required)
ReplicaNode 21101//localhost:1101/ReplicaNode2mTLS (client auth required)
ClientephemeralNone (client only)mTLS (presents client.jks)

Startup Order

The system has a strict startup dependency chain:

1. AuthService → must start first (nodes depend on it for token validation)
2. ReplicaNode 0 → can start after Auth
3. ReplicaNode 1 → discovers peers via NodeRegistry
4. ReplicaNode 2 → discovers peers via NodeRegistry
5. DFSClient → connects to any node after all 3 are up

Why this order?

  1. Auth first: Nodes try to connect to the AuthService at startup. If Auth isn't running, nodes will fail with java.rmi.ConnectException.
  2. Nodes next: The Raft election needs all nodes running to reach quorum. Starting them in any order is fine, but wait ~3 seconds for the election to settle.
  3. Client last: The interactive shell needs working nodes and Auth to function. It will gracefully fail if services aren't up yet.

Certificate Distribution

Every component that participates in mTLS needs two files:

ComponentKeystore (own identity)Truststore (trusted CAs)
AuthServicecerts/auth.jkscerts/truststore.jks
Node 0certs/node0.jkscerts/truststore.jks
Node 1certs/node1.jkscerts/truststore.jks
Node 2certs/node2.jkscerts/truststore.jks
Clientcerts/client.jkscerts/truststore.jks

All passwords: changeit

The truststore is identical for all components — it contains only the CA's public certificate. This means any entity whose certificate was signed by the CA is trusted by every other entity.

Storage Directories

Each node maintains its own completely independent storage:

storage/
├── node0/
│ ├── report.pdf
│ ├── data.csv
│ └── ...
├── node1/
│ ├── report.pdf
│ ├── data.csv
│ └── ...
└── node2/
├── report.pdf
├── data.csv
└── ...

After any write operation, all three directories should contain identical files in the same order. TO-Multicast guarantees this.

info

The storage/ directory is gitignored and created at runtime. You can delete it to reset the file system: rm -rf storage/

Configuration File

The nodes.properties file maps node IDs to RMI addresses:

node.0=//localhost:1099/ReplicaNode0
node.1=//localhost:1100/ReplicaNode1
node.2=//localhost:1101/ReplicaNode2
node.count=3

When a new node starts, it reads this file to discover peers. The node.count field tells the TO-Multicast algorithm how many ACKs to expect before a message is deliverable.

Scaling Beyond 3 Nodes

The system is designed for exactly 3 replicas (the Raft implementation assumes N=3). To scale:

  • Update nodes.properties with additional nodes
  • Adjust Raft quorum calculation (> N/2 instead of > 1)
  • Increase TO-Multicast ACK threshold
  • Generate additional keystores (node3.jks, node4.jks, ...)

The core algorithms (Lamport clocks, TO-Multicast, Raft) all generalize to arbitrary N.