Skip to main content

Deployment Topology

This page describes the network layout, port assignments, and directory structure of a running system.

Network Layout​

All components run on localhost for development and demo purposes. In production, each component would run on a separate host. The architecture doesn't change — only the RMI URLs would reference different IPs.

Port Assignments​

ComponentPortRMI BindingTLS
AuthService1098//localhost:1098/AuthServicemTLS (client auth required)
ReplicaNode 01099//localhost:1099/ReplicaNode0mTLS (client auth required)
ReplicaNode 11100//localhost:1100/ReplicaNode1mTLS (client auth required)
ReplicaNode 21101//localhost:1101/ReplicaNode2mTLS (client auth required)
ClientephemeralNone (client only)mTLS (presents client.jks)

Startup Order​

The system has a strict startup dependency chain:

1. AuthService → must start first (nodes depend on it for token validation)
2. ReplicaNode 0 → can start after Auth
3. ReplicaNode 1 → discovers peers via NodeRegistry
4. ReplicaNode 2 → discovers peers via NodeRegistry
5. DFSClient → connects to any node after all 3 are up

Why this order?​

  1. Auth first: Nodes try to connect to the AuthService at startup. If Auth isn't running, nodes will fail with java.rmi.ConnectException.
  2. Nodes next: The Raft election needs all nodes running to reach quorum. Starting them in any order is fine, but wait ~3 seconds for the election to settle.
  3. Client last: The interactive shell needs working nodes and Auth to function. It will gracefully fail if services aren't up yet.

Certificate Distribution​

Every component that participates in mTLS needs two files:

ComponentKeystore (own identity)Truststore (trusted CAs)
AuthServicecerts/auth.jkscerts/truststore.jks
Node 0certs/node0.jkscerts/truststore.jks
Node 1certs/node1.jkscerts/truststore.jks
Node 2certs/node2.jkscerts/truststore.jks
Clientcerts/client.jkscerts/truststore.jks

All passwords: changeit

The truststore is identical for all components — it contains only the CA's public certificate. This means any entity whose certificate was signed by the CA is trusted by every other entity.

Storage Directories​

Each node maintains its own completely independent storage:

storage/
├── node0/
│ ├── report.pdf
│ ├── data.csv
│ └── ...
├── node1/
│ ├── report.pdf
│ ├── data.csv
│ └── ...
└── node2/
├── report.pdf
├── data.csv
└── ...

After any write operation, all three directories should contain identical files in the same order. TO-Multicast guarantees this.

info

The storage/ directory is gitignored and created at runtime. You can delete it to reset the file system: rm -rf storage/

Configuration File​

The nodes.properties file maps node IDs to RMI addresses:

node.0=//localhost:1099/ReplicaNode0
node.1=//localhost:1100/ReplicaNode1
node.2=//localhost:1101/ReplicaNode2
node.count=3

When a new node starts, it reads this file to discover peers. The node.count field tells the TO-Multicast algorithm how many ACKs to expect before a message is deliverable.

Scaling Beyond 3 Nodes​

The system is designed for exactly 3 replicas (the Raft implementation assumes N=3). To scale:

  • Update nodes.properties with additional nodes
  • Adjust Raft quorum calculation (> N/2 instead of > 1)
  • Increase TO-Multicast ACK threshold
  • Generate additional keystores (node3.jks, node4.jks, ...)

The core algorithms (Lamport clocks, TO-Multicast, Raft) all generalize to arbitrary N.