Skip to content

Commit

Permalink
Merge pull request facebookresearch#24 from asonnino/master
Browse files Browse the repository at this point in the history
Minor changes
  • Loading branch information
Alberto Sonnino authored Dec 15, 2021
2 parents 94cc9a8 + 88c757e commit 727530c
Show file tree
Hide file tree
Showing 6 changed files with 19 additions and 19 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Narwhal and Tusk

[![build status](https://img.shields.io/github/workflow/status/facebookresearch/narwhal/Rust/master?style=flat-square&logo=github)](https://github.com/facebookresearch/narwhal/actions)
[![build status](https://img.shields.io/github/workflow/status/asonnino/narwhal/Rust/master?style=flat-square&logo=github)](https://github.com/asonnino/narwhal/actions)
[![rustc](https://img.shields.io/badge/rustc-1.51+-blue?style=flat-square&logo=rust)](https://www.rust-lang.org)
[![license](https://img.shields.io/badge/license-Apache-blue.svg?style=flat-square)](LICENSE)

Expand All @@ -10,7 +10,7 @@ This repo provides an implementation of [Narwhal and Tusk](https://arxiv.org/pdf
The core protocols are written in Rust, but all benchmarking scripts are written in Python and run with [Fabric](http://www.fabfile.org/).
To deploy and benchmark a testbed of 4 nodes on your local machine, clone the repo and install the python dependencies:
```
$ git clone https://github.com/facebookresearch/narwhal.git
$ git clone https://github.com/asonnino/narwhal.git
$ cd narwhal/benchmark
$ pip install -r requirements.txt
```
Expand Down Expand Up @@ -52,9 +52,9 @@ This command may take a long time the first time you run it (compiling rust code
```

## Next Steps
The next step is to read the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). It is then recommended to have a look at the README files of the [worker](https://github.com/facebookresearch/narwhal/tree/master/worker) and [primary](https://github.com/facebookresearch/narwhal/tree/master/primary) crates. An additional resource to better understand the Tusk consensus protocol is the paper [All You Need is DAG](https://arxiv.org/abs/2102.08325) as it describes a similar protocol.
The next step is to read the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). It is then recommended to have a look at the README files of the [worker](https://github.com/asonnino/narwhal/tree/master/worker) and [primary](https://github.com/asonnino/narwhal/tree/master/primary) crates. An additional resource to better understand the Tusk consensus protocol is the paper [All You Need is DAG](https://arxiv.org/abs/2102.08325) as it describes a similar protocol.

The README file of the [benchmark folder](https://github.com/facebookresearch/narwhal/tree/master/benchmark) explains how to benchmark the codebase and read benchmarks' results. It also provides a step-by-step tutorial to run benchmarks on [Amazon Web Services (AWS)](https://aws.amazon.com) accross multiple data centers (WAN).
The README file of the [benchmark folder](https://github.com/asonnino/narwhal/tree/master/benchmark) explains how to benchmark the codebase and read benchmarks' results. It also provides a step-by-step tutorial to run benchmarks on [Amazon Web Services (AWS)](https://aws.amazon.com) accross multiple data centers (WAN).

## License
This software is licensed as [Apache 2.0](LICENSE).
18 changes: 9 additions & 9 deletions benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This document explains how to benchmark the codebase and read benchmarks' result
When running benchmarks, the codebase is automatically compiled with the feature flag `benchmark`. This enables the node to print some special log entries that are then read by the python scripts and used to compute performance. These special log entries are clearly indicated with comments in the code: make sure to not alter them (otherwise the benchmark scripts will fail to interpret the logs).

### Parametrize the benchmark
After cloning the repo and [installing all dependencies](https://github.com/facebookresearch/narwhal#quick-start), you can use [Fabric](http://www.fabfile.org/) to run benchmarks on your local machine. Locate the task called `local` in the file [fabfile.py](https://github.com/facebookresearch/narwhal/blob/master/benchmark/fabfile.py):
After cloning the repo and [installing all dependencies](https://github.com/asonnino/narwhal#quick-start), you can use [Fabric](http://www.fabfile.org/) to run benchmarks on your local machine. Locate the task called `local` in the file [fabfile.py](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.py):
```python
@task
def local(ctx):
Expand Down Expand Up @@ -104,7 +104,7 @@ $ ssh-keygen -f ~/.ssh/aws
```

### Step 3. Configure the testbed
The file [settings.json](https://github.com/facebookresearch/narwhal/blob/master/benchmark/settings.json) (located in [narwhal/benchmarks](https://github.com/facebookresearch/narwhal/blob/master/benchmark)) contains all the configuration parameters of the testbed to deploy. Its content looks as follows:
The file [settings.json](https://github.com/asonnino/narwhal/blob/master/benchmark/settings.json) (located in [narwhal/benchmarks](https://github.com/asonnino/narwhal/blob/master/benchmark)) contains all the configuration parameters of the testbed to deploy. Its content looks as follows:
```json
{
"key": {
Expand All @@ -114,7 +114,7 @@ The file [settings.json](https://github.com/facebookresearch/narwhal/blob/master
"port": 5000,
"repo": {
"name": "narwhal",
"url": "https://github.com/facebookresearch/narwhal.git",
"url": "https://github.com/asonnino/narwhal.git",
"branch": "master"
},
"instances": {
Expand Down Expand Up @@ -143,7 +143,7 @@ The third block (`repo`) contains the information regarding the repository's nam
```json
"repo": {
"name": "narwhal",
"url": "https://github.com/facebookresearch/narwhal.git",
"url": "https://github.com/asonnino/narwhal.git",
"branch": "master"
},
```
Expand All @@ -159,12 +159,12 @@ The the last block (`instances`) specifies the [AWS instance type](https://aws.a
The instance type selects the hardware on which to deploy the testbed. For example, `m5d.8xlarge` instances come with 32 vCPUs (16 physical cores), 128 GB of RAM, and guarantee 10 Gbps of bandwidth. The python scripts will configure each instance with 300 GB of SSD hard drive. The `regions` field specifies the data centers to use. If you require more nodes than data centers, the python scripts will distribute the nodes as equally as possible amongst the data centers. All machines run a fresh install of Ubuntu Server 20.04.

### Step 4. Create a testbed
The AWS instances are orchestrated with [Fabric](http://www.fabfile.org) from the file [fabfile.py](https://github.com/facebookresearch/narwhal/blob/master/benchmark/fabfile.pyy) (located in [narwhal/benchmarks](https://github.com/facebookresearch/narwhal/blob/master/benchmark)); you can list all possible commands as follows:
The AWS instances are orchestrated with [Fabric](http://www.fabfile.org) from the file [fabfile.py](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.pyy) (located in [narwhal/benchmarks](https://github.com/asonnino/narwhal/blob/master/benchmark)); you can list all possible commands as follows:
```
$ cd narwhal/benchmark
$ fab --list
```
The command `fab create` creates new AWS instances; open [fabfile.py](https://github.com/facebookresearch/narwhal/blob/master/benchmark/fabfile.py) and locate the `create` task:
The command `fab create` creates new AWS instances; open [fabfile.py](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.py) and locate the `create` task:
```python
@task
def create(ctx, nodes=2):
Expand All @@ -189,13 +189,13 @@ This may take a long time as the command will first update all instances.
The commands `fab stop` and `fab start` respectively stop and start the testbed without destroying it (it is good practice to stop the testbed when not in use as AWS can be quite expensive); and `fab destroy` terminates all instances and destroys the testbed. Note that, depending on the instance types, AWS instances may take up to several minutes to fully start or stop. The command `fab info` displays a nice summary of all available machines and information to manually connect to them (for debug).

### Step 5. Run a benchmark
After setting up the testbed, running a benchmark on AWS is similar to running it locally (see [Run Local Benchmarks](https://github.com/facebookresearch/narwhal/tree/master/benchmark#local-benchmarks)). Locate the task `remote` in [fabfile.py](https://github.com/facebookresearch/narwhal/blob/master/benchmark/fabfile.py):
After setting up the testbed, running a benchmark on AWS is similar to running it locally (see [Run Local Benchmarks](https://github.com/asonnino/narwhal/tree/master/benchmark#local-benchmarks)). Locate the task `remote` in [fabfile.py](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.py):
```python
@task
def remote(ctx):
...
```
The benchmark parameters are similar to [local benchmarks](https://github.com/facebookresearch/narwhal/tree/master/benchmark#local-benchmarks) but allow to specify the number of nodes and the input rate as arrays to automate multiple benchmarks with a single command. The parameter `runs` specifies the number of times to repeat each benchmark (to later compute the average and stdev of the results), and the parameter `collocate` specifies whether to collocate all the node's workers and the primary on the same machine. If `collocate` is set to `False`, the script will run one node per data center (AWS region), with its primary and each of its worker running on a dedicated instance.
The benchmark parameters are similar to [local benchmarks](https://github.com/asonnino/narwhal/tree/master/benchmark#local-benchmarks) but allow to specify the number of nodes and the input rate as arrays to automate multiple benchmarks with a single command. The parameter `runs` specifies the number of times to repeat each benchmark (to later compute the average and stdev of the results), and the parameter `collocate` specifies whether to collocate all the node's workers and the primary on the same machine. If `collocate` is set to `False`, the script will run one node per data center (AWS region), with its primary and each of its worker running on a dedicated instance.
```python
bench_params = {
'nodes': [10, 20, 30],
Expand All @@ -214,7 +214,7 @@ Once you specified both `bench_params` and `node_params` as desired, run:
```
$ fab remote
```
This command first updates all machines with the latest commit of the GitHub repo and branch specified in your file [settings.json](https://github.com/facebookresearch/narwhal/blob/master/benchmark/settings.json) (step 3); this ensures that benchmarks are always run with the latest version of the code. It then generates and uploads the configuration files to each machine, runs the benchmarks with the specified parameters, and downloads the logs. It finally parses the logs and prints the results into a folder called `results` (which is automatically created if it doesn't already exists). You can run `fab remote` multiple times without fearing to override previous results, the command either appends new results to a file containing existing ones or prints them in separate files. If anything goes wrong during a benchmark, you can always stop it by running `fab kill`.
This command first updates all machines with the latest commit of the GitHub repo and branch specified in your file [settings.json](https://github.com/asonnino/narwhal/blob/master/benchmark/settings.json) (step 3); this ensures that benchmarks are always run with the latest version of the code. It then generates and uploads the configuration files to each machine, runs the benchmarks with the specified parameters, and downloads the logs. It finally parses the logs and prints the results into a folder called `results` (which is automatically created if it doesn't already exists). You can run `fab remote` multiple times without fearing to override previous results, the command either appends new results to a file containing existing ones or prints them in separate files. If anything goes wrong during a benchmark, you can always stop it by running `fab kill`.

### Step 6. Plot the results
Once you have enough results, you can aggregate and plot them:
Expand Down
2 changes: 1 addition & 1 deletion benchmark/data/paper-data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The content of our [settings.json](https://github.com/asonnino/narwhal/blob/mast
"port": 5000,
"repo": {
"name": "narwhal",
"url": "https://github.com/facebookresearch/narwhal",
"url": "https://github.com/asonnino/narwhal",
"branch": "master"
},
"instances": {
Expand Down
6 changes: 3 additions & 3 deletions benchmark/settings.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"key": {
"name": "aws-fb",
"path": "/Users/asonnino/.ssh/aws-fb.pem"
"name": "aws",
"path": "/Users/asonnino/.ssh/aws.pem"
},
"port": 5000,
"repo": {
"name": "narwhal",
"url": "https://github.com/facebookresearch/narwhal",
"url": "https://github.com/asonnino/narwhal",
"branch": "master"
},
"instances": {
Expand Down
2 changes: 1 addition & 1 deletion primary/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
The diagram below illustrates the primary's architecture and could be useful to keep in mind while going through the code.

<p align="center">
<img src="https://github.com/facebookresearch/narwhal/blob/master/.assets/diagram-primary.svg">
<img src="https://github.com/asonnino/narwhal/blob/master/.assets/diagram-primary.svg">
</p>
2 changes: 1 addition & 1 deletion worker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
The diagram below illustrates the worker's architecture and could be useful to keep in mind while going through the code.

<p align="center">
<img src="https://github.com/facebookresearch/narwhal/blob/master/.assets/diagram-worker.svg">
<img src="https://github.com/asonnino/narwhal/blob/master/.assets/diagram-worker.svg">
</p>

0 comments on commit 727530c

Please sign in to comment.