Directed Portion (25%)

General Methodology

In this lab, you will write multithreaded C programs and evaluate their correctness and performance on a cycle-accurate RTL simulator of the dual-core system. The workflow consists of two steps:

  1. Build the simulator: Build the cycle-accurate RTL simulator for each of the two cache coherence protocols (MI and MSI).

  2. Run and evaluate: Run your programs on the simulators to verify correctness, measure performance, and compare the behavior of the two coherence protocols.

Setup

This lab assumes you have already set up your environment in Lab 0, in particular the Miniforge install that puts conda in your home directory. You do not need to re-create the conda environment or rebuild either RISC-V toolchain.

First-time setup

Run this once. The clone has no submodules, and the RTL build needs most of them, so the build-setup.sh step is not optional.

cd ~
git clone -b fa26-lab4 https://github.com/cs152-teach/chipyard-cs152.git cs152-lab4-fa26
cd cs152-lab4-fa26
ln -sfn /home/ff/cs152/fa26/chipyard/.conda-env .conda-env
source ~/conda/etc/profile.d/conda.sh
conda activate .conda-env/
./build-setup.sh riscv-tools --skip-conda --skip-toolchain --skip-circt --skip-firesim --skip-marshal

Every new shell

After the first-time setup, source the environment script for a quick start. Update the ~/cs152-env.sh you created in Lab 0:

cat > ~/cs152-env.sh << 'EOF'
# CS152 lab environment. Usage: source ~/cs152-env.sh [lab-directory]
# With no argument, enters the chipyard tree used by Labs 0 and 1.
source ~/conda/etc/profile.d/conda.sh
cd ~/"${1:-chipyard}"
conda activate .conda-env/
export LAB4ROOT=$PWD
export TESTDIR=${LAB4ROOT}/lab
export SIMDIR=${LAB4ROOT}/sims/verilator
source ./env.sh
# The lab 4 programs are rv32; this is the toolchain that builds them.
export PATH="$PATH:/home/ff/cs152/bin/riscv-gnu-toolchain/riscv32/bin"
EOF

Then, for new shells, run the following to enter the lab environment:

source ~/cs152-env.sh cs152-lab4-fa26

The remainder of this lab will use ${LAB4ROOT} to denote the path of the working tree. Its directory structure is outlined below:

${LAB4ROOT}/
├── lab/                              # ${TESTDIR}: benchmark source code
│   ├── Makefile
│   ├── runtime/                      # bare-metal runtime: crt.S, barrier(), setStats(), printf
│   ├── mt-vvadd-naive/               # naive vvadd code
│   ├── mt-vvadd-opt/                 # your optimized vvadd
│   ├── mt-matmul-naive/              # naive matmul code
│   └── mt-matmul-opt/                # your optimized matmul
├── generators/
│   ├── chipyard/src/main/scala/cs152/  # the dual-core system, caches and bus
│   ├── riscv-sodor/                    # the Sodor core
│   ├── ...
└── sims/
    └── verilator/                    # ${SIMDIR}: Verilator simulation flow

You do not have to read the RTL in generators/chipyard/src/main/scala/cs152/. It is short enough to skim if you are curious (CoherentBus.scala describes a bus transaction step by step), but no exercise here requires reading or modifying it.

Measuring Vector-Vector Add with MSI Coherence

To familiarize yourself with the Lab 4 infrastructure, you will first measure the performance of a naive multithreaded implementation of vvadd, which performs element-wise addition of two vectors.

Navigate to ${TESTDIR}/mt-vvadd-naive. The directory contains several files of interest:

  • dataset.h: Contains the input vectors and the expected output vector.
  • mt-vvadd_main.c: Manages the benchmark execution. Both cores synchronize, enable their performance counters, execute vvadd, and synchronize again before core 0 verifies the result.
  • mt-vvadd_naive.c: Contains a naive multithreaded implementation of vvadd, which you will use as the starting point for this experiment.

Build the benchmarks:

cd ${TESTDIR}
make

Then build the simulator and run mt-vvadd-naive on the dual-core configuration with MSI coherence:

cd ${SIMDIR}
make -j4 CONFIG=CS152Lab4MSIConfig
make CONFIG=CS152Lab4MSIConfig run-binary-fast BINARY=${TESTDIR}/mt-vvadd-naive.riscv

At the end of the run the benchmark prints one column of counters per core, followed by the verdict:

=== CS152 L1D counters (region of interest, D-side only) ===
                     core 0     core 1      total
  cycles              54414      54416
  instret              4570       4568       9138
  loads                1006       1012       2018
  stores                505        503       1008
  hits                  750        756       1506
  load misses           256        256        512
  store misses          503        501       1004
  upgrades                2          2          4
  writebacks              2        110        112
  invalidations         502        377        879
  interventions         501        377        878
  bus-wait cyc        25226      22246      47472
*** PASSED ***

The output is also saved to ${SIMDIR}/output/chipyard.harness.TestHarness.CS152Lab4MSIConfig/mt-vvadd-naive.log. The meaning of each counter is in the Counters table in the Background.

Measuring Vector-Vector Add with MI Coherence

Run the mt-vvadd-naive benchmark again, this time under MI:

cd ${SIMDIR}
make -j4 CONFIG=CS152Lab4MIConfig
make CONFIG=CS152Lab4MIConfig run-binary-fast BINARY=${TESTDIR}/mt-vvadd-naive.riscv

Questions to Consider:

Taking into account that the code is executed on a multi-core cache-coherent system, consider the naive implementation in vvadd_naive(). Every array starts on a cache-line boundary, so each line holds eight consecutive elements.

  • Performance bottlenecks: Why does the naive implementation perform poorly under both MI and MSI? Which performance counters reveal these inefficiencies? Why do invalidations and interventions occur even though the two cores never write to the same array element?
  • Load misses: Compare the load misses counters under MI and MSI. Given that neither core modifies the input arrays x and y, why does MI incur more load misses than MSI?
  • Optimization: What changes could you make to the implementation to improve performance? Consider how work is distributed between the cores and how that affects cache coherence traffic.

Optimizing Multi-Threaded Vector-Vector Add

Now that you are familiar with running benchmarks, collecting performance measurements, and switching between cache coherence protocols, your next task is to optimize vvadd for the dual-core system.

Implement your optimized version in the vvadd_opt() function in ${TESTDIR}/mt-vvadd-opt/mt-vvadd_opt.c, then rebuild the benchmarks:

cd ${TESTDIR}
make

Then, run the benchmark on the simulator with both protocols:

cd ${SIMDIR}
make CONFIG=CS152Lab4MIConfig  run-binary-fast BINARY=${TESTDIR}/mt-vvadd-opt.riscv
make CONFIG=CS152Lab4MSIConfig run-binary-fast BINARY=${TESTDIR}/mt-vvadd-opt.riscv

Submission

Use the following command to prepare your code for submission, and upload the resulting file to the Gradescope autograder. 

cd ${TESTDIR}
make zip-vvadd

Note that code outside of mt-vvadd_opt.c will be ignored. Your submitted function must only compute the result and, if needed, call barrier(nc). Before submitting, remove all debugging and performance measurement code. The autograder will assign no credit if your implementation:

  • Calls printf(), setStats(), printStats(), or exit().
  • Accesses performance counters.
  • Reads verify_data.

There is no written section for the directed portion that you need to submit.