Background

The Core

This lab runs on the same Sodor 5-stage core you used in Lab 1; refer to that lab for the pipeline. The core is deliberately uninteresting here, so that everything you measure is attributable to the memory system.

The Sodor Memory Hierarchy

In Lab 1, Sodor had no memory hierarchy at all: every load and store completed in the same cycle out of a tile-local scratchpad. For this lab the data side of the core has been given a memory system, shown below.

The Lab 2 memory subsystem

The core has a single data port, and a couple of components sit behind it. Every time the core issues a load or a store, the D port router decides where that access goes: it looks at the address and sends it one of two ways.

Ordinary addresses — your arrays, your stack, anything the program actually computes on — go to the L1 data cache. One small reserved range of addresses goes to the counter block instead. Each address in that range is wired directly to one hardware counter, so reading that address with a plain lw hands you the count: cycles, loads, stores, hits, misses, or writebacks. Giving a piece of hardware an address, so that software can reach it with an ordinary load, is called memory-mapped I/O (MMIO). Nothing is stored at those addresses — they are not memory — but from the program’s point of view they are read exactly like memory. And because such an access is routed away from the cache, reading a counter perturbs neither the cache nor the counts it reports.

Those counters are how you will measure everything in this lab: the benchmark reads them itself and prints them at the end of each run.

The L1 data cache is blocking, write-back with write-allocate, and replaces with true LRU. Its geometry, sets x ways x line size, is fixed when the design is elaborated, so each design point in this lab is a separate build of the simulator. A miss goes out to the fixed-latency DRAM model, which answers after a fixed number of cycles. The snoop port ensures the host’s view of memory is consistent with the cache’s view.

A few things about this hierarchy are worth knowing before you measure it:

  • Only the D side is cached. Instruction fetch bypasses all of this, so no counter reflects fetch.

  • There is no L2, no TLB, no virtual memory, and no MSHRs. One cache, one line in flight, and the core stalls until a miss is serviced.

  • You do not have to read the RTL. It lives in ${LAB2ROOT}/generators/chipyard/src/main/scala/cs152/ and is short enough to skim if you are curious, but no exercise here requires reading or modifying it.

Directed Portion

Setup

This lab assumes you have already set up your environment in Lab 0, in particular the Miniforge install that puts conda in your home directory. You do not need to re-create the conda environment or rebuild either RISC-V toolchain: both are shared across the class, and you link to them. The kernels in this lab are 32-bit bare-metal programs, so they are built by a separate riscv32-unknown-elf-gcc; the environment script below puts it on your PATH. You do need to clone this lab’s branch and initialize its submodules, which is what the one-time setup below does.

First-time setup

Run this once. The clone has no submodules, and the RTL build needs most of them, so the build-setup.sh step is not optional.

cd ~
git clone -b fa26-lab2 https://github.com/cs152-teach/chipyard-cs152.git cs152-lab2-fa26
cd cs152-lab2-fa26
ln -sfn /home/ff/cs152/fa26/chipyard/.conda-env .conda-env
source ~/conda/etc/profile.d/conda.sh
conda activate .conda-env/
./build-setup.sh riscv-tools --skip-conda --skip-toolchain --skip-circt --skip-firesim --skip-marshal

Every new shell

After the first-time setup, source the environment script for a quick start. Update the ~/cs152-env.sh you created in Lab 0:

cat > ~/cs152-env.sh << 'EOF'
# CS152 lab environment. Usage: source ~/cs152-env.sh [lab-directory]
# With no argument, enters the chipyard tree used by Labs 0 and 1.
source ~/conda/etc/profile.d/conda.sh
cd ~/"${1:-chipyard}"
conda activate .conda-env/
export LAB2ROOT=$PWD
export TESTDIR=${LAB2ROOT}/lab
export SIMDIR=${LAB2ROOT}/sims/verilator
source ./env.sh
# The lab 2 kernels are rv32; this is the toolchain that builds them.
export PATH="$PATH:/home/ff/cs152/bin/riscv-gnu-toolchain/riscv32/bin"
EOF

Then, for new shells, run the following to enter the lab environment:

source ~/cs152-env.sh cs152-lab2-fa26

L1 Data Cache Associativity Sweep

In this exercise you will hold the total capacity of the L1 data cache fixed, vary its associativity, and explain what the counters do in response.

The workload

The benchmark is rsort, a radix sort taken from the RISC-V test suite. Build it:

cd ${TESTDIR}/directed
make

This produces rsort.riscv. Radix sort is a useful workload here because its bucket scatter spreads writes across a working set several times the size of the cache, and because it writes nearly as much as it reads — so both misses and writebacks are worth watching.

The benchmark measures itself. setStats(1) flushes the cache and zeroes the counters; setStats(0) reads them back and prints them. Everything outside that bracket (the setup pass, the verification scan, and all host I/O) is excluded, so what you get is the sort and nothing else.

Running a design point

Each geometry is a named CONFIG; the simulator is built once per config. Taking the 2-way point as the example:

cd ${SIMDIR}
make -j4 CONFIG=CS152Lab2Config2Way

Then run the benchmark:

make CONFIG=CS152Lab2Config2Way run-binary-fast BINARY=${TESTDIR}/directed/rsort.riscv

The output is also saved to ${SIMDIR}/output/chipyard.harness.TestHarness.CS152Lab2Config2Way/rsort.log.

Reading the counters

At the end of the region of interest the benchmark prints:

=== CS152 L1D counters (region of interest, D-side only) ===
  cycles      : 577479
  loads       : 33803
  stores      : 26632
  hits        : 52501
  misses      : 7934
  writebacks  : 5893
  miss rate   : 13.12 %
*** PASSED ***

That block is from the 64-set, 2-way point. Your cycles, hits, misses, and writebacks will differ at the other geometries; loads and stores will not. The bare-metal runtime has no floating point — printf supports the integer conversions but not %f — so anything you compute on the target has to be formatted as an integer.

The sweep

Sweep associativity from 1 to 8 at a constant 4 KiB, halving the sets each time you double the ways. All four design points are already provided as named configs at the bottom of ${LAB2ROOT}/generators/chipyard/src/main/scala/cs152/CS152SodorConfigs.scala, so there is no Scala to edit:

CONFIG ways sets line size capacity
CS152Lab2Config1Way 1 128 32 B 4 KiB
CS152Lab2Config2Way 2 64 32 B 4 KiB
CS152Lab2Config4Way 4 32 32 B 4 KiB
CS152Lab2Config8Way 8 16 32 B 4 KiB

Build and run each of the four exactly as above, substituting its name for CS152Lab2Config2Way.

Running the four points in parallel

Done one after another, the sweep is four rebuilds plus four simulations, which is around half an hour of waiting. You can run one of the configs by itself first, then you can run the other configs in parallel. This is because every config in your clone is generated from a single shared file, .classpath_cache/chipyard.jar, which does not exist until the first build creates it. If several first builds start together, they all write that jar at once and corrupt it, and the builds will fail.

[error] java.util.zip.ZipException: zip END header not found
make: *** [.../common.mk:140: .../.classpath_cache/chipyard.jar] Error 1

A build is keyed by its CONFIG name, not by the parameters inside it, and everything a design point produces lands in a path containing that name:

generated-src/chipyard.harness.TestHarness.<CONFIG>/     the generated RTL
simulator-chipyard.harness-<CONFIG>                      the simulator binary
output/chipyard.harness.TestHarness.<CONFIG>/rsort.log   the run output

Because the four configs have four different names, none of that collides, which is why you never have to copy a counter block out of the terminal or name a log file yourself. The four results simply sit side by side when you are done:

output/chipyard.harness.TestHarness.CS152Lab2Config1Way/rsort.log
output/chipyard.harness.TestHarness.CS152Lab2Config2Way/rsort.log
output/chipyard.harness.TestHarness.CS152Lab2Config4Way/rsort.log
output/chipyard.harness.TestHarness.CS152Lab2Config8Way/rsort.log

Part 1

Record the counters for all four design points — cycles, loads, stores, hits, misses, writebacks, and miss rate — in a single table. Remember: do not provide raw dumps of rsort.log. Miss rate against associativity is worth plotting as well. Then, answer the following questions:

  • How do the hit, miss, and writeback counts change across the sweep? Do the writebacks track the misses?

  • This sweep can only be moving one of the three kinds of miss. Say which one, and rule the other two out from the way the experiment is set up rather than from the numbers. Then say what the counters alone cannot tell you about how large that category was to begin with, and what extra measurement would settle it.

  • At what associativity do the returns flatten out, and what does that tell you about the kernel’s access pattern?

  • Capacity is held fixed, yet associativity is not free. What does more associativity cost in hardware, and what else changes about the cache when you take ways up and sets down?

Feedback Portion

In order to improve the labs for the next offering of this course, we would like your feedback. This is no longer part of your report: instead, fill out the Lab 2 feedback form. Every student should submit the form individually, including the questions about your team. It asks how long each portion took you, whether the lab was boring, what you learned, what you would change, and how the work was divided within your team.

Feel free to write as much or as little as you prefer (a point will be deducted only if the form is left completely empty).