Skip to content

Occamy Manycore System

Based on the Manticore architecture. The system itself is described in System Overview.

The Occamy system is located in hw/system/occamy. Run the following command to get an overview of all available Makefile targets:

make help

Hardware Generation

Go to the occamy directory, where most of your efforts will take place:

cd hw/system/occamy

Note: from now on, assume all the path to be relative to hw/system/occamy.

Occamy is generated based on the config file src/occamy_cfg.hjson. In addition many peripherals are based on memory-mapped Register-files generated by the Reggen tool. To generate all HW and SW sources, run the following command:

make all

In case you want to run a smaller Occamy configuration with, e.g. only 1 snitch cluster (for faster RTL simulation), you can open ./src/occamy_cfg.hjson, and reduce nr_s1_quadrant and nr_clusters (e.g. both to 1). To make the changes effective, you can run again the previously mentioned command:

make all

Then compile the hardware, e.g. for Questasim (see setup-iis.md for the needed environment to run the simulation on the ETH IIS machines):

# Compile the RTL for Questasim
make bin/occamy_top.vsim

# Compile the RTL for VCS
make bin/occamy_top.vcs

This make target compiled the RTL simulation model in ./work-vsim and the frontend server (fesvr) C++ sources into ./work. It also generated a script ./bin/occamy_top.vsim or ./bin/occamy_top.vcs (you can read the vsim file, the vcs is a binary) that you can use to start a Questasim or VCS session initialized with the ELF of the app/kernel you want to run. This script relies on the fesvr utilities to connect to the RTL simulation and load your ELF program into the simulated DRAM memory.

Simulation of SW Kernels

All Occamy software is currently compiled with CMake so you might also want to look into ./sw/CMakeLists.txt in this directory to see which applications or tests are included. In general you can run the following commands to build applications:

mkdir sw/build
cd sw/build
cmake ..
make

Cmake tests can be run with:

make tests

You can now simulate a single binary as follows:

# Questasim example
#bin/occamy_top.vsim <path-to-riscv-binary>
bin/occamy_top.vsim sw/build/snRuntime/test-snRuntime-simple

# VCS example
bin/occamy_top.vcs sw/build/snRuntime/test-snRuntime-simple

Makefile Structure

Run the following command to get an overview of all available Makefile targets:

make help

Utility Targets

In general, the Makefile includes the Makefrag in util/Makefrag at the root of this repository. In this util directory also contains scripts to analyze and visualize traces generated by the RTL simulations. Have a look at the following utility Makefile targets (you can execute them after running RTL simulation):

  • traces Generate the better readable traces in .logs/trace_hart_.txt with spike-dasm
  • annotate Annotate the better readable traces in .logs/trace_hart_.s with the source code related with the retired instructions.

In addition, we have some general clean targets:

  • clean Clean everything except traces in logs directory.
  • clean.logs Delete all traces in logs director

HW and Source Generation

  • update-source Update all SW and HW related sources (by, e.g., re-generating the RegGen registers and their c-header files).

Simulator Compilations

  • bin/occamy_top.vcs Build compilation script and compile all sources for VCS simulation.
  • @IIS: run the command as follows: vcs-2020.12 make bin/occamy_top.vcs

  • clean.vcs Clean all build directories and temporary files for VCS simulation.

  • bin/occamy_top.vlt Build compilation script and compile all sources for Verilator simulation.

  • clean.vlt Clean all build directories and temporary files for Verilator simulation.

  • bin/occamy_top.vsim Build compilation script and compile all sources for Questasim simulation.

  • @IIS: QUESTA_HOME=/usr/pack/modelsim-10.7b-kgf/questasim/ CC=/gcc-5.3.0-linux_x86_64/bin/gcc CXX=/gcc-5.3.0-linux_x86_64/bin/g++ LD=/gcc-5.3.0-linux_x86_64/bin/ld make bin/occamy_top.vsim
  • clean.vsim Clean all build directories and temporary files for Questasim simulation.

Addressmap Generation

The following command generate various graphics of Occamy's Address Map based on the config file src/occamy_cfg.hjson. Therefore, the commands run in the background the all target.

  • update-addrmap Generate the addressmap in the following formats: MD, TEX.
  • addrmap Generate the addressmap in the following formats: MD, PNG, TEX, PDF.
  • addrmap.tex Generate the TEX addrmap source.
  • addrmap.md Generate the MD addrmap graphic.
  • addrmap.png Generate the PNG addrmap graphic.
  • addrmap.pdf Generate the PDF addrmap graphic.
  • clean.addrmap Delete all addrmap outputs.
  • clean.addrmap.pdf Delete all Latex outputs when generating the addrmap.pdf from addrmap.tex

Software

The runtime and tests can be compiled as follows:

make DEBUG=ON update-sw

The DEBUG flag is used to include debugging symbols in the binaries, and can be omitted if this is not required. It is required if you later want to annotate the traces.

Running

You can run a Snitch binary on the simulator by passing it as a command-line argument to bin/occamy_top, for example:

bin/occamy_top.vsim sw/build/<some test>

SW Build

Depending on which toolchain you want to use and whether you target banshee (our instruction-accurate emulator) you cann add the following flags when calling cmake: Interesting CMake options that can be set via -D<option>=<value>: Each simulation will generate a unique trace file for each hart in the system. The trace file can be disassembled to instruction mnemonics by using the traces target.

  • CMAKE_TOOLCHAIN_FILE: The compiler toolchain configuration to use. Acceptable values:
    • toolchain-gcc for a GNU tolchain
    • toolchain-llvm for a LLVM/Clang toolchain (coming soon)
    • Your own custom <toolchain>.cmake file; see ${SNITCH_REPO_ROOT}/cmake/toolchain-gcc.cmake for reference
  • SNITCH_SIMULATOR: Path to simulation binary absolute or relaive from the sw/build/ directory such as ../../../../../hw/system/snitch_cluster/bin/snitch_cluster.vsim
  • SNITCH_BANSHEE: The banshee simulator binary to use for test execution.
  • BUILD_TESTS: Build Cmake tests. Can be ON or OFF

The following Makefile targets do set these command already:

  • sw Build SW into sw/build with the LLVM.
  • sw.vcs Build SW into sw/build with the LLVM (incl. tests) for VCS simulator.
  • sw.vlt Build SW into sw/build with the LLVM (incl. tests) for Verilator simulator.
  • sw.vsim Build SW into sw/build with the LLVM (incl. tests) for Questasim simulator.
  • clean.sw Delete sw/build.

The following command build and run all specified cmake tests with the corresponding simulator:

  • sw.test.vcs Build SW and run all tests with VCS simulator.
  • sw.test.vlt Build SW and run all tests with Verilator simulator.
  • sw.test.vsim Build SW and run all tests with Questasim simulator.

FPGA

For the FPGA build flow have a look at the directory fpga. The following Makefile target basically enters this directory and builds Occamy's FPGA version:

  • fpga Build a small Occamy version (CVA6 + 1xcluster) for the VCU128 FPGA board. In addition to generating readable traces, the above command also dumps several performance metrics to file for each hart. These can be collected into a single CSV file with the following target:

    make perf-csv

Among these performance metrics are start and end times of particular regions marked in the traces (via mcycle CSR reads). It can sometimes be useful to visualize these regions in a timeline. You can use the util/trace/eventvis.py tool to generate a JSON file starting from a CSV file (similar to the output of the previous command) which can be visualized in a Chrome browser at chrome://tracing. A detailed description of the expected CSV file format can be found in the tool's source code.

A source-code annotated trace can be generated using the annotate target. The Snitch binary with the debugging symbols should be passed to the target:

make BINARY=sw/build/sn_<some test>.elf annotate

Notes

All Snitch cores are initially isolated and are not able to fetch instructions from the bootrom. The cva6 manager core de-isolates the Snitch cores during booting. After that the manager core is trapped in an exception loop.