Environment

Environment#

Tool Environment Requirements#

Many of the great tools for microelectronic design are functional only through Linux environments. This leads to an inherent barrier-to-entry for designers that have not used Linux before. However, it is important to note that not all tools require a Linux flow, and it is possible to do co-design at different stages in different environments. This could be useful to distributed teams, or at different stages of a design flow between prototyping a design and full optimisation iterations.

One of the main complexities of multi-tool open source design is the version management of the tools, and making sure they all speak to each other without conflicting tool environment requirements. In piel, we provide a set of environment solutions to guarantee the shared functionality between the toolsets. This means we forcibly tie down each tool version to what we know works and that our examples pass as a result. This means that if you want to upgrade to a latest version, you will need to do the upgrade and verification that the tool compatibility passes.

The implementation of this consists of tying down the requirements of the primary dependencies of the project, but not the secondary ones. This means that we expect the secondary dependencies to resolve through the dependency management system when providing a suitable primary dependency environment.

To guarantee full functionality of piel, we provide some customised environment installations, particularly through nix in Linux.

Dependency Environment Breakdown#

This table of tools compatibility is provided should you use a specific set of toolsets in a given environment. No complete functionality guarantees are provided in this case.

Tools Compatibility#
Tool	Windows	Linux	OS X ?	Possible Integrations
`cocotb`	`iverilog`	`iverilog`, `verilator`, Cadence `xcelium`	`iverilog`	`iverilog`, `verilator`, Cadence `xcelium`
`gdsfactory`				Many
`openlane`
`hdl21`	`ngspice` & `xyce`	`ngspice` & `xyce`	`ngspice` & `xyce`	`ngspice`, `xyce`
`sax`
`thewalrus`

This is a preliminary table, I mostly develop on Linux, you need to verify your system configuration or use the recommended Docker environment. TODO I am unsure about the OS X integrations.

In the future, we will have custom installations for different types of users so they can install minimal dependencies for their use case.

Upgrading the tools versions#

TODO add the instructions of how to upgrade the tool versions accordingly. TODO add links to the tests that need to be run in order not to break compatibility between tools.

`piel CLI` (Recommended - In Active Development)#

piel is a command line interface (CLI) that is designed to be a simple and easy to use tool for managing the piel toolchain. It is necessary to first install piel in a given python environment and then the CLI tool should automatically be active in that envrionment. You then just need to run $ piel in the terminal to see the available commands.

`piel` Useful Commands#

All commands are echoed to the terminal before they are executed. This is to help you understand what is happening under the hood.

Useful `piel` commands#
Description	Command
Builds the piel project documentation. Assumes correct documentation environment requirements.	`piel develop build-docs`
Gets the install directory for the piel project.	`piel get-install-directory`
Installs `nix` on your system.	`piel environment install-nix`
Installs `openlane` on your system.	`piel environment install-openlane`
List all the piel CLI functions relating to environment configuration.	`piel environment`
List all the piel CLI functions relating to configuring a piel project development environment.	`piel develop`

`nix` Configuration (In Passive Development)#

OpenLane 2 via nix have recently released another way to package their python-driven Openlane 2 digital chip layout flow. We have previously had issues reproducibly building the docker configuration, and because most users are likely to use these tools for developing their chips rather than distributing software, nix might be well suited for these applications.

`nix` Development Installation#

This process sets up a nix development environment, good for when developing examples. Make sure to follow the nix installation instructions for your platform.

The Fast Lane

Assuming you already have piel installed in a local environment, you can simply run the commands to perform the installation.

piel environment # To see all commands
piel environment install-nix # To install nix
piel environment install-openlane # To install openlane

To enter the nix environment, run:

piel environment activate-piel-nix
# piel environment activate-openlane-nix # if you want to enter the openlane one instead

System requirements#

Before starting, make sure you system has:

curl
git
bash

`nix` installation#

The Fast Lane

Assuming you already have piel installed in a local environment, you can simply run:

piel environment install-nix

The Detailed Lane

First, install nix. The approach below is not the recommended install method, as it creates a ‘single-user’ installation, with the /nix cache owned by the invoking user, rather than shared between all users. The recommended ‘multi-user’ method, however, doesn’t work well on systems with SELinux, i.e. Fedora, so we’ll live with this alternative for now.

sh <(curl -L https://nixos.org/nix/install) --no-daemon

Provide your user password when prompted.

The following line should have been added to both your ~/.bash_profile and ~/.bashrc:

if [ -e /users/<youruser>/.nix-profile/etc/profile.d/nix.sh ]; then . /users/<youruser>/.nix-profile/etc/profile.d/nix.sh; fi

This ensures that nix-shell will be available in $PATH whether you’re starting a shell in “login” mode or in “non-login” mode. More info can be found here.

After verifying this, to update the changes to $PATH either close and reopen your terminal, or run:

. ~/.nix-profile/etc/profile.d/nix.sh

OpenLane2 installation#

The Fast Lane

Assuming you already have piel installed in a local environment, you can simply run:

piel environment install-openlane

The Detailed Lane

Before installing piel, let’s first get OpenLane2, as it will automatically also give us OpenROAD, Yosys, Magic, KLayout, and Verilator. The instructions below are copied from here.

Cachix allows the reproducible Nix builds to be stored on a cloud server so you do not have to build OpenLane’s dependencies from scratch on every computer, which will take a long time.

First, you want to install Cachix by running the following in your terminal:

nix-env -f "<nixpkgs>" -iA cachix

Then set up the OpenLane binary cache as follows:

cachix use openlane

cd to a working directory of choice, and clone down openlane2:

git clone https://github.com/efabless/openlane2

OpenLane Nix environment#

The Fast Lane

Assuming you already have piel installed in a local environment, you can simply run:

piel environment activate-openlane-nix

**The Detailed Lane - (Depreciated) Pre-Flakes Migration **

Now move inside the folder:

cd openlane2

And build the environment of dependencies. Wait for it to fetch and cache the dependencies.

nix-shell

Piel Nix environment#

The Fast Lane

Assuming you already have piel installed in a local environment and have followed the previous installation process, you can simply run:

piel environment activate-piel-nix

The Detailed Lane

We are now using nix-flakes to manage the nix environment. This is an experimental nix feature, but far more powerful than the previous nix-shell approach. To learn more about nix-flakes, see here.

To run our nix flakes environment run the following:

cd environment/nix
nix develop --extra-experimental-features nix-command --extra-experimental-features flakes

This will take some time as it is both installing the openlane2 nix dependencies and the piel ones, and building them into a specific environment. The total installation size is approximately 4 Gb. All the python packages that are dependencies of pip are installed from the wheels in PyPi from the versions defined by the poetry.lock file.

In my computer, running this command for the first time took about 20 minutes. Eventually we will distribute this in a binary.

**The Detailed Lane - (Depreciated) Pre-Flakes Migration **

We’ll similarly use nix to grab all the compiled dependencies for piel, including:

ngspice: 41 (latest)
gtkwave: 3.3.117, from Aug 2023 (latest)
Xyce: 7.6, from Nov 2022 (7.7 is latest)
verilog: 12.0, from Jun 2023 (latest)

Do do this, simply move inside of the piel directory:

cd piel/environment/nix

And run the command below, which implicitly reads in the local shell.nix file:

nix-shell

Piel Python environment#

The Fast Lane

piel environment # TODO

The Detailed Lane - Depreciated

For the time being, PyPI and pip isn’t easily compatible with Nix. See the complexity here.. machnix used to solve this problem, but it’s unmaintained/deprecated in favor of the migration to dream2nix. So hopefully in a couple months, this whole section below will be rolled into shell.nix file mentioned above.

Anyways, in the interim, we’ll use the requirements_dev.txt file to fetch all the Python dependencies for piel.

While still inside of the nix-shell, check you’re using python 3.10.9:

python --version

This is essential because we want all of our piel Python virtual environment to be on the same version as Openlane. Create and activate this venv on the top level of the .piel directory:

python -m venv ~/.piel/.venv

source ~/.piel/.venv/bin/activate

And fetch the dependencies via:

pip install -e .[develop]

The .venv folder should be created inside the top level of the .piel in your home directory.

You’re now done!

Subsequent usage - Depreciated#

cd /path/to/piel

And implicitly load the piel shell.nix environment via:

nix-shell

This will automatically add to $PATH the packages , but it will also add to path the packages managed the pip venv by the OpenLane2 nix environment. This is accomplished via a the shell hook in shell.nix:

  shellHook = ''
    export LD_LIBRARY_PATH=${pkgs.lib.makeLibraryPath [
      pkgs.stdenv.cc.cc
    ]}
      # Reinstalling the pacakges should guarantee a reproducible build every time
      set -e
      echo "Running: micromamba shell hook --shell=bash"
      eval "$(micromamba shell hook --shell=bash)"
      echo "Running: micromamba shell init --shell=bash --prefix=~/micromamba"
      micromamba shell init --shell=bash --prefix=~/micromamba
      echo "Running: micromamba create --yes -q -n pielenv -c conda-forge python=3.10"
      micromamba create --yes -q -n pielenv -c conda-forge python=3.10
      echo "Running: micromamba activate pielenv --yes"
      micromamba activate pielenv --yes
      set +e
      export PATH="$PATH:$HOME/.local/bin/"
      echo "Running: micromamba run -n pielenv pip install -r $HOME/.piel/openlane2/requirements_dev.txt --user --break-system-packages;"
      micromamba run -n pielenv pip install -r $HOME/.piel/openlane2/requirements_dev.txt --user --break-system-packages;
      echo "Running: micromamba run -n pielenv pip install ../../[develop] --user --break-system-packages;"
      micromamba run -n pielenv pip install -e "../../[develop]" --user --break-system-packages;
      source $HOME/.piel/.venv/bin/activate;
    fi
    nix-shell ~/.piel/openlane2/shell.nix
  '';
  LOCALE_ARCHIVE="/usr/lib/locale/locale-archive";  # let's nix read the LOCALE, to silence warning messages
}

VSCode support for nix#

There is a useful plugin for automatically running the top-level shell.nix file after setup. More info can be found in this blog.

Creating your own `nix` Installation - Depreciated#

You might have your own set of tools that you are using alongside piel. One common complexity in using multiple open-source tools is distributing them and guaranteeing that they run consistently between different machines.

We follow some of the principles in the Setting up the OpenLane Nix Cache page .

Relevant tools and commands:

cachix Binary Cache platform for open source and business.
nix-shell “Build the dependencies of the specified derivation, but not the derivation itself”.

You might want to clone only the maintained nix configuration of the openlane2 design flow.

Find out what is a git sparse-checkout

What we would like to do is have a version controlled installation that tracks the version controlled nix configuration in openlane 2.

git clone https://github.com/efabless/openlane2.git
cd openlane2/nix/
# TODO We want to track this directory only, how?

Useful `nix` commands#
Description	Command
Build with all available CPU cores	`nix-shell --cores 0`

`nix` Useful Commands#

Useful `nix` commands#
Description	Command
Build production shell with all available CPU cores	`nix-build --cores 0`
Build development shell with all available CPU cores	`nix-shell --cores 0`
Install a nix environment package eg. mach-nix	`nix-env -if https://github.com/DavHau/mach-nix/tarball/3.5.0 -A mach-nix`
List all available packages	`nix-env -qaP --description`
Search for a specific <package>	`nix-env -qaP <package>`

`apptainer` Configuration (In Passive Development)#

apptainer is a good open-source container management system that aims to be optimised for high performance computation. We want to have a distributed container environment where all the open-source piel toolchain is pre-installed and ready for custom design. What this distribution aims to provide is both an easy installation script for Ubuntu environments which are common in open-source development and a specific environment configuration that resolves the particular supported versions of the toolchains.

`apptainer` Installation#

The installation instructions from precompiled binaries are detailed here for both Debian, Ubuntu, and CentOS.

`apptainer` Useful Commands#

Useful `apptainer` commands#
Description	Command
Build production shell with all available CPU cores	`apptainer instance start piel.sif pielapptainer`
Build production shell with all available CPU cores	`apptainer shell instance://pielapptainer`
Build production shell with all available CPU cores	`apptainer build piel.sif piel.def`
Build production shell with all available CPU cores	`apptainer instance stop pielapptainer`

`docker` Configuration (In Passive Development)#

Linux - Docker Setup Instructions#

We aim for piel to be automatically installed with all dependencies in the docker environment provided by IIC-OSIC-TOOLS. This is an issue in progress

Start your docker environment accordingly running, for example:

cd iic-osic-tools
./start_vnc.sh

Find out which docker instance is running:

docker ps

You can start a bash terminal in the correct docker environment by running:

docker exec -it <yourdockercontainername> bash

You can explore the environment a little bit just between running ls and cd commands. If it has started from the default installation you should have started under the foss/designs folder. It is here that we will load our initial design for evaluation and interconnectivity.

Now, we can begin and follow the other tutorials.

Then go into your localhost:8888 to access Jupyter Lab directly from your Chrome notebook.

Extension of IIC-OSIC-TOOLS#

Existing Toolset:

Jupyter Notebook#

From the IIC-OSIC-TOOLS directory run:

./start_jupyter.sh

Jupyter Lab#

Now, there is no direct jupyterlab running script as of June 2023:

Equivalently, what you can do is simple:

cd icc-osic-tools
source ./start_x.sh

# In the docker environment terminal
python -m jupyterlab

And you will get a Jupyter Lab firefox instance.

However, to view our Jupytext notebooks you need to run:

pip install jupytext

Required Toolset#

As of right now, the openlane flow can only be run in a root environment.

To install jupyter lab and related plugins.

Run in the docker environment:

cd /foss/designs
pip install virtualenv
python -m venv pielenv
source pielenv/bin/activate
pip install jupyterlab

Docker Useful Commands#

You might want to interact a lot with the filesystem when running OSIC EDA tools, so here is a set of useful Docker commands that might make the experience easier.

Useful Commands Table#

Docker Commands#
Description	Command
Build a docker image in CD	`docker build .`
Build a docker image into a local tag	`docker build . -t user/app --load`
List available docker images	`docker images`
Remove all docker images	`docker system prune -a`
Remove docker image by id	`docker rmi <your-image-id>`
List running docker containers	`docker ps`
Start bash terminal on running container	`docker exec -it <containername> bash`
Stop docker container	`docker stop <containerid>`
IIC-OSIC-TOOLS sudo shell control	`./start_shell.sh`

For Developers#

Note that in the docker container, it is just an environment, it is not an independent file system. It appears like the only folder that can be edited from iic-osic-tools is the designs folder, maybe I will see how to disable this in another set of instructions. This means that, say, you want to install piel in the default docker environment you might have to run:

cd $HOME/eda/designs # The default $DESIGN environment
git clone https://github.com/daquintero/piel.git

`mamba` Recipie Configuration (In Passive Development)#

Installation#

Stable release#

To install piel, run this command in your terminal:

$ pip install piel

This is the preferred method to install piel, as it will always install the most recent stable release.

If you don’t have pip installed, this Python installation guide can guide you through the process.

From sources#

The sources for piel can be downloaded from the Github repo.

You can either clone the public repository:

$ git clone git://github.com/daquintero/piel

Or download the tarball:

$ curl -OJL https://github.com/daquintero/piel/tarball/master

Once you have a copy of the source, you can install it with:

$ pip install -e .

You might also need to run the following commands to run the examples, documentation, and full environment:

mamba install pandoc
mamba install jupyterlab jupytext
pip install -r requirements_dev.txt

Usage#

To use piel in a project you can then do:

import piel

Installation Environment Verification#

We have verified the piel, on the latest Ubuntu LTS. You can then run the above commands and the dependencies should be resolved. In the future, we will provide a Docker environment. Note that because piel is a microservice and the flow depends on multiple packages, the first import statement during indexing might take a bit of time.

On the first import, the package will create a folder in your home directory called .piel. This folder is used to manage installation requirements and guarantee reproducible behaviours of the project interactions with the filesystem with the necessary tools.

Required Project Structure#

It is very simple and follows convention. You need to structure your logic design files in a directory with this structure to follow a full-design flow.

The aim of this structure is that it allows a common interface to the conventional design process of a photonic and electronic chip run. It allows defining shared models, having source files, co-simulating, creating analysis scripts in Python, allows common open-source tool interfaces, and more.

This treats the selected design directories as parts of a python package, whilst still enabling the standard microelectronics design flow project structure.

Post Full-Flow Structure#

If you run the full flow, the design folder will look like:

design_folder_name
    docs/
        README.md # Optional: Document your design
    design_folder_name/
        __init__.py
        analogue/
            __init__.py # Analog GDSFactory top-level layout files in here.
        components/
            __init__.py # Optional: Custom layout `gdsfactory` components in here.
            photonics/
                __init__.py # `gdsfactory` components in here.
            analogue/
                __init__.py # `gdsfactory` custom analogue layout in here.
            digital/
                __init__.py # Any digital macros part of the OpenLane, `amaranth`, `cocotb` or other flows.
        io/
            pin_order.cfg # CUSTOM REQUIRED: OpenLane
        models/
            __init__.py # Optional: `gplugins` models in here.
            analogue/
                __init__.py # `hdl21 SPICE` models too.
            frequency/
                __init__.py # `sax` and related photonics models here.
            logic/
                __init__.py # `digital` behavioural models here.
            physical/
                __init__.py # `femwell`-generated models here on particular components.
            transient/
                __init__.py # Photonic and specific behavioural analogue time-domain models in here.
        photonic/
            __init__.py # Photonic GDSFactory top-level layout files in here.
        scripts/
            openlane.sh # eg. generated by piel for automation with script based tools.
        sdc/
            design.sdc # Required: OpenLane SDC timing configuration files.
        src/
            source_files.v # Digital flow source files in here.
        runs/
            openlane_run_folder # AUTO GENERATED: OpenLane v1 digital layout outputs.
        tb/
            __init__.py
            Makefile # # AUTO GENERATED: cocotb in here
            test_design.py # # AUTO GENERATED: Required cocotb
            out/
                output.csv # # AUTO GENERATED: Recommended: cocotb simulation output files go here.
        config.json # Optional, Required only for OpenLane V1
    .gitignore
    setup.py
    README.md

You need to configure the setup.py basic script so that this package matches your design requirements.

You can install and interact with the corresponding project model and some design files using:

pip install -e .

In a tapeout, this design_folder could be treated as a standalone package with version control via a .gitignore and local installation via a packaged setup.py script. Documentation could be written through Sphinx via python and verilog plugins.

Note that this structure is beneficial for a number of reasons. When your design is imported, it can be used as a directory input to any piel function. As long as you have followed the structure of the project, then you do not have to deal with filepaths, and you can automate the design and simulation operations of the co-design tools. This project structure enables compatibility in a design flow that uses all the tools in piel. This means you can do co-design of analogue, digital and photonic chips in a single project without thinking of how to structure your files or getting conflicts when you want to verify with further tools down the line.

There are several piel functions that generate this project structure for you so you can get cracking from the start. Check out the examples.

Installing the cookiecutter project#

piel provides a cookiecutter template of this project structure. It is really easy to use and the recommended way of creating a project. Run the following commands from the piel directory.

You can either install the piel development requirements to make sure you have the cookiecutter dependency installed. You can do either:

pip install -r requirements_dev.txt --user
# or
pip install cookiecutter

# In piel top directory
cd environment
cookiecutter piel_cookiecutter_template -o "<output_parent_directory_path>"

Then you just fill in the details.

Design Files Interaction#

In this project, it is assumed design and path variables are inputted from the user as strings, rather than pathlib components to maintain universality of use for most Python users.

Python-Filesystem Useful Commands#

You might want to interact a lot with the filesystem when running OSIC EDA tools, so here is a set of useful commands that might make the experience easier.

Important Libraries

import os  # Operating System Utilities
import pathlib  # Path and directory utilities
import stat  # File permissions and status
import shutil  # Shell utilities
import subprocess  # Shell commands control

Useful Commands Table#

Python File and System Operations#
Description	Command
Copy a file from a source to a destination filepaths	`shutil.copyfile(<sourcefilepath>, <destinationfilepath>)`
Change a file permission to executable	`file.chmod(file.stat().st_mode \| stat.S_IEXEC)`
Check if “examplepath” directory exists	`pathlib.Path.exists("<examplepath>")`
Get absolute PATH of current running file	`pathlib.Path(__file__).resolve()`
Get absolute PATH of the directory of current running file	`pathlib.Path(__file__).parent.resolve()`
Get current working directory PATH	`pathlib.Path(".")`
Get environment variable	`os.environ["<variablename>"]`
Get POSIX representation of PATH	`pathlib.Path("<examplepath>").as_posix()`
Get relative PATH of current running file	`pathlib.Path(__file__)`
Get string representation of PATH	`str(pathlib.Path("<examplepath>"))`
Get subpath from existing PATH	`pathlib.Path("<examplepath>") / "<subdirectory>"`
List all files and directories in PATH	`list(pathlib.Path("<examplepath>").iterdir())`
Run shell command	`subprocess.call(<examplescriptpath>)`
Set environment variable	`os.environ["<variablename>"] = "<newvalue>"`

Environment

Contents

Environment#

Tool Environment Requirements#

Dependency Environment Breakdown#

Upgrading the tools versions#

piel CLI (Recommended - In Active Development)#

piel Useful Commands#

nix Configuration (In Passive Development)#

nix Development Installation#

System requirements#

nix installation#

OpenLane2 installation#

OpenLane Nix environment#

Piel Nix environment#

Piel Python environment#

Subsequent usage - Depreciated#

VSCode support for nix#

Creating your own nix Installation - Depreciated#

nix Useful Commands#

apptainer Configuration (In Passive Development)#

apptainer Installation#

apptainer Useful Commands#

docker Configuration (In Passive Development)#

Linux - Docker Setup Instructions#

Extension of IIC-OSIC-TOOLS#

Jupyter Notebook#

Jupyter Lab#

Required Toolset#

Docker Useful Commands#

Useful Commands Table#

For Developers#

mamba Recipie Configuration (In Passive Development)#

Installation#

Stable release#

From sources#

Usage#

Installation Environment Verification#

Required Project Structure#

Post Full-Flow Structure#

Installing the cookiecutter project#

Design Files Interaction#

Python-Filesystem Useful Commands#

Useful Commands Table#

`piel CLI` (Recommended - In Active Development)#

`piel` Useful Commands#

`nix` Configuration (In Passive Development)#

`nix` Development Installation#

`nix` installation#

Creating your own `nix` Installation - Depreciated#

`nix` Useful Commands#

`apptainer` Configuration (In Passive Development)#

`apptainer` Installation#

`apptainer` Useful Commands#

`docker` Configuration (In Passive Development)#

`mamba` Recipie Configuration (In Passive Development)#