mirror of https://github.com/Nonannet/copapy.git
readme updated
This commit is contained in:
parent
9f77ef5642
commit
5bdd77db91
50
README.md
50
README.md
|
|
@ -1,22 +1,31 @@
|
||||||
# Copapy
|
# Copapy
|
||||||
|
Copapy is a python framework for deterministic low latency realtime computations, targeting hardware applications - for example in the field of robotics, aerospace, embedded systems and control systems in general.
|
||||||
|
|
||||||
Copapy is a python based embedded domain specific language (eDSL) with copy & patch compiler. It uses the python interpreter for compilation. It generates a directed graph of variables and operations. The compiler generates machine code by composing pre-compiled stencils derived from compiled C code.
|
GPU frameworks like PyTorch, JAX and TensorFlow jump started the development in the field of AI. With the right balance of flexibility and performance they allow for fast iterations of new ideas while being performant enough to test them or even use them in production.
|
||||||
|
|
||||||
The Project targets applications that profit from fast implementation (e.g. prototyping) and require low latency realtime execution as well as minimizing risk of implementation errors not caught during compile time. This applies primarily to applications interfacing hardware, where runtime errors might lead to physical damage - for example in the field of robotics, aerospace, embedded systems and control systems in general.
|
This is exactly what Copapy is aiming for - but in the field of embedded realtime computation. While making use of the ergonomics of Python, the tooling and the general Python ecosystem, Copapy runs seamlessly optimized machine code. Despite being highly portable, the **copy and patch** compiler allows for effortless and fast deployment, without any dependencies beyond Python. It's designed to feel like writing python scripts, with a flat learning curve. But under the hood it produces high performance static typed and memory save code with a minimized set of possible runtime errors[^1]. To maximize productivity the framework provides detailed type hints to catch most errors even before compilation.
|
||||||
|
|
||||||
The language aims to be:
|
Embedded systems comes with a variety of CPU architectures. The **copy and patch** compiler already supports the most common ones [^3] and porting it to new architectures is effortless if a C compiler for the target architecture is available [^2]. The generated code depends only on the CPU architecture. The actual generated code does neither do system calls nor calling external libraries like libc. This allows Copapy for one to be highly deterministic and for the other it makes targeting different realtime operating systems or bare metal straight forward.
|
||||||
|
|
||||||
|
The summarized main features are:
|
||||||
- Fast to write & easy to read
|
- Fast to write & easy to read
|
||||||
- Type safe
|
- Memory and type safety, minimal set of runtime errors [^1]
|
||||||
- Having a predictable runtime
|
- deterministic execution
|
||||||
- No runtime errors
|
- Auto grad for efficient realtime optimizations
|
||||||
|
- Optimized machine code for the target architectures x68_64, Aarch64 and ARMv7 [^3]
|
||||||
|
- Very portable to new architectures [^2]
|
||||||
|
- Small python package, minimal dependencies, no cross compile toolchain required
|
||||||
|
|
||||||
Because the language is an embedded language, it can relay heavily on **python tooling**. While copapy is static typed, it uses Python to derive types during compile time wherever possible. It can get full benefit from python type checkers, to catch type errors even before compilation to improve ergonomics.
|
## Current state
|
||||||
|
While obviously hardware IO is a core aspect, this is not yet available. Therefore this package is at the moment a proof of concept with limited direct use. However the computation part is fully working and available for testing and playing with it by simply installing the package. At this point the project is quite close to being ready for integration into the first demonstration hardware platform.
|
||||||
|
|
||||||
## How it works
|
Currently worked on:
|
||||||
The **Compilation** step starts with tracing the python code to generate an acyclic directed graph (DAG) of variables and operations. The DAG can be optimized and gets than linearized to a sequence of operations. Each operation gets mapped to a pre-compiled stencil, which is a piece of machine code with placeholders for memory addresses. The compiler generates patch instructions to fill the placeholders with the correct memory addresses. The binary code build from the stencils, data for constants and the patch instructions are than passed to the Runner for execution. The runner allocates memory for the code and data, applies the patch instructions to correct memory addresses and finally executes the code.
|
- Array stencils for handling very large arrays and generate SIMD optimized code - e.g. for machine vision and neural network applications.
|
||||||
|
- For targeting Crossover‑MCUs, support for Thumb instructions required by ARM*-M is on the way.
|
||||||
|
- Constant-regrouping for symbolic optimization of the computation graph.
|
||||||
|
|
||||||
## Getting started
|
## Getting started & example
|
||||||
To install copapy, you can use pip:
|
To install copapy, you can use pip. Precompiled wheels are available for Linux (x86_64, Aarch64 and ARMv7), Windows (x86_64) and Mac OS (x86_64, Aarch64):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install copapy
|
pip install copapy
|
||||||
|
|
@ -47,10 +56,13 @@ print("Result d:", tg.read_value(d))
|
||||||
print("Result e:", tg.read_value(e))
|
print("Result e:", tg.read_value(e))
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
The **Compilation** step starts with tracing the python code to generate an acyclic directed graph (DAG) of variables and operations. The DAG can be optimized and gets than linearized to a sequence of operations. Each operation gets mapped to a pre-compiled stencil, which is a piece of machine code with placeholders for memory addresses. The compiler generates patch instructions to fill the placeholders with the correct memory addresses. The binary code build from the stencils, data for constants and the patch instructions are than passed to the runner for execution. The runner allocates memory for the code and data, applies the patch instructions to correct memory addresses and finally executes the code.
|
||||||
|
|
||||||
## Developer Guide
|
## Developer Guide
|
||||||
Contributions are welcome, please open an issue or submit a pull request on GitHub.
|
Contributions are welcome, please open an issue or submit a pull request on GitHub.
|
||||||
|
|
||||||
To get started with developing the package, first clone the repository to your local machine using Git:
|
To get started with developing the package, first clone the repository using Git:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/Nonannet/copapy.git
|
git clone https://github.com/Nonannet/copapy.git
|
||||||
|
|
@ -70,13 +82,13 @@ Build and install the package and dev dependencies:
|
||||||
pip install -e .[dev]
|
pip install -e .[dev]
|
||||||
```
|
```
|
||||||
|
|
||||||
If the build fails because you have no suitable c compiler installed, you can either install a compiler or use the binary from pypi:
|
If the build fails because you have no suitable c compiler installed, you can either install a compiler (obviously) or use the binary from pypi:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install copapy[dev]
|
pip install copapy[dev]
|
||||||
```
|
```
|
||||||
|
|
||||||
When running pytest it will use the binary from pypi but the local python code gets executed from the local repo.
|
When running pytest it will use the binary part from pypi but all the python code gets executed from the local repo.
|
||||||
|
|
||||||
For running all tests you need the stencil object files and the compiled runner. You can download the stencils and binary runner from GitHub or build them with gcc yourself.
|
For running all tests you need the stencil object files and the compiled runner. You can download the stencils and binary runner from GitHub or build them with gcc yourself.
|
||||||
|
|
||||||
|
|
@ -92,12 +104,6 @@ To build the binaries from source on Linux run:
|
||||||
bash tools/build.sh
|
bash tools/build.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
The runner (without the stencils) can be build on windows with:
|
|
||||||
|
|
||||||
```
|
|
||||||
tools\build
|
|
||||||
```
|
|
||||||
|
|
||||||
Ensure that everything is set up correctly by running the tests:
|
Ensure that everything is set up correctly by running the tests:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -106,3 +112,7 @@ pytest
|
||||||
|
|
||||||
## License
|
## License
|
||||||
This project is licensed under GPL - see the [LICENSE](LICENSE) file for details.
|
This project is licensed under GPL - see the [LICENSE](LICENSE) file for details.
|
||||||
|
|
||||||
|
[^1]: Currently errors like divide by zero are possible. The feasibility of tacking value ranges in the type system is under investigation to be able to do checks at compile time.
|
||||||
|
[^2]: The compiler must support TCO (tail call optimization). Currently gcc as C compiler is supported. Porting to a new architecture requires to implement a subset of relocation types used by the architecture.
|
||||||
|
[^3]: Supported are x68_64, Aarch64, ARMv7 (non-Thumb); ARMv6/7-M (Thumb) is under development; code for x68 32 Bit is present but has open issues (low priority).
|
||||||
Loading…
Reference in New Issue