From d66df6c7fe9d57618337ad833bab9867ea2c6ab5 Mon Sep 17 00:00:00 2001 From: auphelia Date: Thu, 27 Feb 2020 16:37:17 +0000 Subject: [PATCH 1/6] [Sphinx Documentation] Extend title overline --- docs/finn/example_networks.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/finn/example_networks.rst b/docs/finn/example_networks.rst index 1abe82d9f9..975a49a5b9 100644 --- a/docs/finn/example_networks.rst +++ b/docs/finn/example_networks.rst @@ -1,8 +1,8 @@ .. _example_networks: -*************** +**************** Example Networks -*************** +**************** FINN uses `several pre-trained QNNs `_ that serve as examples and testcases. You can find a status summary below for each network. From d837aff53ce11b04db0305f7ca351c8f5c308316 Mon Sep 17 00:00:00 2001 From: auphelia Date: Thu, 27 Feb 2020 16:37:39 +0000 Subject: [PATCH 2/6] [Sphinx Documentation] Add first draft of network preparation page --- docs/finn/nw_prep.rst | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/docs/finn/nw_prep.rst b/docs/finn/nw_prep.rst index ea800b9f26..96dfcc1842 100644 --- a/docs/finn/nw_prep.rst +++ b/docs/finn/nw_prep.rst @@ -9,3 +9,44 @@ Network Preparation .. image:: /img/nw-prep.png :scale: 70% :align: center + +The main principle of FINN are analysis and transformation passes. If you like to have more information about these please have a look at section :ref:`analysis_pass` and :ref:`transformation_pass` or at chapter :ref:`tutorials` about the provided Jupyter notebooks. + +This page is about the network preparation, the flow step that comes after the :ref:`brevitas_export`. Its main idea is to optimize the network and convert the nodes to custom nodes that correspond to `finn-hlslib `_ functions. In this way we get a network that we can bring to hardware with the help of Vivado. For that we have to apply several transformations on the ONNX model, which this flow step receives wrapped in the :ref:`modelwrapper`. + +Various transformations are involved in the network preparation. The following is a short overview of these. + +Tidy-up transformations +======================= + +These transformations do not appear in the diagram above, but are applied in many steps in the FINN flow to postprocess the model after a transformation and/or prepare it for the next transformation. They ensure that all information is set and behave like a "tidy-up". These transformations are the following: + +* :py:mod:`finn.transformation.general.GiveReadableTensorNames` and :py:mod:`finn.transformation.general.GiveUniqueNodeNames` + +* :py:mod:`finn.transformation.infer_datatypes.InferDataTypes` and :py:mod:`finn.transformation.infer_shapes.InferShapes` + +* :py:mod:`finn.transformation.fold_constants.FoldConstants` + +Streamlining Transformations +============================ + +The idea behind streamlining is to eliminate floating point operations in a model by moving them around, collapsing them into one operation and transforming them into multithresholding nodes. Several transformations are involved in this step. For details have a look at the module :py:mod:`finn.transformation.streamline` and for more information on the theoretical background of this, see `this paper `_. + +After this transformation the ONNX model is streamlined and contains now custom nodes in addition to the standard nodes. At this point we can use the :ref:`verification` to simulate the model using Python and in the next step some of the nodes can be converted into HLS layers that correspond to finn_hlslib functions. + +Convert to HLS Layers +===================== + +Pairs of binary XNORPopcountMatMul layers are converted to StreamingFCLayers and following Multithreshold layers are absorbed into the MVTU. The result is a model consisting of a mixture of HLS and non-HLS layers. For more details, see :py:mod:`finn.transformation.fpgadataflow.convert_to_hls_layers`. + +Dataflow Partitioning +===================== + +In the next step the graph is split and the part consisting of HLS layers is further processed in the FINN flow. The parent graph containing the non-HLS layers remains. The PE and SIMD are set to 1 by default, so the result is a network of only HLS layers with maximum folding. The model can be verified using the *npysim* simulation. It is a simulation using C++ and is described in more detail in chapter :ref:`verification`. + +Folding +======= + +To adjust the folding, the values for PE and SIMD can be increased to achieve also an increase in the performance. The result can be verified using the same simulation flow as for the network with maximum folding (*npysim* using C++), for details please have a look at chapter :ref:`verification`. + +The result is a network of HLS layers with desired folding and it can be passed to :ref:`vivado_synth`. From a6cbe89b25d6e72ef1335ddab8cf4bd90305baea Mon Sep 17 00:00:00 2001 From: auphelia Date: Thu, 27 Feb 2020 17:30:36 +0000 Subject: [PATCH 3/6] [Sphinx Documentation] Add first draft of vivado hls and ipi page --- docs/finn/vivado_synth.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/finn/vivado_synth.rst b/docs/finn/vivado_synth.rst index 57a4213ae5..26a1f7a852 100644 --- a/docs/finn/vivado_synth.rst +++ b/docs/finn/vivado_synth.rst @@ -9,3 +9,7 @@ Vivado HLS and Vivado IPI .. image:: /img/vivado-synth.png :scale: 70% :align: center + +In this step the system is handed over to Vivado. To do this, IP blocks are created from each layer using Vivado HLS and then stitched together using Vivado IP Integrator. This creates a Vivado design of the entire network. The design can be verified using `PyVerilator `_ either on the network with the unstitched IP blocks or on the stitched IP. The generated verilog files are passed to PyVerilator and in this way the model can be emulated. This procedure is called *rtlsim* in FINN flow and details can be found in the chapter :ref:`verification`. + +Once the model is in the form of a stitched IP, it can be passed to the next flow step :ref:`pynq_deploy`. From cfd37551859716ff87754507f9b557bd0092dd29 Mon Sep 17 00:00:00 2001 From: auphelia Date: Thu, 27 Feb 2020 18:06:23 +0000 Subject: [PATCH 4/6] [Sphinx Documentation] Add first draft of page about pynq deployment --- docs/finn/pynq_deploy.rst | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/finn/pynq_deploy.rst b/docs/finn/pynq_deploy.rst index 1c850b1fb0..70bd6d7219 100644 --- a/docs/finn/pynq_deploy.rst +++ b/docs/finn/pynq_deploy.rst @@ -9,3 +9,28 @@ PYNQ Deployment .. image:: /img/pynq-deploy.png :scale: 70% :align: center + +This chapter is about the hardware generation and deployment on PYNQ. If you need more information about PYNQ, please have a look at the `PYNQ website `_. + +Create PYNQ Shell Project +========================= + +To deploy the network on A PYNQ platform, it needs to be put inside an appropriate *shell*. This *shell* bridges the network with the interfaces the underlying system exposes. This can be done using the transformation MakePYNQProject, see :py:mod:`finn.transformation.fpgadataflow.make_pynq_proj.MakePYNQProject`. + +Test on Hardware +================ + +Synthesis, Place and Route +-------------------------- + +After integrating the model into the PYNQ shell, Vivado *Synthesis, Place and Route* can be launched. The result is a bitfile which can be used for the PYNQ board. In FINN this can be done using a transformation pass. For details, please have a look at :py:mod:`finn.transformation.fpgadataflow.synth_pynq_proj.SynthPYNQProject`. + +Generate PYNQ runtime code +-------------------------- + +Additionally, a Python code is necessary to execute the model on the board. This is done by transformation pass :py:mod:`finn.transformation.fpgadataflow.make_pynq_driver.MakePYNQDriver`. + +Deployment and Remote Execution +------------------------------- + +The bitfile and the driver file(s) are copied to the PYNQ board and can be executed there using the *onnx_exec* function with the right *exec_mode* settings. For details please have a look at transformation :py:mod:`finn.transformation.fpgadataflow.make_deployment.DeployToPYNQ` and the execution function :py:mod:`finn.core.onnx_exec`. From a200df33a4b4bac9b4f93371ebc742d0511b3e28 Mon Sep 17 00:00:00 2001 From: auphelia Date: Thu, 27 Feb 2020 18:34:45 +0000 Subject: [PATCH 5/6] [Sphinx Documentation] Add first draft of verification page --- docs/finn/verification.rst | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/finn/verification.rst b/docs/finn/verification.rst index 01261dbabd..dac3721d64 100644 --- a/docs/finn/verification.rst +++ b/docs/finn/verification.rst @@ -9,3 +9,21 @@ Functional Verification .. image:: /img/verification.png :scale: 70% :align: center + +There are three ways to verify a network in FINN functionally. These ways of verification can be applied in different stages of the network inside FINN. All can be accessed using the execution function in module :py:mod:`finn.core.onnx_exec`. The execution happens in most cases node by node, which supports networks that have a mixture of standard ONNX nodes, custom nodes and HLS custom nodes. Because the single node can be executed using one of the following methods. + +Simulation using Python +======================= + +This simulation can be used right after the :ref:`brevitas_export` or when the network does not contain any HLS custom nodes, so right after the streamlining transformations and before the nodes are converted into HLS layers. + +Simulation using C++ +==================== + +This simulation can be used for a model containing several HLS custom operations. Because they are based on finn-hlslib function, C++ code can be generated from this single nodes and they can be executed by compiling the code and running the resulting executables. + + +Emulation using PyVerilator +=========================== + +The emulation using PyVerilator can be used when IP blocks were generated, either node by node or of a whole design. For that purpose PyVerilator gets the generated verilog files. From 12b732bec3a8f7417e813c967fc85a738f37198d Mon Sep 17 00:00:00 2001 From: auphelia Date: Fri, 28 Feb 2020 10:42:20 +0000 Subject: [PATCH 6/6] [Sphinx Documentation] Change introduction on verification page --- docs/finn/verification.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/finn/verification.rst b/docs/finn/verification.rst index dac3721d64..03e12fd695 100644 --- a/docs/finn/verification.rst +++ b/docs/finn/verification.rst @@ -10,7 +10,7 @@ Functional Verification :scale: 70% :align: center -There are three ways to verify a network in FINN functionally. These ways of verification can be applied in different stages of the network inside FINN. All can be accessed using the execution function in module :py:mod:`finn.core.onnx_exec`. The execution happens in most cases node by node, which supports networks that have a mixture of standard ONNX nodes, custom nodes and HLS custom nodes. Because the single node can be executed using one of the following methods. +When the network is transformed it is important to verify the functionality to make sure the transformation did not change the behaviour of the model. There are multiple ways of verification that can be applied in different stages of the network inside FINN. All can be accessed using the execution function in module :py:mod:`finn.core.onnx_exec`. The execution happens in most cases node by node, which supports networks that have a mixture of standard ONNX nodes, custom nodes and HLS custom nodes. Because the single node can be executed using one of the following methods. Simulation using Python =======================