[BoM][Adde] "KICAD" format

- This is some sort of configurable CSV
- We get the options from the project

CHANGELOG.md

@@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - `ref_range_separator` option to configure the character used for reference
     ranges
   - `use_ref_ranges` alias for `use_alt`
+  - New *kicad* format to mimic KiCad's internal BoM.
 - PCB Print: a mechanism to filter components for a particular layer (#706)
 
 ### Fixed

docs/samples/generic_plot.kibot.yaml

@@ -574,9 +574,10 @@ outputs:
       footprint_populate_values: 'no,yes'
       # [string|list(string)='SMD,THT,VIRTUAL'] {comma_sep} {L:3} Values for the `Footprint Type` column
       footprint_type_values: 'SMD,THT,VIRTUAL'
-      # [string='Auto'] [HTML,CSV,TXT,TSV,XML,XLSX,HRTXT,Auto] format for the BoM.
+      # [string='Auto'] [HTML,CSV,TXT,TSV,XML,XLSX,HRTXT,KICAD,Auto] format for the BoM.
       # `Auto` defaults to CSV or a guess according to the options.
-      # HRTXT stands for Human Readable TeXT
+      # HRTXT stands for Human Readable TeXT.
+      # KICAD is used to get the options from KiCad project. In KiCad you can configure CSV like options
       format: 'Auto'
       # [boolean=true] Connectors with the same footprints will be grouped together, independent of the name of the connector
       group_connectors: true
@@ -679,7 +680,9 @@ outputs:
       normalize_values: false
       # [number=1] Number of boards to build (components multiplier)
       number: 1
-      # [string='%f-%i%I%v.%x'] filename for the output (%i=bom). Affected by global options
+      # [string='%f-%i%I%v.%x'] filename for the output (%i=bom). The extension depends on the selected format.
+      # In the case of the **KICAD** format the extension comes from the name you selected in KiCad's
+      # internal BoM. Affected by global options
       output: '%f-%i%I%v.%x'
       # [boolean=true] Parse the `Value` field so things like *1k* and *1000* are interpreted as equal.
       # Note that this implies that *1k 1%* is the same as *1k 5%*. If you really need to group using the
@@ -691,9 +694,10 @@ outputs:
       pre_transform: '_null'
       # [string=''] A prefix to add to all the references from this project. Used for multiple projects
       ref_id: ''
-      # [string='-'] Separator used for ranges in the list of references. Used when `use_alt` is enabled
+      # [string='-'] Separator used for ranges in the list of references. Used when `use_alt` is enabled.
+      # Ignored when using the KICAD format
       ref_range_separator: '-'
-      # [string=' '] Separator used for the list of references
+      # [string=' '] Separator used for the list of references. Ignored when using the KICAD format
       ref_separator: ' '
       # [boolean=true] Sort in ascending order
       sort_ascending: true
@@ -708,7 +712,8 @@ outputs:
       # [string='millimeters'] [millimeters,inches,mils] Units used for the positions ('Footprint X' and 'Footprint Y' columns).
       # Affected by global options
       units: 'millimeters'
-      # [boolean=false] Print grouped references in the alternate compressed style eg: R1-R7,R18
+      # [boolean=false] Print grouped references in the alternate compressed style eg: R1-R7,R18.
+      # Ignored when using the KICAD format
       use_alt: false
       # [boolean=true] Use the auxiliary axis as origin for coordinates (KiCad default) (for XYRS)
       use_aux_axis_as_origin: true

docs/source/Changelog.rst

@@ -36,6 +36,7 @@ Added
    -  ``ref_range_separator`` option to configure the character used for
       reference ranges
    -  ``use_ref_ranges`` alias for ``use_alt``
+   -  New *kicad* format to mimic KiCad’s internal BoM.
 
 -  PCB Print: a mechanism to filter components for a particular layer
    (#706)

docs/source/configuration/outputs.rst

@@ -684,7 +684,7 @@ something like this:
        comment: "Bill of Materials using KiCad settings"
        type: bom
        options:
-         format: CSV
+         format: KICAD
          ignore_dnf: false
          group_not_fitted: true
          group_fields: []
@@ -694,6 +694,8 @@ something like this:
 
 Here is what the options do:
 
+- **format**: Choosing KiCad the output format will mimic the file generated by
+  KiCad. Usually you'll want to select another format.
 - **ignore_dnf**: Do Not Fit componets goes to the same list, not separated
 - **group_not_fitted**: Not fitted components are grouped with fitted components
 - **group_fields**: An empty list so they get imported from KiCad

docs/source/configuration/outputs/BoMOptions.rst

@@ -10,9 +10,10 @@ BoMOptions parameters
    this will be replaced by the list from KiCad. |br|
    In addition to all user defined fields you have various special columns, consult :ref:`bom_columns`.
 -  **csv** :index:`: <pair: output - bom - options; csv>`  [:ref:`BoMCSV parameters <BoMCSV>`] [:ref:`dict <dict>`] (default: empty dict, default values used) Options for the CSV, TXT and TSV formats.
--  **format** :index:`: <pair: output - bom - options; format>` [:ref:`string <string>`] (default: ``'Auto'``) (choices: "HTML", "CSV", "TXT", "TSV", "XML", "XLSX", "HRTXT", "Auto") format for the BoM.
+-  **format** :index:`: <pair: output - bom - options; format>` [:ref:`string <string>`] (default: ``'Auto'``) (choices: "HTML", "CSV", "TXT", "TSV", "XML", "XLSX", "HRTXT", "KICAD", "Auto") format for the BoM.
    `Auto` defaults to CSV or a guess according to the options. |br|
-   HRTXT stands for Human Readable TeXT.
+   HRTXT stands for Human Readable TeXT. |br|
+   KICAD is used to get the options from KiCad project. In KiCad you can configure CSV like options.
 -  **group_fields** :index:`: <pair: output - bom - options; group_fields>` [:ref:`list(string) <list(string)>`] (default: ``['part', 'part lib', 'value', 'footprint', 'footprint lib', 'voltage', 'tolerance', 'current', 'power']``) [:ref:`case insensitive <no_case>`]List of fields used for sorting individual components into groups.
    Components which match (comparing *all* fields) will be grouped together. |br|
    Field names are case-insensitive. |br|
@@ -31,7 +32,9 @@ BoMOptions parameters
 -  **ignore_dnf** :index:`: <pair: output - bom - options; ignore_dnf>` [:ref:`boolean <boolean>`] (default: ``true``) Exclude DNF (Do Not Fit) components.
 -  **normalize_values** :index:`: <pair: output - bom - options; normalize_values>` [:ref:`boolean <boolean>`] (default: ``false``) Try to normalize the R, L and C values, producing uniform units and prefixes.
 -  **number** :index:`: <pair: output - bom - options; number>` [:ref:`number <number>`] (default: ``1``) Number of boards to build (components multiplier).
--  **output** :index:`: <pair: output - bom - options; output>` [:ref:`string <string>`] (default: ``'%f-%i%I%v.%x'``) filename for the output (%i=bom). Affected by global options.
+-  **output** :index:`: <pair: output - bom - options; output>` [:ref:`string <string>`] (default: ``'%f-%i%I%v.%x'``) filename for the output (%i=bom). The extension depends on the selected format.
+   In the case of the **KICAD** format the extension comes from the name you selected in KiCad's
+   internal BoM. Affected by global options.
 -  **sort_style** :index:`: <pair: output - bom - options; sort_style>` [:ref:`string <string>`] (default: ``'type_value'``) (choices: "type_value", "type_value_ref", "ref", "kicad_bom") Sorting criteria.
 
    - type_value: component kind (reference prefix), then by value
@@ -118,10 +121,12 @@ BoMOptions parameters
 
 -  ``ref_id`` :index:`: <pair: output - bom - options; ref_id>` [:ref:`string <string>`] (default: ``''``) A prefix to add to all the references from this project. Used for multiple projects.
 -  ``ref_range_separator`` :index:`: <pair: output - bom - options; ref_range_separator>` [:ref:`string <string>`] (default: ``'-'``) Separator used for ranges in the list of references. Used when `use_alt` is enabled.
--  ``ref_separator`` :index:`: <pair: output - bom - options; ref_separator>` [:ref:`string <string>`] (default: ``' '``) Separator used for the list of references.
+   Ignored when using the KICAD format.
+-  ``ref_separator`` :index:`: <pair: output - bom - options; ref_separator>` [:ref:`string <string>`] (default: ``' '``) Separator used for the list of references. Ignored when using the KICAD format.
 -  ``sort_ascending`` :index:`: <pair: output - bom - options; sort_ascending>` [:ref:`boolean <boolean>`] (default: ``true``) Sort in ascending order.
 -  ``source_by_id`` :index:`: <pair: output - bom - options; source_by_id>` [:ref:`boolean <boolean>`] (default: ``false``) Generate the `Source BoM` column using the reference ID instead of the project name.
 -  ``use_alt`` :index:`: <pair: output - bom - options; use_alt>` [:ref:`boolean <boolean>`] (default: ``false``) Print grouped references in the alternate compressed style eg: R1-R7,R18.
+   Ignored when using the KICAD format.
 -  ``use_aux_axis_as_origin`` :index:`: <pair: output - bom - options; use_aux_axis_as_origin>` [:ref:`boolean <boolean>`] (default: ``true``) Use the auxiliary axis as origin for coordinates (KiCad default) (for XYRS).
 -  *use_ref_ranges* :index:`: <pair: output - bom - options; use_ref_ranges>` Alias for use_alt.
 -  ``variant`` :index:`: <pair: output - bom - options; variant>` [:ref:`string <string>`] (default: ``'_kibom_simple'``) Board variant, used to determine which components are output to the BoM.

kibot/bom/bom.py

@@ -264,7 +264,7 @@ def sort_components(self):
 
     def get_refs(self):
         """ Return a list of the components """
-        return self.cfg.ref_separator.join([c.ref for c in self.components])
+        return self.cfg._ref_separator.join([c.ref for c in self.components])
 
     def get_alt_refs(self):
         """ Alternative list of references using ranges """
@@ -274,10 +274,10 @@ def get_alt_refs(self):
             for n in self.components:
                 if n.project == sch.name:
                     S.add(n.ref_id+n.ref_prefix, _suffix_to_num(n.ref_suffix))
-            result = S.flush(self.cfg.ref_separator, self.cfg.ref_range_separator)
+            result = S.flush(self.cfg._ref_separator, self.cfg._ref_range_separator)
             if result:
                 if refs:
-                    refs += self.cfg.ref_separator
+                    refs += self.cfg._ref_separator
                 refs += result
         return refs
 
@@ -496,7 +496,7 @@ def group_components(cfg, components):
         g.sort_components()
         # Fill the columns
         g.update_fields(cfg.conv_units, cfg.bottom_negative_x, x_origin, y_origin, cfg.angle_positive,
-                        cfg.footprint_populate_values, cfg.footprint_type_values, uses_fp_info, cfg.use_alt)
+                        cfg.footprint_populate_values, cfg.footprint_type_values, uses_fp_info, cfg._use_alt)
         if cfg.normalize_values:
             g.fields[ColumnList.COL_VALUE_L] = normalize_value(g.components[0], decimal_point)
     # Sort the groups

kibot/bom/bom_writer.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
-# Copyright (c) 2020-2022 Salvador E. Tropea
-# Copyright (c) 2020-2022 Instituto Nacional de Tecnología Industrial
+# Copyright (c) 2020-2024 Salvador E. Tropea
+# Copyright (c) 2020-2024 Instituto Nacional de Tecnología Industrial
 # Copyright (c) 2016-2020 Oliver Henry Walters (@SchrodingersGat)
 # License: MIT
 # Project: KiBot (formerly KiPlot)
@@ -11,6 +11,7 @@
 This is just a hub that calls the real BoM writer:
 - csv_writer.py
 - html_writer.py
+- kicad_writer.py
 - xml_writer.py
 - xlsx_writer.py
 """
@@ -36,7 +37,7 @@ def write_bom(filename, ext, groups, headings, cfg):
     headings = [h.lower() for h in headings]
     result = False
     # CSV file writing
-    if ext in ["csv", "tsv", "txt", "hrtxt"]:
+    if ext in ["csv", "tsv", "txt", "hrtxt", "kicad"]:
         result = write_csv(filename, ext, groups, headings, head_names, cfg)
     elif ext in ["htm", "html"]:
         result = write_html(filename, groups, headings, head_names, cfg)

kibot/bom/csv_writer.py

@@ -6,9 +6,14 @@
 # Project: KiBot (formerly KiPlot)
 # Adapted from: https://github.com/SchrodingersGat/KiBoM
 """
-CSV Writer: Generates a CSV, TSV or TXT BoM file.
+CSV Writer: Generates a CSV, TSV, TXT or HRTXT BoM file.
+            Can also mimic KiCad BoM output
 """
+from ..gs import GS
 import csv
+from .. import log
+
+logger = log.get_logger()
 ALIGN_CODE = {'right': '>', 'left': '<', 'center': '^'}
 
 
@@ -127,6 +132,16 @@ def dummy():
     pass
 
 
+def process_special_chars(row, keep_line_breaks, keep_tabs):
+    if keep_line_breaks and keep_tabs:
+        return
+    for index in range(len(row)):
+        if not keep_line_breaks:
+            row[index] = row[index].replace('\n', '')
+        if not keep_tabs:
+            row[index] = row[index].replace('\t', '')
+
+
 def write_csv(filename, ext, groups, headings, head_names, cfg):
     """
     Write BoM out to a CSV file
@@ -137,40 +152,52 @@ def write_csv(filename, ext, groups, headings, head_names, cfg):
     cfg = BoMOptions object with all the configuration
     """
     is_hrtxt = ext == "hrtxt"
+    is_kicad = ext == "kicad"
+
     ops = cfg.hrtxt if is_hrtxt else cfg.csv
+    # KiCad BoM options
+    kops = GS.load_pro_bom_fmt_settings() if is_kicad else None
     # Delimiter is assumed from file extension
     # Override delimiter if separator specified
     if is_hrtxt or (ext == "csv" and ops.separator):
         delimiter = ops.separator
+    elif is_kicad:
+        delimiter = kops.get("field_delimiter", ",")
     else:
         if ext == "csv":
             delimiter = ","
         elif ext == "tsv" or ext == "txt":
             delimiter = "\t"
 
     quoting = csv.QUOTE_MINIMAL
-    if hasattr(ops, 'quote_all') and ops.quote_all:
+    if is_kicad or (hasattr(ops, 'quote_all') and ops.quote_all):
         quoting = csv.QUOTE_ALL
+    quotechar = kops.get("string_delimiter", '"') if is_kicad else '"'
+
+    # KiCad has a couple of options to remove \t and \n
+    keep_line_breaks = kops.get("keep_line_breaks", False) if is_kicad else True
+    keep_tabs = kops.get("keep_tabs", False) if is_kicad else True
 
     with open(filename, "wt") as f:
         if is_hrtxt:
             writer = HRTXT(f, delimiter=delimiter, hsep=ops.header_sep, align=ops.justify)
         else:
-            writer = csv.writer(f, delimiter=delimiter, lineterminator="\n", quoting=quoting)
+            writer = csv.writer(f, delimiter=delimiter, lineterminator="\n", quoting=quoting, quotechar=quotechar)
         write_sep = writer.add_sep if is_hrtxt else dummy
         # Headers
-        if not ops.hide_header:
+        if not ops.hide_header or is_kicad:
             writer.writerow(head_names)
             write_sep()
         # Body
         for group in groups:
             if cfg.ignore_dnf and not group.is_fitted():
                 continue
             row = group.get_row(headings)
+            process_special_chars(row, keep_line_breaks, keep_tabs)
             writer.writerow(row)
         write_sep()
         # PCB info
-        if not (ops.hide_pcb_info and ops.hide_stats_info):
+        if not (ops.hide_pcb_info and ops.hide_stats_info) and not is_kicad:
             # Add some blank rows
             for _ in range(5):
                 writer.writerow([])

kibot/out_bom.py

@@ -468,24 +468,29 @@ def __init__(self):
                 The combination between the default for this option and the defaults for the filters provides
                 a behavior that mimics KiBoM default behavior """
             self.output = GS.def_global_output
-            """ *filename for the output (%i=bom)"""
+            """ *filename for the output (%i=bom). The extension depends on the selected format.
+                In the case of the **KICAD** format the extension comes from the name you selected in KiCad's
+                internal BoM """
             self.format = 'Auto'
-            """ *[HTML,CSV,TXT,TSV,XML,XLSX,HRTXT,Auto] format for the BoM.
+            """ *[HTML,CSV,TXT,TSV,XML,XLSX,HRTXT,KICAD,Auto] format for the BoM.
                 `Auto` defaults to CSV or a guess according to the options.
-                HRTXT stands for Human Readable TeXT """
+                HRTXT stands for Human Readable TeXT.
+                KICAD is used to get the options from KiCad project. In KiCad you can configure CSV like options """
             # Equivalent to KiBoM INI:
             self.ignore_dnf = True
             """ *Exclude DNF (Do Not Fit) components """
             self.fit_field = 'config'
             """ {no_case} Field name used for internal filters (not for variants) """
             self.use_alt = False
-            """ Print grouped references in the alternate compressed style eg: R1-R7,R18 """
+            """ Print grouped references in the alternate compressed style eg: R1-R7,R18.
+                Ignored when using the KICAD format """
             self.use_ref_ranges = None
             """ {use_alt} """
             self.ref_separator = ' '
-            """ Separator used for the list of references """
+            """ Separator used for the list of references. Ignored when using the KICAD format """
             self.ref_range_separator = '-'
-            """ Separator used for ranges in the list of references. Used when `use_alt` is enabled """
+            """ Separator used for ranges in the list of references. Used when `use_alt` is enabled.
+                Ignored when using the KICAD format """
             self.columns = BoMColumns
             """ *[list(dict)|list(string)=?] List of columns to display.
                 One entry can be just the name of the field (a string).
@@ -636,6 +641,9 @@ def _get_columns():
     def _guess_format(self):
         """ Figure out the format """
         if self.format == 'Auto':
+            # If we use KiCad sorting assume we also want the format defined in KiCad
+            if self.sort_style == 'kicad_bom':
+                return 'kicad'
             # If we have HTML options generate an HTML
             if self.get_user_defined('html'):
                 return 'html'
@@ -753,7 +761,18 @@ def config(self, parent):
         super().config(parent)
         self._format = self._guess_format()
         self._expand_id = 'bom'
-        self._expand_ext = 'txt' if self._format == 'hrtxt' else self._format
+        if self._format == 'kicad':
+            kops = GS.load_pro_bom_fmt_settings()
+            self._expand_ext = os.path.splitext(GS.pro_bom_export_filename)[1]
+            self._expand_ext = self._expand_ext[1:] if self._expand_ext else 'txt'
+            self._ref_separator = kops.get('ref_delimiter', ',')
+            self._ref_range_separator = kops.get('ref_range_delimiter', '')
+            self._use_alt = self._ref_range_separator != ''
+        else:
+            self._expand_ext = 'txt' if self._format == 'hrtxt' else self._format
+            self._ref_separator = self.ref_separator
+            self._ref_range_separator = self.ref_range_separator
+            self._use_alt = self.use_alt
         # Variants, make it an object. Do it early because is needed by other initializations (i.e. title)
         self._normalize_variant()
         # Do title %X and ${var} expansions on the BoMLinkable titles

tests/GUI/outputs

@@ -806,6 +806,20 @@
           ],
           null
         ],
+        [
+          "ref_separator",
+          [
+            "DataTypeString"
+          ],
+          null
+        ],
+        [
+          "ref_range_separator",
+          [
+            "DataTypeString"
+          ],
+          null
+        ],
         [
           "columns",
           [
@@ -966,13 +980,6 @@
           ],
           null
         ],
-        [
-          "ref_separator",
-          [
-            "DataTypeString"
-          ],
-          null
-        ],
         [
           "html",
           [