Skip to content

Latest commit

 

History

History
347 lines (272 loc) · 12.6 KB

cEP-0029.md

File metadata and controls

347 lines (272 loc) · 12.6 KB
Raw

Support TOML as a configuration format

Metadata
cEP 29
Version 1.0
Title Support TOML as a configuration format
Authors Prajwal M mailto:[email protected]
Status Implementation Due
Type Feature

Abstract

This document describes how support for TOML will be implemented and integrated with the existing configuration system. It also describes how other features that complement the base idea will be implemented as part of this GSoC project.

Introduction

Having a configuration file is essential to make full use of coala. Having a standardized configuration format like TOML helps. coala currently has an INI style configuration. INI style configurations has certain limitations when it comes to custom sub-level parsing. The LineParser and ConfParser becomes complex and error-prone. A standardized configuration format allows developers to quickly implement new features and will allow users to write configuration files in a format that they are probably more familiar with.

This project is a step towards making coala easier to use. As a part of this project a repository of good configuration files will be created that will allow users to quickly bootstrap their projects.

TOML syntax for coala configuration

Basic TOML concepts

  • The basic building blocks of toml are key-value pairs.
  • Values can be integers, booleans, arrays, floats, date and time
  • A collection of key-value pairs form a table. A table can be used to group rules into a section.
  • Array of tables can be used to list collections. eg:- collection of friends with their attributes

coala configuration concepts

From the perspective of coala, only tables are of importance. Configuration concepts currently in use in coala:

  • Section as a collection of settings (key-value pairs)
  • Concept of inheritance in sections
  • Aspects and tastes

Keeping the TOML concepts and coala configuration concepts in mind, TOML syntax for coala has been decided.

  • Settings shall be declared as key-value pairs
  • Sections shall be declared as tables
  • Inheritance shall be supported through the use of a key called inherits. Appending of an inherited setting shall be through a key called append.
  • Aspects from a section shall be declared in the form of [section_name.aspect_name]

Aspects or settings as a group of settings

The syntax [all.python] cannot be used to depict inheritance in toml as in coafile. This is because TOML parsers consider python to be a setting in the section all. This type of syntax is more suitable when a setting itself is defined by a group of settings within a section.

[all]
a.b = 10
a.c = 20
a.d = 30
b = 45

can also be represented as

[all]
b = 10

[all.a]
b = 10
c = 20
d = 30

Since aspects are settings within a section and since aspects have their own group of settings, this syntax perfectly suits aspects.

Aspects in coafile

[all]
files = **
aspects = aspectname1, AspectName2 # case-insensitive
# defining an aspect's taste
aspectname1:aspect_taste = 80
# we can define subaspect taste through its parent
aspectname1:subaspect_taste = word1, word2, word3

is equivalent to

[all]
files = '**'
aspects = ['aspectname1', 'AspectName2'] # case-insensitive

[all.aspectname1]
aspect_taste = 80
subaspect_taste = ['word1', 'word2', 'word3']

or

[all]
files = '**'
aspects = ['aspectname1', 'AspectName2'] # case-insensitive
aspectname1.aspect_taste = 80
AspectName2.subaspect_taste = ['word1', 'word2', 'word3']

in TOML.

Inheritance

Inheritance in coafile

[all]
enabled = True
overridable = 2
ignore = vendor1/

[all.section1]
overridable = 3
ignore += vendor2/
other = some_value

is equivalent to

[all]
enabled = true
overridable = 2
ignore = 'vendor1/'

[section1]
inherits = 'all'
appends = 'ignore'
overrideable = 3
ignore = 'value2/'
other = 'some_value'

in TOML

Trying to have syntax like coafile

If inheritance were to represented as [all.python] as in coafile, then there would be no way to represent aspects or setting as a group of settings in TOML. This would also confuse users of TOML.

Implementation

The implementation shall consist of following parts

  • Infrastructure to load the toml configuration files
  • Implementation of TomlConfParser
  • Implementation of TomlConfWriter
  • Tool to convert .coafile to TOML config file.
  • Support coala-quickstart to generate config files in TOML.
  • Develop TOMLLintBear
  • A repository of good config files to allow users to easily bootstrap their projects.

Infrastructure to load the toml configuration files

Before toml files can be used to configure coala, a mechanism must exist to load them. The loaded file can then be passed into toml.load. For this a new configuration flag called coala -T shall be created. This will enable coala to run on toml config files. Also creating such a flag is highly beneficial. This way the flags -c , -F, -I and -s can also be used with respect to toml.

To achieve this, I will refactor the load_configuration function. I will not change the nature or number of function parameters and return values. This way the behaviour of the function shall remain the same. If coala -T is will call a function that will provide toml sections and targets else it will provide normal sections and targets. A function called load_toml_config_file equivalent to load_config_file will be created.

Implementation of TomlConfParser

The goal of this part is to get the OrderedDict from toml.load and convert it into sections. For each table, the individual key-value pairs have to converted to Settings. Finally these settings have to grouped into a section. The concepts of aspects and inheritance has to be accounted for. It's function is similar to the present ConfParser.

This enables coala to run on configurations written in TOML file.

It should convert

OrderedDict([('Default',
              OrderedDict([('files',
                            ['*.py',
                             'coalib/**/*.py',
                             './coala',
                             'tests/**/*.py',
                             'docs/conf.py']),
                           ('ignore',
                            ['tests/bearlib/languages/documentation/documentation_extraction_testdata/*.py',
                             'tests/collecting/collectors_test_dir/bears/incorrect_bear.py',
                             'coalib/misc/Asyncio.py']),
                           ('max_line_length', 80),
                           ('use_spaces', True)]))])

into

OrderedDict([('default',
              <Section object(aspects=None,
              contents=OrderedDict([('files', <Setting object(key='files', value='*.py, coalib/**/*.py, ./coala, tests/**/*.py, docs/conf.py', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cd780>),
              ('ignore', <Setting object(key='ignore', value='tests/bearlib/languages/documentation/documentation_extraction_testdata/*.py,\ntests/collecting/collectors_test_dir/bears/incorrect_bear.py,\ncoalib/misc/Asyncio.py,',
              origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cdbe0>),
              ('comment0', <Setting object(key='comment0', value='', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cdb70>),
              ('max_line_length', <Setting object(key='max_line_length', value='80', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cda20>),
              ('use_spaces', <Setting object(key='use_spaces', value='True', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cd898>)]), defaults=None,
              language=None, name='Default') at 0x108bdf550>)])

Implementation of TomlConfWriter

The TomlConfWriter receives sections and it has to convert it into a OrderedDict for toml.dump. This writes the sections to a TOML config file. It has a function that is opposite to the TomlConfParser. It's function is similar to the present ConfWriter.

This will enable users to use --save in TOML. It will also allow coala-quickstart to generate TOML config files.

It should convert

OrderedDict([('default',
              <Section object(aspects=None,
              contents=OrderedDict([('files', <Setting object(key='files', value='*.py, coalib/**/*.py, ./coala, tests/**/*.py, docs/conf.py', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cd780>),
              ('ignore', <Setting object(key='ignore', value='tests/bearlib/languages/documentation/documentation_extraction_testdata/*.py,\ntests/collecting/collectors_test_dir/bears/incorrect_bear.py,\ncoalib/misc/Asyncio.py,',
              origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cdbe0>),
              ('comment0', <Setting object(key='comment0', value='', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cdb70>),
              ('max_line_length', <Setting object(key='max_line_length', value='80', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cda20>),
              ('use_spaces', <Setting object(key='use_spaces', value='True', origin='/Users/userx/Documents/coala/coala/.coafile', from_cli=False, to_append=False) at 0x1092cd898>)]), defaults=None,
              language=None, name='Default') at 0x108bdf550>)])

into

OrderedDict([('Default',
              OrderedDict([('files',
                            ['*.py',
                             'coalib/**/*.py',
                             './coala',
                             'tests/**/*.py',
                             'docs/conf.py']),
                           ('ignore',
                            ['tests/bearlib/languages/documentation/documentation_extraction_testdata/*.py',
                             'tests/collecting/collectors_test_dir/bears/incorrect_bear.py',
                             'coalib/misc/Asyncio.py']),
                           ('max_line_length', 80),
                           ('use_spaces', True)]))])

into

Tool to convert .coafile to TOML config file

A new flag will be created to allow .coafile to be converted to TOML config files and vice versa.

$ coala --create-toml .coafile
$ coala --create-coafile .coafile.toml

The loaded sections from either .coafile or .coafile.toml can be supplied to:

  • TomlConfWriter to create TOML config file
  • ConfWriter to create coafile

Support coala-quickstart to generate config files in TOML.

coala-quickstart will be made to generate config files in TOML. A new option -T that will allow config files to be created in TOML.

New functions write_toml_config and write_toml_info will be created which will perform the functions similar to existing write_coafile and write_info functions but with respect to TOML.

Regarding green mode, I will create write_toml_sections function similar to write_sections for TOML. Also the function generate_green_mode_sections will be modified to accommodate for TOML config file.

Develop TOMLLintBear

coala-bears currently has bear for TOML called TOMLBear.py which uses a external linter. But it requires Go to be installed. So this bear can mainly be used only by Go users.

The task is to create TOMLLintBear in python. This way it can be used by all users without any additional language requirements as python comes installed in most computers.

The bear shall catch syntax errors in a TOML file. This shall be done in 2 passes.

  • In the first pass, the TOML file shall be validated per character
  • In the second pass, the TOML file shall be validated per line

This approach is similar to the toml library. As part of this project the bear shall be a validator and only check the validity of the TOML file. In the future when style guidelines are created formatting capability can be added.

A repository of good config files to allow users to easily bootstrap their projects.

This part aims to start a project which contains a repository of known and good coafile configurations. This will help users to quickly bootstrap their projects. A minimal version of this project shall be implemented here.