Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement #128: runtime-editable config for default parameters and module settings #185

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f473b80
Initial, working impl. of #128 (not hooked up)
ChanceNCounter Apr 2, 2021
b0db4bf
add working getter: config.get(setting, lang)
ChanceNCounter Apr 2, 2021
bd675f6
add mostly-working get/set
ChanceNCounter Apr 2, 2021
6602a0f
load config when language is loaded
ChanceNCounter Apr 2, 2021
f97df73
working partial impl (still not hooked up)
ChanceNCounter Apr 2, 2021
1cbcad8
address failing tests, break different test
ChanceNCounter Apr 2, 2021
dbb1caa
"finish" locale part, hook up to short_scale
ChanceNCounter Apr 3, 2021
3e3d2e2
fix positional args (in case of heathens)
ChanceNCounter Apr 3, 2021
73110ed
move from localized_function() param to data type
ChanceNCounter Apr 3, 2021
b0a488f
bump version to help downstreams test in prod =P
ChanceNCounter Apr 3, 2021
9a88d12
impr. setting find-pick, begin proper en integrat.
ChanceNCounter Apr 3, 2021
69ee1c0
cleanup pass
ChanceNCounter Apr 3, 2021
8cc7289
begin integration, prep for variant params
ChanceNCounter Apr 4, 2021
6d657ef
complete integration; + TimeVariant to ca-es conf
ChanceNCounter Apr 4, 2021
fb93906
fix variant param/config integration
ChanceNCounter Apr 4, 2021
5d3ef16
use 1st loaded lang's full code as default loc
ChanceNCounter Apr 4, 2021
1874671
properly unload all locs when load_langs_on_demand
ChanceNCounter Apr 4, 2021
603ce7e
fix overzealous unloading
ChanceNCounter Apr 4, 2021
f21246e
fix load_langs_on_demand(<non-default locale>)
ChanceNCounter Apr 4, 2021
d563fa5
fix introduced bug in loaded module checks
ChanceNCounter Apr 4, 2021
4ba5a65
document usage of `ConfigVar` in project-structure
ChanceNCounter Apr 4, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,4 @@ venv.bak/
.vscode/
vscode/
*.code-workspace
.vscode-*
3 changes: 3 additions & 0 deletions .pylintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[MESSAGES CONTROL]

disable=C0103,C0114,C0115,C0116,W0613
19 changes: 14 additions & 5 deletions lingua_franca/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
from .internal import get_default_lang, set_default_lang, get_default_loc, \
get_active_langs, _set_active_langs, get_primary_lang_code, \
get_full_lang_code, resolve_resource_file, load_language, \
load_languages, unload_language, unload_languages, get_supported_langs
### DO NOT CHANGE THIS IMPORT ORDER ###
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we detail why?

If it's a circular import issue is there a way we can restructure to avoid this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's exactly what it is, that's for the duration of the feature branch until I figure out exactly where the line is.

It may be possible to reduce the scope of internal's config imports and avoid the circle.

from .internal import get_active_langs, get_supported_locs, \
get_full_lang_code, get_supported_langs, get_default_loc, \
get_primary_lang_code
Comment on lines +2 to +4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move to using full words in variable names?
lang I think is understandable but could still be changed, however loc I think really starts getting ambiguous. Does it mean locale, location, localization - is there a difference between these things?
Could it be referring to something else? 🚂

I think this also reduces the cognitive load when reading through code - particularly for people unfamiliar with it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for standardizing. loc is the only one I didn't inherit.

get_full_lang_code() is an existing function which returns "the" full code corresponding to a primary lang code. This can't be removed wholesale, but it doesn't serve the whole purpose, and it doesn't serve "nondefault" locales in any way.

get_default_loc() returns the current "default" full code for a given primary code, which is configurable. On your rig, for instance, get_full_lang_code('en') will always return en-us, but I imagine get_default_loc('en') will return en-au.

I was thinking 'locale', but I didn't give it too much thought. I am by no means married to the names.


from lingua_franca import config
from .configuration import Config

### END OF IMPORT ORDER ###

from .internal import get_default_lang, set_default_lang, \
_set_active_langs, resolve_resource_file, \
load_language, load_languages, unload_language, unload_languages


config = Config()
2 changes: 1 addition & 1 deletion lingua_franca/bracket_expansion.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,4 +172,4 @@ def _expand_tree(self, tree):

def expand_parentheses(self):
tree = self._parse()
return self._expand_tree(tree)
return self._expand_tree(tree)
1 change: 0 additions & 1 deletion lingua_franca/config.py

This file was deleted.

122 changes: 122 additions & 0 deletions lingua_franca/configuration.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import json

from lingua_franca import get_active_langs, get_supported_locs, \
get_supported_langs, get_primary_lang_code, get_full_lang_code, \
get_default_loc
from lingua_franca.internal import resolve_resource_file

default_global_values = \
{
'load_langs_on_demand': False
}


class LangConfig(dict):
def __init__(self, lang_code):
super().__init__()
if lang_code not in get_supported_locs():
# DO NOT catch UnsupportedLanguageError!
# If this fails, we want to crash. This can *only* result from
# someone trying to override sanity checks upstairs. There are no
# circumstances under which this should fail and allow the program
# to continue.
lang_code = get_full_lang_code(lang_code)

resource_file = resolve_resource_file(f'text/{lang_code}/config.json')
try:
with open(resource_file, 'r', encoding='utf-8') as i_file:
default_values = json.load(i_file)
for k in default_values:
self[k] = default_values[k]
except (FileNotFoundError, TypeError):
pass


class Config(dict):
def __init__(self):
super().__init__()
self['global'] = dict(default_global_values)
for lang in get_active_langs():
self.load_lang(lang)

def load_lang(self, lang):
if all((lang not in get_supported_locs(),
lang in get_supported_langs())):
if lang not in self.keys():
self[lang] = {}
self[lang]['universal'] = LangConfig(lang)

full_loc = get_full_lang_code(lang)
else:
full_loc = lang
lang = get_primary_lang_code(lang)

self[lang][full_loc] = LangConfig(full_loc)

def _find_setting(self, setting=None, lang=''):
if setting is None:
raise ValueError("lingua_franca.config requires "
"a setting parameter!")

setting_available_in = []
possible_locs = []

while True:
if setting in self['global']:
setting_available_in.append('global')
if lang == 'global':
break

lang = lang or get_default_loc()

if lang in get_supported_langs():
possible_locs.append((lang, 'universal'))
possible_locs.append((lang, get_full_lang_code(lang)))

if lang in get_supported_locs():
primary_lang_code = get_primary_lang_code(lang)
possible_locs.append((primary_lang_code, 'universal'))
possible_locs.append(
(primary_lang_code, get_default_loc(primary_lang_code)))
possible_locs.append((primary_lang_code, lang))

for place in possible_locs:
if setting in self[place[0]][place[1]]:
setting_available_in.append(place)

break
try:
return setting_available_in[-1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we take the last one? I haven't reasoned this through completely but I assume the order would roughly be:

[
global, 
default_lang.universal, 
default_lang.specific, 
other_lang.universal, 
other_lang.specific,
default_locale.universal,
default_locale.specific,
other_locale.universal,
other_locale.specific,
place # which I'm unclear what this is
]

Is that right?

If so it seems like we would want the default lang setting over an other locale setting. But I could be getting the whole thing mixed up.

Given that I need to reason about it we should document what is intended in the docstring and write some tests for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely one of the things that will need the most discussion, but you're about right re: priority. The list is just in the opposite order of what you're thinking =P

'place' might not be a great variable name in this context, but it's referring to the possible indices of the desired setting, nothing to do with LF's job. Could be in this dict or this dict or that dict...

possible_places becomes a list of dicts to check for the config value. setting_available_in is the result, and it's ordered from global to local.

If the setting is found at all, setting_available_in[-1] will be, in the following order:

  1. The locale passed in the lang param, or the default locale for the lang param, or the current default locale when no lang param is passed
  2. Language-wide settings, which start out identical to their "default" locale. For instance, en's default locale (from get_full_lang_code() is en-us, so English's default language-wide settings are loaded from en-us's config.json
  3. "Global," LF-wide settings

If the value isn't present in any of those places, it's unavailable in that language.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perusing stales, rephrased. At runtime, LF config is a dict of dicts. It's got settings for the whole module and settings for each locale. Each language defers to a particular locale as its "default" locale. This is configurable. Gez's default English locale will be en-AU, Chance's will be en-US, Aditya might change it up on the fly!

So. We want to get(setting).

  1. Given a list of possible_places (dicts) where the requested setting might be stored, look for it
  2. Let the list of dicts in which you found the value be setting_available_in
  3. setting_available_in should, thanks to the algorithm that built it, be ordered as follows:
    [global_value, fallback_lang_value, fallback_loc_value, specified_lang_value, specified_loc_value] (where "fallback" is the default language, if different from the one specified)
  4. Return the last element of setting_available_in, representing the "most local" value that conforms with the input query

This way, a "more local" config dict will always override its less-local counterpart, assuming the requested key is present in the more local dict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of populating each field, let's say I am running with

  • Default language English, default locale en-US
  • Current user language and locale en/en-AU
  • Spanish loaded, default locale es-ES
  • Alternate locale es-US also loaded

If I ask for an es-US setting that's fully localized, I expect setting_available_in to offer values in the order:

[en-US, en-AU, es-ES, es-US]

representing, per the previous comment,

[fallback_lang_value, fallback_loc_value, specified_lang_value, specified_loc_value]

but, if any of the config dicts are missing the setting I asked for, they'll be absent from this list and LF will move on with its life, returning instead the next value "up" the chain.

At the end, this bit would return [en-US, en-AU, es-ES, es-US][-1], getting es-US and, per the initial function call, causing the getter to return the config value from es-US.

except IndexError:
return None

def get(self, setting=None, lang=''):
if lang != 'global':
if all((lang,
get_primary_lang_code(lang) not in get_active_langs())):
raise ModuleNotFoundError(f"{lang} is not currently loaded")

try:
setting_location = self._find_setting(setting, lang)

if setting_location == 'global':
return self['global'][setting]
return self[setting_location[0]][setting_location[1]][setting]

except TypeError:
return None

def set(self, setting=None, value=None, lang='global'):
if lang == 'global':
if setting in self['global']:
self['global'][setting] = value
return

setting_location = self._find_setting(setting, lang if lang !=
'global' else get_default_loc())
if all((setting_location, setting_location != 'global')):
self[setting_location[0]][setting_location[1]][setting] = value
return

raise KeyError(
f"{setting} is not available as a setting for language: '{lang}'")
31 changes: 24 additions & 7 deletions lingua_franca/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@
from warnings import warn
from os.path import join

from lingua_franca import config

from lingua_franca.bracket_expansion import SentenceTreeParser
from lingua_franca.internal import localized_function, \
populate_localized_function_dict, get_active_langs, \
get_full_lang_code, get_default_lang, get_default_loc, \
is_supported_full_lang, _raise_unsupported_language, \
UnsupportedLanguageError, NoneLangWarning, InvalidLangWarning, \
FunctionNotLocalizedError
FunctionNotLocalizedError, ConfigVar


_REGISTERED_FUNCTIONS = ("nice_number",
Expand Down Expand Up @@ -241,7 +242,7 @@ def year_format(self, dt, lang, bc):
'res/text'))


@localized_function(run_own_code_on=[UnsupportedLanguageError])
@localized_function(run_own_code_on=[UnsupportedLanguageError, TypeError])
def nice_number(number, lang='', speech=True, denominators=None):
"""Format a float to human readable functions

Expand All @@ -255,12 +256,23 @@ def nice_number(number, lang='', speech=True, denominators=None):
Returns:
(str): The formatted string.
"""
args = locals()
if denominators:
try:
denominators.__iter__
except (AttributeError, TypeError):
try:
args[denominators] = range(*denominators)
except TypeError:
raise ValueError("nice_number(denominators) must be "
"iterable, or a valid param for range()")
nice_number(**args)
return str(number)


@localized_function()
def nice_time(dt, lang='', speech=True, use_24hour=False,
use_ampm=False, variant=None):
def nice_time(dt, lang='', speech=True, use_24hour=ConfigVar,
use_ampm=ConfigVar, variant=ConfigVar):
"""
Format a time to a comfortable human format

Expand All @@ -281,7 +293,7 @@ def nice_time(dt, lang='', speech=True, use_24hour=False,


@localized_function()
def pronounce_number(number, lang='', places=2, short_scale=True,
def pronounce_number(number, lang='', places=2, short_scale=ConfigVar,
scientific=False, ordinals=False):
"""
Convert a number to it's spoken equivalent
Expand Down Expand Up @@ -321,8 +333,8 @@ def nice_date(dt, lang='', now=None):
return date_time_format.date_format(dt, full_code, now)


def nice_date_time(dt, lang='', now=None, use_24hour=False,
use_ampm=False):
def nice_date_time(dt, lang='', now=None, use_24hour=ConfigVar,
use_ampm=ConfigVar):
"""
Format a datetime to a pronounceable date and time

Expand All @@ -346,6 +358,11 @@ def nice_date_time(dt, lang='', now=None, use_24hour=False,
full_code = get_full_lang_code(lang)
date_time_format.cache(full_code)

if use_24hour is ConfigVar:
use_24hour = config.get(setting='use_24hour', lang=full_code)
if use_ampm is ConfigVar:
use_ampm = config.get(setting='use_ampm', lang=full_code)

return date_time_format.date_time_format(dt, full_code, now, use_24hour,
use_ampm)

Expand Down
Loading