feat(map): update to v5 map.jinja #225
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update `_mapdata` reference files to include the `map.jinja` configuration sources.
baby-gnu
changed the title
|
@baby-gnu Some initial feedback, just diff-ing across the three formulas. This PR and --- .../openssh-formula/openssh/libmapstack.jinja
+++ .../template-formula/TEMPLATE/libmapstack.jinja
@@ -95,8 +95,10 @@
{%- set stack = defaults | default({"values": {} }, boolean=True) %}
{#- Build configuration file names based on matchers #}
+{%- set config_get_strategy = salt["config.get"](tplroot ~ ":strategy", None) %}
{%- set matchers = parse_matchers(
matchers,
+ config_get_strategy=config_get_strategy,
log_prefix=log_prefix
)
| load_yaml %}
@@ -194,6 +196,19 @@
{%- set yaml_names = [yaml_names] %}
{%- endif %}
+{#- Try to load a `.yaml.jinja` file for each `.yaml` file #}
+{%- set all_yaml_names = [] %}
+{%- for name in yaml_names %}
+{%- set extension = name.rpartition(".")[2] %}
+{%- if extension not in ["yaml", "jinja"] %}
+{%- do all_yaml_names.extend([name ~ ".yaml", name ~ ".yaml.jinja"]) %}
+{%- elif extension == "yaml" %}
+{%- do all_yaml_names.extend([name, name ~ ".jinja"]) %}
+{%- else %}
+{%- do all_yaml_names.append(name) %}
+{%- endif %}
+{%- endfor %}
+
{#- `yaml_dirname` can be an empty string with literal path like `myconf.yaml` #}
{%- set yaml_dir = [
param_dir,
@@ -202,15 +217,10 @@
| select
| join("/") %}
-{%- for yaml_name in yaml_names %}
-{#- Make sure to have a `.yaml` extension #}
-{#- Use `.rpartition` to strip last `.yaml` in `dir.yaml/file.yaml` #}
+{%- for yaml_name in all_yaml_names %}
{%- set yaml_filename = [
yaml_dir.rstrip("/"),
- yaml_name.rpartition(".yaml")
- | reject("equalto", ".yaml")
- | join
- ~ ".yaml"
+ yaml_name
]
| select
| join("/") %}
@@ -228,7 +238,7 @@
{%- do salt["log.debug"](
log_prefix
~ "loaded configuration values from "
- ~ yaml_name
+ ~ yaml_filename
~ ":\n"
~ yaml_values
| yaml(False)
@@ -271,7 +281,7 @@
{%- do salt["log.debug"](
log_prefix
~ "merged configuration values from "
- ~ yaml_name
+ ~ yaml_filename
~ ", merge: strategy='"
~ strategy
~ "', merge_lists='"--- .../openssh-formula/openssh/map.jinja
+++ .../template-formula/TEMPLATE/map.jinja
@@ -38,6 +38,7 @@
{#- Load formula parameters values #}
{%- set _formula_matchers = ["defaults.yaml"] + map_sources %}
+
{%- set _formula_settings = mapstack(
matchers=_formula_matchers,
dirs=[formula_param_dir],
@@ -59,3 +60,7 @@
{%- do salt["log.debug"]("map.jinja: save parameters in variable 'mapdata'") %}
{%- set mapdata = _formula_settings["values"] %}
+
+{#- Per formula post-processing of `mapdata` if it exists #}
+{%- do salt["log.debug"]("map.jinja: post-processing of 'mapdata'") %}
+{%- include tplroot ~ "/post-map.jinja" ignore missing %} |
The v5 `map.jinja` is a generic and configurable system to load
configuration values, exposed as the `mapdata` variable, from
different places:
- YAML files and templates from the fileserver for non-secret data
- pillars or SDB are preferred for secret data
- grains or `config.get`
The `map.jinja` optional sources are configured with compound targeting like
syntax `[<TYPE>[:<OPTION>[:<DELIMITER>]]@]<KEY>` with the following
default ordered sources:
- `Y:G@osarch`: YAML file and Jinja template named after `osarch` grain
- `Y:G@os_family`: YAML file and Jinja template named after `os_family` grain
- `Y:G@os` YAML file and Jinja template named after `os` grain
- `Y:G@osfinger` YAML file and Jinja template named after `osfinger` grain
- `C@{{ tplroot ~ ':lookup' }}`: dict retrieved with `salt["config.get"]`
- `C@{{ tplroot }}`: dict retrieved with `salt["config.get"]`
- `Y:G@id`: YAML file and Jinja template named after `osarch` grain
This is done by two new libraries:
- `libmatchers.jinja` provides the `parse_matchers` macro to parse
strings looking like compound matchers, for example `Y:G@osarch`
- `libmapstack.jinja` provides the `mapstack` macro to load
configuration from different sources described by matchers
Post-processing of `mapdata` variable can be done in a `parameters/post-map.jinja`.
The v5 `map.jinja` is documented in `docs/map.jinja.rst`.
BREAKING CHANGE: `map.jinja` now exports a generic `mapdata` variable
BREAKING CHANGE: The per grain parameter values are now under `TEMPLATE/parameters/`
baby-gnu
force-pushed
the
feature/update-to-v5-map.jinja
branch
from
97700ca
to
42e1932
Compare
|
@myii yes, the I push a fixed commit because I forgot to delete the old |
Thanks @myii. This time will give me some time for the next move ;-) |
|
🎉 This PR is included in version 5.0.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR progress checklist (to be filled in by reviewers)
What type of PR is this?
Primary type
[build]Changes related to the build system[chore]Changes to the build process or auxiliary tools and libraries such as documentation generation[ci]Changes to the continuous integration configuration[feat]A new feature[fix]A bug fix[perf]A code change that improves performance[refactor]A code change that neither fixes a bug nor adds a feature[revert]A change used to revert a previous commit[style]Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)Secondary type
[docs]Documentation changes[test]Adding missing or correcting existing testsDoes this PR introduce a
BREAKING CHANGE?Yes.
BREAKING CHANGE:
map.jinjanow exports a genericmapdatavariableBREAKING CHANGE: The per grain parameter values are now under
TEMPLATE/parameters/Related issues and/or pull requests
map.jinjaopenvpn-formula#134Describe the changes you're proposing
feat(map): update to v5
map.jinjaThe v5
map.jinjais a generic and configurable system to load configuration values, exposed as themapdatavariable, from different places:config.getThe
map.jinjaoptional sources are configured with compound targeting like syntax[<TYPE>[:<OPTION>[:<DELIMITER>]]@]<KEY>with the following default ordered sources:Y:G@osarch: YAML file and Jinja template named afterosarchgrainY:G@os_family: YAML file and Jinja template named afteros_familygrainY:G@osYAML file and Jinja template named afterosgrainY:G@osfingerYAML file and Jinja template named afterosfingergrainC@{{ tplroot ~ ':lookup' }}: dict retrieved withsalt["config.get"]C@{{ tplroot }}: dict retrieved withsalt["config.get"]Y:G@id: YAML file and Jinja template named afterosarchgrainThis is done by two new libraries:
libmatchers.jinjaprovides theparse_matchersmacro to parse strings looking like compound matchers, for exampleY:G@osarchlibmapstack.jinjaprovides themapstackmacro to load configuration from different sources described by matchersPost-processing of
mapdatavariable can be done in aparameters/post-map.jinja.The v5
map.jinjais documented indocs/map.jinja.rst.Pillar / config required to test the proposed changes
Debug log showing how the proposed changes work
Log of the processing of map.jinja
Documentation checklist
README(e.g.Available states).pillar.example.Testing checklist
state_top).Additional context