slsutil.findup does not work with salt-ssh

[baby-gnu]

Description

In version 3004 the slsutil.findup is added and wanted to try it with salt-ssh but it does not work.

Setup

Two identical Debian machines, testmachine5 and testmachine6:

lsb_release -a

The same on the machine with salt-ssh and the target

Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

With salt-ssh package from https://repo.saltproject.io/py3/debian/11/amd64/archive/3004 installed on testmachine5.

Please be as specific as possible and give set-up details.

• on-prem machine
• VM (KVM)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD

Steps to Reproduce the behavior

On the salt-ssh machine named testmachine5, create the following files:

/srv/salt/test-import/map.jinja

{#- -*- mode: jinja; coding: utf-8 -*- #}
{% set mapdata = {"test": "foo"} %}

/srv/salt/test-import/findup.sls

# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

/etc/salt/roster

# -*- yaml -*-

testmachine6:
  host: 192.168.0.111
  user: root

I try to execute either the .sls

salt-ssh testmachine6 state.apply test-import.findup

[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 261, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 547, in render_jinja_tmpl
    raise SaltRenderError(
salt.exceptions.SaltRenderError: Jinja error: unhashable type: 'dict'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

; line 3

---
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

[...]
---
[CRITICAL] Rendering SLS 'base:test-import.findup' failed: Jinja error: unhashable type: 'dict'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
    return self[key]
  File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
    rv = self._mapping[key]
TypeError: unhashable type: 'dict'

; line 3

---
# -*- mode: salt; coding: utf-8 -*-
# vim: ft=sls
{% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================

test-import/findup/test.nop:
  test.nop:
    - name: {{ mapdata | json }}

[...]
---
testmachine6:
    - Rendering SLS 'base:test-import.findup' failed: Jinja error: unhashable type: 'dict'
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
          output = template.render(**decoded_context)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1090, in render
          self.environment.handle_exception()
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 832, in handle_exception
          reraise(*rewrite_traceback_stack(source=source))
        File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 28, in reraise
          raise value.with_traceback(tb)
        File "<template>", line 3, in top-level template code
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 389, in get
          return self[key]
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 431, in __getitem__
          rv = self._mapping[key]
      TypeError: unhashable type: 'dict'
      
      ; line 3
      
      ---
      # -*- mode: salt; coding: utf-8 -*-
      # vim: ft=sls
      {% from salt["slsutil.findup"](tplfile, "map.jinja") import mapdata %}    <======================
      
      test-import/findup/test.nop:
        test.nop:
          - name: {{ mapdata | json }}
      
      [...]
      ---

Or run the command directly as noted in the documentation:

salt-ssh testmachine6 slsutil.findup test-import map.jinja

testmachine6:
    ----------
    retcode:
        1
    stderr:
        Passed invalid arguments: Starting path not found in the state tree: test-import.
        
        Usage:
    stdout:
        Find the first path matching a filename or list of filenames in a specified
            directory or the nearest ancestor directory. Returns the full path to the
            first file found.
        
            .. versionadded:: 3004
        
            :param str startpath: The fileserver path from which to begin the search.
                An empty string refers to the state tree root.
            :param filenames: A filename or list of filenames to search for. Searching for
                directory names is also supported.
            :param str saltenv: The fileserver environment to search. Default: ``base``
        
            Example: return the path to ``defaults.yaml``, walking up the tree from the
            state file currently being processed.
        
            .. code-block:: jinja
        
                {{ salt["slsutil.findup"](tplfile, "defaults.yaml") }}
        
            CLI Example:
        
            .. code-block:: bash
        
                salt '*' slsutil.findup formulas/shared/nginx map.jinja

Expected behavior

The state.apply and slsutil.findup should have worked as expected.

Versions Report

salt-ssh --versions-report

(Provided by running salt-ssh --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3004
 
Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.9.7
        pygit2: Not Installed
        Python: 3.9.2 (default, Feb 28 2021, 17:03:44)
  python-gnupg: Not Installed
        PyYAML: 5.3.1
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-8-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

Additional context

I tested this new feature in an attempt to workaround #31531 where importing the map.jinja starts not working since version 3003 (it was working on 3002.7).

[myii]

I can also confirm this. We were discussing issues in Slack after our last working group meeting, when we came across slsutil.findup here:

• https://salt.tips/new-features-in-salt-silicon/#file-lookup-functions

I tried this out using the apt-formula, just on a random state file. Tested the following change on a standard minion as well as one accessed using salt-ssh:

-{% from "apt/map.jinja" import apt as apt_map with context %}
+{% from salt['slsutil.findup']('apt', 'map.jinja') import apt as apt_map with context %}

Works as expected on the standard minion but crashes out with salt-ssh:

minion:
    - Rendering SLS 'base:apt.unattended' failed: Jinja error: unhashable type: 'dict'
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
          output = template.render(**decoded_context)
        File "/usr/lib/python3/dist-packages/jinja2/asyncsupport.py", line 76, in render
          return original_render(self, *args, **kwargs)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
          return self.environment.handle_exception(exc_info, True)
        File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
          reraise(exc_type, exc_value, tb)
        File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
          raise value.with_traceback(tb)
        File "<template>", line 3, in <module>
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 356, in get
          return self[key]
        File "/usr/lib/python3/dist-packages/jinja2/utils.py", line 405, in __getitem__
          rv = self._mapping[key]
      TypeError: unhashable type: 'dict'
      
      ; line 3
      
      ---
      # This is the main state file for configuring unattended upgrades with apt
      {# from "apt/map.jinja" import apt as apt_map with context #}
      {% from salt['slsutil.findup']('apt', 'map.jinja') import apt as apt_map with context %}    <======================
      {% set apt = pillar.get('apt:unattended', {}) -%}
      {% set unattended_config_template = apt.get('unattended_config_template', 'salt://apt/templates/unattended_config.jinja') -%}
      {% set periodic_config_template = apt.get('periodic_config_template', 'salt://apt/templates/periodic_config.jinja') -%}
      
      apt_unattended_pkgs:
      [...]
      ---