[BUG] breaking change in 0.21.3 (version_repo)

[OrangeDog]

Your setup

Formula commit hash / release tag

0.21.3

Versions reports (master & minion)

Salt Version:
           Salt: 2019.2.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.26.0
        libnacl: Not Installed
       M2Crypto: 0.32.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.2
         Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: ISO-8859-1
        machine: x86_64
        release: 4.15.0-91-generic
         system: Linux
        version: Ubuntu 18.04 bionic

Pillar / config used

zabbix:
  lookup:
    version_repo: '4.2'

---

Bug details

Describe the bug

Upgrading from 0.21.2 to 0.21.3 changed how the repo is configured.
It's now trying to use the default, which does not exist

The repository 'https://repo.zabbix.com/zabbix/2.2/ubuntu bionic Release' does not have a Release file.

Steps to reproduce the bug

See above.

Expected behaviour

No changes to existing behaviour on minor/patch releases.

Attempts to fix the bug

Had a look at the code and tried this pillar instead, but it had no effect.

zabbix:
  version_repo: '4.2'

[myii]

@OrangeDog I'm certain that this problem actually exists in 0.21.2 as well, since absolutely nothing changed in the map to 0.21.3 -- I tested that extensively:

• https://travis-ci.org/github/myii/zabbix-formula/builds/665299638

The issue is the upstream repo itself. 2.2 is ancient and only available for precise and trusty:

• https://repo.zabbix.com/zabbix/2.2/ubuntu/dists/

Try setting the pillar like this:

zabbix-formula/test/salt/pillar/default.sls

Lines 4 to 7 in 8291d7c

	zabbix:
	# Overrides map.jinja
	lookup:
	version_repo: '4.4'

[myii]

Let me just add, the formula definitely needs updating from 2.2 as the default. There's already an active discussion about that here: #123. Just need someone to propose an appropriate PR.

[OrangeDog]

I'm certain that this problem actually exists in 0.21.2 as well

No. I just reverted back to that and the problem went away.

Try setting the pillar like this

That's exactly what I did, as I put in the report above.

[myii]

@OrangeDog You've missed the lookup from the pillar.

[OrangeDog]

No I didn't.

[myii]

@OrangeDog Apologies, I saw the last entry in your original report. Well, this is going to be fun to resolve, we've got the same configuration (ubuntu-1804-2019-2-py3) working in Travis for 0.21.3:

• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317

[OrangeDog]

Hmm, I don't know what else would cause a difference, assuming your tests aren't broken in some way.

[myii]

@OrangeDog We've got debug logging active, so you can see exactly what's rendered. So the key places clearly show what's going on.

Rendered pillar data:

• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L638-L645

Various repos configured for 4.4:

• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L1012-L1014
• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L1256-L1258
• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L1476-L1478

Various 4.4 packages installed:

• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L1649-L1650
• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L2067
• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L2075
• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L2997

InSpec tests passing for the specific package versions installed for 4.4:

• https://travis-ci.com/github/saltstack-formulas/zabbix-formula/jobs/300744317#L4089-L4093

[OrangeDog]

I also have the zabbix module properties set on this host. Maybe that messes up the merging now?
They already have to use different yaml syntax.

zabbix.url: ...
zabbix.user: ...

zabbix:
  lookup:
    version_repo: '4.4'

[myii]

@OrangeDog The one potential difference is the use of config.get vs. pillar.get. Can you share any differences in the output of the following:

• salt <minion> pillar.get zabbix
• salt <minion> config.get zabbix

[OrangeDog]

On my currently working system (the psk is in a grain).

pillar.get:
{
    "local": {
        "lookup": {
            "version_repo": "4.4"
        }
    }
}

config.get:
{
    "local": {
        "psk": "..."
    }
}

[myii]

@OrangeDog Back when we first started using config.get, there was a question about the merge strategy to use. We did consider allowing that to be configurable (see the Jinja snippet in this comment). Would you mind testing the following change:

zabbix-formula/zabbix/map.jinja

Line 11 in 8291d7c

{%- set _config = salt['config.get']('zabbix', default={}) %}

-{%- set _config = salt['config.get']('zabbix', default={}) %}
+{%- set _config = salt['config.get']('zabbix', default={}, merge='recurse')

[OrangeDog]

Using that commit hasn't had any apparent effect: the state still tries to use the 2.2 repo.

[myii]

@OrangeDog So is there any difference between the output of these two commands?

• salt <minion> config.get zabbix
• salt <minion> config.get zabbix merge=recurse

If not, would you mind sharing the layout of how you're providing that grain, so that I can test it at my end?