[BUG]pkgrepo.managed (apt) ignores [trusted=yes]

[ustolt]

pkgrepo.managed (apt) ignores [trusted=yes]
Since we upgraded our system from Salt V3004.2 to V3005.1, the following problem occurs:
When trying to add an unsigned repository, e.g.: "deb [trusted=yes] http://reposerver.net/ubuntu/focal main", there is an error "E: The repository 'http://reposerver.net/ubuntu Focal Release' is unsigned."

Setup

       ID: repositories_managed_repository_ubuntu-3rd
 Function: pkgrepo.managed
     Name: deb  [trusted=yes] http://reposerver.net/ubuntu/  focal main
   Result: False
  Comment: Failed to configure repo 'deb  [trusted=yes] http://reposerver.net/ubuntu/  focal main': E: The repository 'http://reposerver.net/ubuntu focal Release' is not signed.
  Started: 10:31:53.925434
 Duration: 4487.955 ms
  Changes:   

Please be as specific as possible and give set-up details.

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD
• classic packaging
• onedir packaging
• used bootstrap to install

Steps to Reproduce the behavior

test_unsigned_repo:
  pkgrepo.managed:
    - humanname: unsigned_repo
    - name: deb [trusted=yes] http://reposerver.net/ubuntu main
    - file: /etc/apt/sources.list.d/unsigned_repo_test

Reading package lists...
2023-04-21 09:47:26,053 [salt.loaded.int.module.cmdmod:909 ][ERROR   ][278327] stderr: E: The repository 'http://reposerver.net/ubuntu focal Release' is not signed.
2023-04-21 09:47:26,053 [salt.loaded.int.module.cmdmod:911 ][ERROR   ][278327] retcode: 100
2023-04-21 09:47:26,054 [salt.state       :321 ][ERROR   ][278327] Failed to configure repo 'deb  [trusted=yes] http://reposerver.net/ubuntu/  focal main': E: The repository 'http://reposerver.net/ubuntu focal Release' is not signed.
2023-04-21 09:47:30,931 [salt.loaded.int.module.cmdmod:905 ][ERROR   ][278327] Command 'apt-get' failed with return code: 100
2023-04-21 09:47:30,931 [salt.loaded.int.module.cmdmod:907 ][ERROR   ][278327] stdout: Ign:1 http://reposerver.net/ubuntu focal InRelease

Expected behavior
The expected source list file should look like this

deb [trusted=yes] http://reposerver.net/ubuntu/ focal main

After completion of the state it looks like this:

deb http://reposerver.net/ubuntu/ focal main

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

salt-call --versions-report
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
        Python: 3.9.16 (main, Jan  6 2023, 22:49:56)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: ubuntu 20.04 focal
        locale: utf-8
       machine: x86_64
       release: 5.4.0-147-generic
        system: Linux
       version: Ubuntu 20.04 focal

Additional context
Add any other context about the problem here.

[OrangeDog]

The pkgrepo.managed had significant changes in 3005, to align with updated Debian practices.

[rumfellow]

I experience the same issue in 3006.0

[HL-SaltBae]

Also experiencing this, with additional bad side effects:
On the first run on the formula, the result looks like this:

----------
          ID: <redacted>
    Function: pkgrepo.managed
        Name: deb [ trusted=yes arch=amd64 ] https://<redacted> bionic main
      Result: False
     Comment: Failed to configure repo 'deb [ trusted=yes arch=amd64 ] https://<redacted> bionic main': E: The repository 'https://<redacted> bionic Release' is not signed.
     Started: 17:20:57.024479
    Duration: 2496.991 ms
     Changes:   

It appears to be a failure, but the file is actually created, and it looks like this:

deb [arch=amd64] https://<redacted> bionic main

Subsequent running of the same state formula will produce success with no changes messages:

----------
          ID: <redacted>
    Function: pkgrepo.managed
        Name: deb [ trusted=yes arch=amd64 ] https://<redacted> bionic main
      Result: True
     Comment: Package repo 'deb [ trusted=yes arch=amd64 ] https://<redacted> bionic main' already configured
     Started: 17:21:27.562043
    Duration: 2.883 ms
     Changes:   

... which is LIES

[pckroon]

I'm encountering the same issue on 3006.1.
I'll see if I can free up some time to dig through the code and get a PR going that would also accept trusted=yes besides signed-by.

[MartinEmrich]

Same here. the [trusted=yes allow-insecure=yes] string gets completely eaten away. Running Salt 3006.2.

Everything suggests to me that the name: field ist copied as-is as a string, but there seems to be some lossy parse-and-recreate logic in the salt-minion.

Will probably work around for now with file.managed, order: and a cmd.run on apt-get update.

[OrangeDog]

order: and a cmd.run on apt-get update.

Rather than that, you want either require and refresh on your pkg.installed, or onchanges on a module.run for pkg.refresh_db.

[MartinEmrich]

you want either require and refresh on your pkg.installed,

Hmm, certainly do not want that. There are very many pkg states which shall also work for RPM/Yum-based OS, so modifying each of them with this OS- and workaround-specific relation is IMHO not good.

But will try module.watch on pkg.refresh_db certainly looks "saltier" than cmd.watch :)

[MartinEmrich]

It gets worse: If any pkgrepo.managed state (even for a repo with no square bracked options) fires during highstate, it seems that all repositories are stripped of these parameters.

[MartinEmrich]

I had another (really dark) workaround is mind: protect my files from salt by using the filesystem immutable attribute:

my-homebrew-repo:
  cmd.run:
    - order: 2
    - name: chattr -i /etc/apt/sources.list.d/myhomebrew.list ; echo "deb [trusted=yes allow-insecure=yes] https://myhome.zz/myhomebrewrepo/{{ grains['os'] | lower }}-{{ grains['oscodename'] }} devel/" > /etc/apt/sources.list.d/myhomebrew.list ; chattr +i /etc/apt/sources.list.d/myhomebrew.list
    - unless: grep "allow-insecure=yes" /etc/apt/sources.list.d/myhomebrew.list && ( lsattr /etc/apt/sources.list.d/myhomebrew.list | grep "i---" )
    - onchanges_in:
      - module: apt-update
    - watch_in:
      - module: apt-update

But sadly, it locks up the highstate completely, and confirms my suspicion, managing one repo messes around with others:

          ID: repository-adoptium
    Function: pkgrepo.managed
        Name: deb https://packages.adoptium.net/artifactory/deb bookworm main
      Result: False
     Comment: Failed to configure repo 'deb https://packages.adoptium.net/artifactory/deb bookworm main': [Errno 1] Operation not permitted: '/etc/apt/sources.list.d/myhomebrew.list'
     Started: 13:13:44.200043
    Duration: 1621.819 ms

I'm inclined to give up now, apt repo management seems to be broken. 😞

[MartinEmrich]

This is now an example of what seems to be working for me:

{# apt-key, key management, etc needs gnupg binary. Install even earlier. #}
gnupg:
  pkg:
    - installed
    - order: 2

{# hook to do "apt-get update" after adding the files #}
apt-update:
  module:
    - watch
    - order: 5
    - name: pkg.refresh_db

{# 
   A home-made repo: Just a few .deb files, held together with a dpkg-scanpackages-generated 
   Packages.gz. Needs these apt options (at least) as of Debian 12.
#}
my-homebrew-repo:
  file:
    - managed:
    - order: 5
    - contents: "deb [trusted=yes allow-insecure=yes] https://myhome.zz/myhomebrewrepo/{{ grains['os'] | lower }}-{{ grains['oscodename'] }} devel/"
    - name: /etc/apt/sources.list.d/myhomebrew.list
    - onchanges_in:
      - module: apt-update
    - watch_in:
      - module: apt-update
    - onchanges:
      - pkgrepo: '*'
    - watch: 
      - pkgrepo: '*'

{# 
   some other unmodified repo state with no special needs. When it fires, it will mess up the 
   repo above, but will also trigger it via watch/onchanges to have it repaired. 
#}
repository-adoptium:
  pkgrepo.managed:
    - name: "deb https://packages.adoptium.net/artifactory/deb {{ grains['oscodename'] }} main"
    - file: /etc/apt/sources.list.d/adoptium.list
    - key_url: "https://packages.adoptium.net/artifactory/api/gpg/key/public"
    - refresh: True

[oe3gwu]

I also have this bug. It still works in Ubuntu 20.04 but not in 22.04 or 23.10.

[s0undt3ch]

This will be fixed in the next minor release

[Akm0d]

Verified working in 3007.0