Fixing 3.9 (#107)

* Fixing 3.9 * Updated tinydata for rspec tests * Lints * Fixing debug check package on RedHat * lint
example42 · Feb 2, 2023 · c99a195 · c99a195
1 parent a1f900d
commit c99a195
Show file tree

Hide file tree

Showing 30 changed files with 897 additions and 65 deletions.
diff --git a/.devcontainer/README.md b/.devcontainer/README.md
@@ -32,6 +32,3 @@ https://github.com/microsoft/vscode-dev-containers/tree/v0.140.1/containers/pupp
 	"postCreateCommand": "pdk --version",
 }
 ```
-
-
-
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,16 @@
 # CHANGELOG
 
+## 3.9.1
+
+-   Fixed tp debug cli command
+-   Fixed tp test command on apps without packages
+-   Fixed possible resource duplicstion when multiple app with exec_postinstall setting are present in the catalog
+-   Fixed repo keys managmeent when apt_safe_trusted_key is true
+-   Updated metadata.json to reflect factual compatibility with only Puppet 6 or later
+-   Updated metadata.json to have puppetlabs/vcsrepo as a dependency
+-   Updated sample tinydata and OS tests in unit tests
+-   Fixed tp uninstall command to remove also tp data files
+
 ## 3.9.0
 
 -   tp::install::* defines simplified and aligned to semifinal v4 tinydata structure

diff --git a/README_legacy.md → README-legacy.md b/README_legacy.md → README-legacy.md
diff --git a/README.md b/README.md
@@ -5,38 +5,34 @@
 
 ## Table of Contents
 
-1.  [Module description - What Tiny Puppet does?](#module-description)
+-   [Tiny Puppet](#tiny-puppet)
+  - [Table of Contents](#table-of-contents)
+  - [Module description](#module-description)
     -   [Features](#features)
     -   [Use cases](#use-cases)
-
-2.  [Setup](#setup)
+  - [Setup](#setup)
     -   [What tp affects](#what-tp-affects)
     -   [Getting started with tp](#getting-started-with-tp)
-
-3.  [Usage in Puppet code](#usage-in-puppet-code)
+  - [Usage in Puppet code](#usage-in-puppet-code)
     -   [Common uses](#common-uses)
-    -   [Installing packages - tp::install](#installing-packages---tpinstall)
-    -   [Installation alternatives - tp::stdmod](#installation-alternatives---tpstdmod)
-    -   [Managing configurations - tp::conf](#managing-configurations---tpconf)
-    -   [Managing directories - tp::dir](#managing-directories---tpdir)
-    -   [Managing repositories - tp::repo](#managing-repositories---tprepo)
-    -   [Automated and easy testing - tp::test](#automated-and-easy-testing---tptest)
+    -   [Installing packages -   tp::install](#installing-packages---tpinstall)
+    -   [Installation alternatives -   tp::stdmod](#installation-alternatives---tpstdmod)
+    -   [Managing configurations -   tp::conf](#managing-configurations---tpconf)
+      -   [tp::conf file paths conventions](#tpconf-file-paths-conventions)
+    -   [Managing directories -   tp::dir](#managing-directories---tpdir)
+    -   [Managing repositories -   tp::repo](#managing-repositories---tprepo)
+    -   [Automated and easy testing -   tp::test](#automated-and-easy-testing---tptest)
     -   [Configuring tp resources via Hiera](#configuring-tp-resources-via-hiera)
-
-4.  [Updating tiny data and using alternative data sources](#updating-tiny-data-and-using-alternative-data-sources)
-
-5.  [Usage on the command line](#usage-on-the-command-line)
-
-6.  [Reference](#reference)
+  - [Updating tiny data and using alternative data sources](#updating-tiny-data-and-using-alternative-data-sources)
+  - [Usage on the command line](#usage-on-the-command-line)
+  - [Reference](#reference)
     -   [Classes](#classes)
-    -   [Defined Types](#defined-types)
+    -   [Defined types](#defined-types)
     -   [Types](#types)
     -   [Functions](#functions)
     -   [Tasks](#tasks)
-
-7.  [Prerequisites and limitations](#prerequisites-and-limitations)
-
-8.  [Additional info](#additional-info)
+  - [Prerequisites and limitations](#prerequisites-and-limitations)
+  - [Additional info](#additional-info)
 
 ## Module description
 
@@ -77,7 +73,6 @@ The main features of tp module are:
 -   Optional Bolt tasks to perform the above actions on remote nodes.
 -   Usable both by total beginners and experienced sysadmins: interface is simple but allows powerful customisations
 
-
 ### Use cases
 
 Tiny Puppet can be considered a supplement or a replacement of standard component modules.
@@ -147,8 +142,6 @@ Starting from version 3.8.0, a technology preview of tp 4 features is available
 
     tp::use_v4: true
 
-
-
 Here follows an example of tp resources used inside a custom profile where the content of a configuration file is based on a template with custom values.
 
     class profile::openssh (
@@ -220,11 +213,11 @@ Some parameters are available to manage tp::install automation:
 -   **auto_conf** Default: true. If true and tinydata relevant is present a default configuration is provided (this could happen just when some basic configuration is needed to actually activate the service).
 -   **auto_prereq**  Default: false. If true eventual package, tp::install or other dependencies are installed automatically. This is set to false by default in order to minimize duplicated resources risk, but might be required to set up specific applications correctly.
 
-      tp::install { 'consul':
-        upstream_repo => true,
-        auto_conf     => true,
-        auto_prereq   => false,
-      }
+    tp::install { 'consul':
+      upstream_repo => true,
+      auto_conf     => true,
+      auto_prereq   => false,
+    }
 
 Other parameters are available to manage integrations:
 

diff --git a/data/RedHat.yaml b/data/RedHat.yaml
@@ -4,6 +4,7 @@ tp::check_package_command: 'rpm -q'
 tp::check_repo_path: '/etc/yum.repos.d/'
 tp::check_repo_path_post: '.repo'
 tp::info_package_command: 'rpm -qi'
+tp::debug_package_command: 'rpm -qi'
 
 tp::tp_commands:
   check:

diff --git a/data/RedHat9.yaml b/data/RedHat9.yaml
@@ -0,0 +1,15 @@
+---
+tp::check_package_command: 'dnf info'
+tp::info_package_command: 'dnf info'
+tp::debug_package_command: 'dnf info'
+
+tp::tp_commands:
+  check:
+    package:
+      command: 'dnf info'
+  info:
+    package:
+      command: 'dnf info'
+  debug:
+    package:
+      command: 'dnf info'
diff --git a/docs/conf.md b/docs/conf.md
@@ -11,7 +11,6 @@ Given the above example, we can edit the referenced file with:
     [root@centos7-p4 ~]# mkdir -p /vagrant/modules_local/site/templates/nginx
     [root@centos7-p4 ~]# vi /vagrant/modules_local/site/templates/nginx/nginx.conf.erb
 
-
 You can pass an hash of custom key/values using the ```options_hash``` parameter:
 
     $nginx_options = {
@@ -39,6 +38,5 @@ An then, in your ```$modulepath/site/templates/ningx/nginx.conf.erb``` have some
 Some explanations are needed here. Your ```options_hash``` parameter is accessed, in the erb file, via the ```@options``` variable (You can use also ```@options_hash```) because in ```tp::conf``` we plan to merge the values from @options_hash to a set of default options (compliant with the underlying OS).
 You have at disposal also the ```@settings``` hash which contains OS specific data for the managed application. To have an idea of what kind of data we provide for each supported application check in [```data/nginx/default.yaml```](https://github.com/example42/puppet-tp/blob/master/data/nginx/default.yaml).
 
-
 There are many options in ```tp::conf``` that let you manage every aspect of your configurations, for example we can manage its permissions, how to populate its content (with static source, epp or erb templates, plain content...), if to trigger a service restart when the files changes (by default the relevant service, if present, is restarted) and so on.
 The ```tp::conf``` define works perfectly when used with ```tp::install``` but can cope well with packages installed via other modules.
diff --git a/functions/url_replace.pp b/functions/url_replace.pp
@@ -13,7 +13,7 @@ function tp::url_replace (
   }
   $versioned_url = regsubst($url,'\$VERSION', $_version, 'G')
   $os_replaced_url = regsubst($versioned_url,'\$OS', downcase($facts['kernel']), 'G')
-  $arch_replaced_url = regsubst($os_replaced_url,'\$ARCH', downcase($translated_arch), 'G') # lint-ignore: 140chars
+  $arch_replaced_url = regsubst($os_replaced_url,'\$ARCH', downcase($translated_arch), 'G') # lint:ignore:140chars
 
   return $arch_replaced_url
 }
diff --git a/manifests/cli.pp b/manifests/cli.pp
@@ -142,6 +142,7 @@
         content => epp('tp/run_debug/package_debug.epp'),
       }
       file { $debug_script_path:
+        mode    => '0755',
         path    => $debug_script_path,
         content => epp($debug_script_template, { 'options' => $real_options }),
       }

diff --git a/manifests/conf.pp b/manifests/conf.pp
@@ -390,7 +390,7 @@
       true      => $service_ref,
       default   => $config_file_notify,
     }
-    $validate_cmd = pick_default(getvar('settings.validate_cmd'), getvar("settings.${prefix}files.${base_file}.validate_cmd"), undef) # lint-ignore:140chars
+    $validate_cmd = pick_default(getvar('settings.validate_cmd'), getvar("settings.${prefix}files.${base_file}.validate_cmd"), undef) # lint:ignore:140chars
     $default_validate_cmd = $validate_cmd ? {
       ''     => undef,
       String => $validate_cmd,

diff --git a/manifests/install.pp b/manifests/install.pp
@@ -224,8 +224,8 @@
         default  => undef,
       },
       destination    => pick($install_method, getvar('tinydata_settings.install_method')) ? {
-        'source' => pick($destination, "${tp::real_tp_params['data']['path']}/source/${app}"),
-        'file'   => pick($destination, "${tp::real_tp_params['data']['path']}/download/${app}"),
+        'source' => pick($destination, "${tp::data_dir}/source/${app}"),
+        'file'   => pick($destination, "${tp::data_dir}/download/${app}"),
         default  => undef,
       },
   })
@@ -482,9 +482,9 @@
     if $auto_prereq and $settings['exec_postinstall'] and $ensure != 'absent' {
       $settings[exec_postinstall].each | $k , $v | {
         if $settings[package_name] {
-          Package[$settings[package_name]] -> Exec[$k]
+          Package[$settings[package_name]] -> Exec["${app} - ${k}"]
         }
-        exec { $k:
+        exec { "${app} - ${k}":
           * => { 'path' => '/bin:/usr/bin:/sbin:/usr/sbin' } + $v,
         }
       }
@@ -516,7 +516,7 @@
     }
     if $settings[package_name] =~ String[1] and $manage_package {
       $package_defaults = {
-        ensure          => $ensure,
+        ensure          => $plain_ensure,
         provider        => $package_provider,
         source          => $package_source,
         install_options => $package_install_options,
@@ -544,7 +544,7 @@
     if $settings[git_source] {
       if ! $settings[package_name] or $settings[git_use] {
         tp::dir { $app:
-          ensure  => $ensure,
+          ensure  => tp::ensure2dir($ensure),
           path    => pick ($settings[git_destination], "/opt/${app}"),
           source  => $settings[git_source],
           vcsrepo => 'git',
@@ -642,7 +642,7 @@
     if $debug == true {
       $debug_scope = inline_template('<%= scope.to_hash.reject { |k,v| k.to_s =~ /(uptime.*|path|timestamp|free|.*password.*)/ } %>')
       file { "tp_install_debug_${sane_app}":
-        ensure  => $plain_ensure,
+        ensure  => tp::ensure2file($ensure),
         content => $debug_scope,
         path    => "${debug_dir}/tp_install_debug_${sane_app}",
       }

diff --git a/manifests/install/file.pp b/manifests/install/file.pp
@@ -87,20 +87,20 @@
 
   # Download and unpack source
   $real_version = tp::get_version($ensure,$version,$settings)
-  $real_filename = pick(tp::url_replace(getvar('settings.releases.file_name'), $real_version), $app) # lint-ignore: 140chars
+  $real_filename = pick(tp::url_replace(getvar('settings.releases.file_name'), $real_version), $app) # lint:ignore:140chars
   if getvar('settings.releases.base_url') {
     $real_base_url = pick(tp::url_replace(getvar('settings.releases.base_url'), $real_version), $app)
     $real_url = "${real_base_url}/${real_filename}"
   } else {
-    tp::fail($on_missing_data, "tp::install::file - ${app} - Missing tinydata: settings.releases.base_url") # lint-ignore: 140chars
+    tp::fail($on_missing_data, "tp::install::file - ${app} - Missing tinydata: settings.releases.base_url") # lint:ignore:140chars
   }
 
   $real_source = $ensure ? {
     'absent' => false,
     default  => pick($source, $real_url),
   }
   $extracted_dir = getvar('settings.releases.extracted_dir') ? {
-    String  => tp::url_replace(getvar('settings.releases.extracted_dir'), $real_version), # lint-ignore: 140chars
+    String  => tp::url_replace(getvar('settings.releases.extracted_dir'), $real_version), # lint:ignore:140chars
     default => tp::url_replace(basename($real_filename), $real_version),
   }
   $extracted_file = getvar('settings.releases.extracted_file')
@@ -165,7 +165,7 @@
 
     tp::setup { "tp::install::file ${app}":
       ensure          => $ensure,
-      setup_data      => 'release',
+      setup_data      => 'releases',
       source_dir      => $real_postextract_cwd,
       app             => $app,
       on_missing_data => $on_missing_data,

diff --git a/manifests/install/package.pp b/manifests/install/package.pp
@@ -252,9 +252,9 @@
   if $auto_prereq and $settings['exec_postinstall'] and $ensure != 'absent' {
     $settings[exec_postinstall].each | $k , $v | {
       if $settings[package_name] {
-        Package[$settings[package_name]] -> Exec[$k]
+        Package[$settings[package_name]] -> Exec["${app} - ${k}"]
       }
-      exec { $k:
+      exec { "${app} - ${k}":
         * => { 'path' => '/bin:/usr/bin:/sbin:/usr/sbin' } + $v,
       }
     }
@@ -373,7 +373,7 @@
         # do nothing
       }
       default: {
-        tp::fail($on_missing_data,"tp::install::package ${app} - Unsupported type for ${services}. Valid types are String, Array, Hash, Undef.") # lint-ignore:140chars
+        tp::fail($on_missing_data,"tp::install::package ${app} - Unsupported type for ${services}. Valid types are String, Array, Hash, Undef.") # lint:ignore:140chars
       }
     }
   }

diff --git a/manifests/repo.pp b/manifests/repo.pp
@@ -189,6 +189,7 @@
             exec { "Ensure ${apt_gpg_key_dir} exists for ${title}":
               command => "mkdir -p ${apt_gpg_key_dir}",
               creates => $apt_gpg_key_dir,
+              path    => $facts['path'],
             }
             # $key_nospaces = regsubst($settings[key],' ','','G')
             # $unless = "for f in /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d/*.{asc,gpg} /etc/apt/keyrings/*.{asc,gpg} ; do gpg --list-keys --keyid-format short --no-default-keyring --keyring \$f; done | grep -q \"${key_nospaces}\"",  # lint:ignore:140chars
@@ -202,7 +203,7 @@
               command     => $command,
               unless      => $unless,
               creates     => $creates,
-              path        => '/bin:/sbin:/usr/bin:/usr/sbin',
+              path        => $facts['path'],
               before      => File["${aptrepo_title}.list"],
               user        => 'root',
               environment => $exec_environment,

diff --git a/manifests/setup.pp b/manifests/setup.pp
@@ -6,7 +6,7 @@
 # @example
 #   tp::setup { 'namevar': }
 define tp::setup (
-  Enum['source','release'] $setup_data,
+  Enum['source','releases'] $setup_data,
   StdLib::Absolutepath $source_dir,
   String $app,
   Variant[Boolean,String] $ensure             = present,

diff --git a/metadata.json b/metadata.json
@@ -1,6 +1,6 @@
 {
   "name": "example42-tp",
-  "version": "3.9.0",
+  "version": "3.9.1",
   "author": "example42",
   "summary": "Tiny Puppet",
   "license": "Apache-2.0",
@@ -15,6 +15,10 @@
     {
       "name": "example42/tinydata",
       "version_requirement": ">= 0.0.1 < 5.0.0"
+    },
+    {
+      "name": "puppetlabs/vcsrepo",
+      "version_requirement": ">= 0.0.1 < 6.0.0"
     }
   ],
   "operatingsystem_support": [
@@ -113,20 +117,14 @@
   "requirements": [
     {
       "name": "puppet",
-      "version_requirement": ">= 4.9.0 < 9.0.0"
+      "version_requirement": ">= 6.0.0 < 9.0.0"
     }
   ],
-  "checksums": {
-  },
   "optional_dependencies": [
     {
       "name": "puppetlabs/chocolatey",
       "version_requirement": ">= 0.0.1 < 6.0.0"
     },
-    {
-      "name": "puppetlabs/vcsrepo",
-      "version_requirement": ">= 0.0.1 < 6.0.0"
-    },
     {
       "name": "puppetlabs/concat",
       "version_requirement": ">= 0.0.1 < 6.0.0"

diff --git a/spec/functions/tp_lookup_spec.rb b/spec/functions/tp_lookup_spec.rb
@@ -3,7 +3,7 @@
 apps = ['rsyslog','openssh','elasticsearch','sysdig','icinga2']
 
 describe 'tp_lookup' do
-  on_supported_os(facterversion: '2.4').select { |k, _v| k == 'centos-7-x86_64' || k == 'ubuntu-16.04-x86_64' }.each do |os, os_facts|
+  on_supported_os(facterversion: '2.4').select { |k, _v| k == 'redhat-8-x86_64' || k == 'ubuntu-22.04-x86_64' }.each do |os, os_facts|
     context "on #{os}" do
       let(:facts) { os_facts }
       context "the tp_lookup function" do