Foreman » Installer

Ohai,

after installing a fresh 1.15 (using forklift) everything is working fine:

[root@centos7-foreman-1-15 ~]# systemctl status puppetserver.service 
● puppetserver.service - puppetserver Service
   Loaded: loaded (/usr/lib/systemd/system/puppetserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-06-13 06:13:28 UTC; 1min 19s ago
 Main PID: 4996 (java)
   CGroup: /system.slice/puppetserver.service
           └─4996 /usr/bin/java -Xms2G -Xmx2G -XX:MaxPermSize=256m -Djava.security.egd=/dev/urandom -XX:OnOutOfMemoryError=kill -9 %p -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d --bootstrap-config /...

Jun 13 06:12:54 centos7-foreman-1-15.example.com systemd[1]: Starting puppetserver Service...
Jun 13 06:12:54 centos7-foreman-1-15.example.com puppetserver[4987]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Jun 13 06:13:28 centos7-foreman-1-15.example.com systemd[1]: Started puppetserver Service.

However, when I try to restart the puppetserver, it errors out:

[root@centos7-foreman-1-15 ~]# systemctl restart puppetserver
Job for puppetserver.service failed because the control process exited with error code. See "systemctl status puppetserver.service" and "journalctl -xe" for details.

[root@centos7-foreman-1-15 ~]# journalctl -xe
…
-- Unit puppetserver.service has begun starting up.
Jun 13 06:15:28 centos7-foreman-1-15.example.com puppetserver[5697]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: Exception in thread "main" java.io.FileNotFoundException: /etc/puppetlabs/puppet/ssl/crl.pem (Permission denied)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.io.FileOutputStream.open0(Native Method)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.io.FileOutputStream.open(FileOutputStream.java:270)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.java.io$fn__9570.invokeStatic(io.clj:355)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.java.io$fn__9570.invoke(io.clj:354)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.lang.MultiFn.invoke(MultiFn.java:238)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.java.io$copy.invokeStatic(io.clj:406)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.java.io$copy.doInvoke(io.clj:391)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.lang.RestFn.invoke(RestFn.java:425)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at me.raynes.fs$copy.invokeStatic(fs.clj:293)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at me.raynes.fs$copy.invoke(fs.clj:289)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.puppetserver.certificate_authority$eval16660$retrieve_ca_crl_BANG___16665$fn__16666.invoke(certificate_authority.clj:752)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.puppetserver.certificate_authority$eval16660$retrieve_ca_crl_BANG___16665.invoke(certificate_authority.clj:744)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.services.ca.certificate_authority_service$reify__24897$service_fnk__5222__auto___positional$reify__24908.retrieve_ca_crl_BANG_(certificate_authority_service.clj:52)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.services.protocols.ca$eval24835$fn__24836$G__24825__24839.invoke(ca.clj:3)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.services.protocols.ca$eval24835$fn__24836$G__24824__24843.invoke(ca.clj:3)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core$partial$fn__4759.invoke(core.clj:2515)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.services.master.master_service$reify__33186$service_fnk__5222__auto___positional$reify__33207.init(master_service.clj:52)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.services$eval5024$fn__5025$G__5012__5028.invoke(services.clj:8)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.services$eval5024$fn__5025$G__5011__5032.invoke(services.clj:8)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13792$run_lifecycle_fn_BANG___13799$fn__13800.invoke(internal.clj:204)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13792$run_lifecycle_fn_BANG___13799.invoke(internal.clj:187)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13821$run_lifecycle_fns__13826$fn__13827.invoke(internal.clj:238)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13821$run_lifecycle_fns__13826.invoke(internal.clj:215)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval14291$build_app_STAR___14300$fn$reify__14310.init(internal.clj:588)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval14337$boot_services_for_app_STAR__STAR___14344$fn__14345$fn__14347.invoke(internal.clj:616)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval14337$boot_services_for_app_STAR__STAR___14344$fn__14345.invoke(internal.clj:615)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval14337$boot_services_for_app_STAR__STAR___14344.invoke(internal.clj:609)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core$partial$fn__4761.invoke(core.clj:2521)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13860$initialize_lifecycle_worker__13871$fn__13872$fn__13958$state_machine__11832__auto____13959$fn__13961.invoke(internal.clj:255)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at puppetlabs.trapperkeeper.internal$eval13860$initialize_lifecycle_worker__13871$fn__13872$fn__13958$state_machine__11832__auto____13959.invoke(internal.clj:255)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:1012)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:1011)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:1016)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:1014)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async$ioc_alts_BANG_$fn__12000.invoke(async.clj:383)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async$do_alts$fn__11946$fn__11949.invoke(async.clj:252)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.core.async.impl.channels.ManyToManyChannel$fn__6756$fn__6757.invoke(channels.clj:95)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at clojure.lang.AFn.run(AFn.java:22)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: at java.lang.Thread.run(Thread.java:748)
Jun 13 06:15:57 centos7-foreman-1-15.example.com puppetserver[5697]: Background process 5704 exited before start had completed
Jun 13 06:15:57 centos7-foreman-1-15.example.com systemd[1]: puppetserver.service: control process exited, code=exited status=1
Jun 13 06:15:57 centos7-foreman-1-15.example.com systemd[1]: Failed to start puppetserver Service.
-- Subject: Unit puppetserver.service has failed
…

The exception is slightly misleading, as the file is readable by Puppet just fine:

[root@centos7-foreman-1-15 ~]# ls -alh /etc/puppetlabs/puppet/ssl/
total 4.0K
drwxrwx--x. 8 root   puppet 126 Jun 13 06:12 .
drwxr-xr-x. 3 root   root   127 Jun 13 06:12 ..
drwxr-xr-x. 5 puppet puppet 158 Jun 13 06:12 ca
drwxr-xr-x. 2 root   puppet   6 Jun 13 06:12 certificate_requests
drwxr-xr-x. 2 root   puppet  64 Jun 13 06:12 certs
-rw-r--r--. 1 root   puppet 987 Jun 13 06:13 crl.pem
drwxr-x---. 2 root   puppet   6 Jun 13 06:05 private
drwxr-x---. 2 root   puppet  50 Jun 13 06:12 private_keys
drwxr-xr-x. 2 root   puppet  50 Jun 13 06:12 public_keys

But what it actually wants, is being able to write that file, and thus changing the owner to "puppet" fixes the issue:

[root@centos7-foreman-1-15 ~]# chown puppet /etc/puppetlabs/puppet/ssl/crl.pem 
[root@centos7-foreman-1-15 ~]# systemctl restart puppetserver
[root@centos7-foreman-1-15 ~]# systemctl status puppetserver.service
● puppetserver.service - puppetserver Service
   Loaded: loaded (/usr/lib/systemd/system/puppetserver.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-06-13 06:22:01 UTC; 50s ago
  Process: 5679 ExecStop=/opt/puppetlabs/server/apps/puppetserver/bin/puppetserver stop (code=exited, status=0/SUCCESS)
  Process: 7760 ExecStart=/opt/puppetlabs/server/apps/puppetserver/bin/puppetserver start (code=exited, status=0/SUCCESS)
 Main PID: 7767 (java)
   CGroup: /system.slice/puppetserver.service
           └─7767 /usr/bin/java -Xms2G -Xmx2G -XX:MaxPermSize=256m -Djava.security.egd=/dev/urandom -XX:OnOutOfMemoryError=kill -9 %p -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d --bootstrap-config /...

Jun 13 06:21:35 centos7-foreman-1-15.example.com systemd[1]: Starting puppetserver Service...
Jun 13 06:21:35 centos7-foreman-1-15.example.com puppetserver[7760]: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Jun 13 06:22:01 centos7-foreman-1-15.example.com systemd[1]: Started puppetserver Service.

Note, setting the perms to 664 does not help, as Puppet itself tries to enforce the 644 perms.