System Administration & Network Administration
puppet puppetmaster r10k
Updated Wed, 21 Sep 2022 13:29:56 GMT

Unable to pull Puppet Forge modules with r10k


I have what appears to be a functioning r10k setup with respect to publishing branches as environments, but when I run the command to pull modules via

sudo /opt/puppetlabs/puppet/bin/r10k deploy environment -p -v

I get

INFO     -> Deploying environment /etc/puppetlabs/code/environments/production
INFO     -> Environment production is now at cd62f3bb36117b1418672a1039b8212aa8bb36dd
INFO     -> Deploying Puppetfile content /etc/puppetlabs/code/environments/production/modules/ntp
ERROR    -> stream error
INFO     -> Deploying Puppetfile content /etc/puppetlabs/code/environments/production/modules/stdlib
ERROR    -> stream error

My /etc/puppetlabs/r10k/r10k.yaml

:cachedir: '/var/cache/r10k'
# A list of git repositories to create
:sources:
  :operations:
    remote: 'git@git:sysops/puppet.git'
    basedir: '/etc/puppetlabs/code/environments'

My Puppetfile

forge 'forge.puppetlabs.com'
mod 'puppetlabs/ntp', '4.1.0'
mod 'puppetlabs/stdlib'

Neither of those seem relevant to me, just thought I'd add them to preempt that question. If I do a packet capture of HTTPS on my puppet host I get a bunch of outbound traffic like this

15:19:14.507310 IP puppet.domain.net.33955 > ec2-52-10-130-237.us-west-2.compute.amazonaws.com.https: Flags [.], ack 17926, win 545, options [nop,nop,TS val 129210401 ecr 2944333247], length 0

and a bunch of inbound like

15:19:14.520146 IP ec2-52-10-130-237.us-west-2.compute.amazonaws.com.https > puppet.domain.net.33955: Flags [.], seq 20822:22270, ack 628, win 227, options [nop,nop,TS val 2944333252 ecr 129210401], length 1448

So things are communicating, but still the fairly useless "stream error" from r10k. If you turn up debugging with -v debug2 you get the same messages on the modules, just a bunch before and a bunch after.

This is a similar question to https://stackoverflow.com/questions/35712872/r10k-ruby-connection-error-to-the-forge but that question was marked as solved without actually being solved, so not a duplicate IMO.




Solution

It seems that while the ruby rugged module is being built(which is a r10k dependency) it's using zlib symbols compiled in libgit2 instead of those provided by the operating system. For me, it was enough to remove rugged module gem, install zlib-devel package on the system and install rugged again.

gem uninstall rugged
yum install zlib-devel
gem install rugged

If you want to confirm your problem is the same, just run the code below in the ruby console(irb):

require 'rugged'
require 'zlib'
Zlib::Inflate.new(32 + Zlib::MAX_WBITS)

It should raise a Zlib::StreamError.

Sources that led me to this soulution:

https://github.com/prontolabs/pronto/issues/23

https://github.com/libgit2/rugged/issues/705





Comments (1)

  • +0 – I'd given up on this a long time ago but that fixes it, I guess I can give it another go. Thanks! — Jan 09, 2018 at 21:33