Description of problem:

The smart-proxy show page (e.g. https://foreman.example.com/smart_proxies/3-py-proxy-example-com) triggers ajax call on load to fetch info about number of puppet modules and environments on the proxy - this involves iterating / scanning on every environment on the capsule and caching. Given the large number of environments, this present a significant load and is causing smart-proxy process to exhaust all system memory.

How reproducible:

100% Reproducible with a capsule running 32GB RAM and 4000 puppet environments.
Size of all puppet environments is around 50GB

Steps to Reproduce:

1. Configure a capsule with a big number of puppet environments. In my case 4159 environments on a 32GB memory proxy
2. Set foreman-proxy log to DEBUG
3. Access to capsule info show page
4. Check that several GET requests are triggered on proxy.log
GET /puppet/environments HTTP/1.1
GET /pulpnode/status/disk_usage
GET /puppet/environments/production/classes
5. foreman-proxy start to scan all directories in alphabetical order:

D, [2017-12-12T14:17:11.536456 #13791] DEBUG -- : Running scan_directory on _temp_10194_example_com: /etc/puppet/environments/_temp_10194_xxx_es/modules
D, [2017-12-12T14:17:11.536913 #13791] DEBUG -- : Scanning concat classes in /etc/puppet/environments/_temp_10194_example_com/modules/concat/manifests/fragment.pp
I, [2017-12-12T14:17:11.537262 #13791] INFO -- : Initializing from Puppet config file: /etc/puppet/puppet.conf
D, [2017-12-12T14:17:11.572148 #13791] DEBUG -- : Scanning concat classes in /etc/puppet/environments/_temp_10194_example_com/modules/concat/manifests/init.pp
I, [2017-12-12T14:17:11.572546 #13791] INFO -- : Initializing from Puppet config file: /etc/puppet/puppet.conf
D, [2017-12-12T14:17:11.638181 #13791] DEBUG -- : Scanning concat classes in /etc/puppet/environments/_temp_10194_example_com/modules/concat/manifests/setup.pp
I, [2017-12-12T14:17:11.638578 #13791] INFO -- : Initializing from Puppet config file: /etc/puppet/puppet.conf
D, [2017-12-12T14:17:11.664910 #13791] DEBUG -- : Scanning easy_type classes in /etc/puppet/environments/_temp_10194_example_com/modules/easy_type/manifests/init.pp
I, [2017-12-12T14:17:11.665280 #13791] INFO -- : Initializing from Puppet config file: /etc/puppet/puppet.conf
D, [2017-12-12T14:17:11.783869 #13791] DEBUG -- : Scanning filemapper classes in /etc/puppet/environments/_temp_10194_example_com/modules/filemapper/manifests/init.pp
I, [2017-12-12T14:17:11.784218 #13791] INFO -- : Initializing from Puppet config file: /etc/puppet/puppet.conf

Actual results:

foreman-proxy starts to consume more and more memory until reaches swap.

Expected results:

foreman-proxy do not consume all capsule memory available.

Additional info:

There is a workaround that prevents scanning / initializing all puppet classes and enviroments when accessing capsule info webpage. Basically consists on comment line 6 /usr/share/foreman/app/services/proxy_status/puppet.rb: https://github.com/theforeman/foreman/blob/develop/app/services/proxy_status/puppet.rb#L6

However, this workaround would be lost after an update