Foreman » Smart Proxy

12/08/2016

05:31 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

Created http://projects.theforeman.org/issues/17615 for KEA DHCP provider feature.
Let's move all KEA-related conver... Konstantin Orekhov

05:30 PM Feature #17615 (New): Introduce KEA DHCP provider for smart-proxy

Introduce KEA DHCP provider for smart-proxy.
KEA is next-gen fast milti-node DHCP server by ISC. It uses DB backen... Konstantin Orekhov

05:23 PM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

Dmitri Dolguikh wrote:
> You might be able to get syncd to work with smart-proxy, but this will require changes to s... Konstantin Orekhov

02:16 AM Feature #4917 (Ready For Testing): Smart-Proxy Realm Provider for Active Directory

The Foreman Bot

12/07/2016

08:01 AM Bug #17045 (Closed): Templates Smart Proxy API does not pass Hostgroup provisioning

Applied in changeset commit:3b81b1d90378e23bc10f9100c6d8f243379894f0. Anonymous

07:49 AM Revision 3b81b1d9: Fixes #17045 - hostgroup templates URLs proxied correctly

Lukas Zapletal

12/06/2016

05:27 AM Revision fde33bdd: Bump version to 1.15-develop

Dominic Cleal

12/05/2016

10:39 AM Bug #17566: TFTP provisioning fails when grub2 directory does not exist

Was there any PXEGrub2 template associated with the host?
Can you share the proxy.log bit after you created the mi... Lukas Zapletal

12/02/2016

08:43 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> It is not what I've been talking about at all. I've been trying to point out that it is not possible to not drop ... Konstantin Orekhov

04:34 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Well, if it is OK with Foreman developers and community to sacrifice the performance and scope of Foreman as produc... Anonymous

01:38 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> > So, if we step back and review the requirements for caching leases, what would that be? You mentioned keeping t... Konstantin Orekhov

01:34 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> > Well, yes, currently it is pretty much non-existent, but if it were, would you be interested in picking this up... Konstantin Orekhov

12:30 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> So, if we step back and review the requirements for caching leases, what would that be? You mentioned keeping track... Anonymous

12:26 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Well, yes, currently it is pretty much non-existent, but if it were, would you be interested in picking this up as ... Anonymous

12:24 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> There is no way we can drop local state without reducing functionality/scope of smart proxy dhcp api.
Caching ... Konstantin Orekhov

12:10 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Kea has no management api whatsoever (it's a step backwards compared to dhcpd in this regard), which makes it pre... Konstantin Orekhov

04:05 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> If the above approach (no-caching, on-demand queries only) make sense to you, I strongly believe it could be implem... Anonymous

04:02 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

Kea has no management api whatsoever (it's a step backwards compared to dhcpd in this regard), which makes it pretty ... Anonymous

02:39 PM Bug #17566 (Rejected): TFTP provisioning fails when grub2 directory does not exist

After installation of the OS is complete the PXE boot configuration does not get switched to boot off the local disk.... Matt Spaulding

04:41 AM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

You might be able to get syncd to work with smart-proxy, but this will require changes to syncd: on modify event it w... Anonymous

04:22 AM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

> Meanwhile, I have a question for you, Dmitri - is it possible to handle :moved_to event the same way as :modify and... Anonymous

12/01/2016

06:25 PM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

Well, to follow up on what I mentioned above - there's a difference in inotify events on a system that actually runs ... Konstantin Orekhov

05:59 PM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

Alas, I don't see any improvements in large deployment - as soon as inotify events start happening (i'm rsync'ing dhc... Konstantin Orekhov

02:17 PM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

I run ruby 2.0 included with CentOS 7:
ruby 2.0.0p598 (2014-11-13) [x86_64-linux]
I'll test your PR and let you... Konstantin Orekhov

12:23 PM Bug #17554: ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

Konstantin, what ruby version are you using? I think the PR will resolve the problem you are seeing, but I couldn't v... Anonymous

12:12 PM Bug #17554 (Ready For Testing): ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

The Foreman Bot

12:07 PM Bug #17554 (Closed): ISC dhcpd provider leases monitor doesn't keep up with lease file updates in a large network

Anonymous

02:58 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Konstantin, out of curiousity: I am working (#13806) on dnsmasq smart-proxy provider that will work fast. It's a sm... Konstantin Orekhov

12:19 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

I reopened the issue and also created a ticket dedicated to inotify watcher not being able to keep up with the events... Anonymous

12:14 PM Bug #17373 (New): ISC dhcp provider is unable to handle very big networks

Anonymous

05:52 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

Konstantin, out of curiousity: I am working (#13806) on dnsmasq smart-proxy provider that will work fast. It's a smal... Lukas Zapletal

12:39 PM Bug #17556 (Rejected): ISC dhcpd provider potentially discards leases too early or too late

Time comparison on this line: https://github.com/theforeman/smart-proxy/blob/develop/modules/dhcp_isc/isc_state_chang... Anonymous

11/30/2016

08:46 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

Well, the underlying issue is not resolved, though. It is unfortunate that you think that it is resolved.
As I was... Konstantin Orekhov

11/28/2016

05:59 PM Bug #17506 (Closed): smart-proxy Error while getting IP from AD DHCP (undefined method `key?')

Anonymous

05:56 PM Bug #17506 (Resolved): smart-proxy Error while getting IP from AD DHCP (undefined method `key?')

Thanks guys, I've applied the change in your commit and that fixed the issue for me. Kevin Carrasco

03:20 PM Bug #17506: smart-proxy Error while getting IP from AD DHCP (undefined method `key?')

The fix merged in https://github.com/theforeman/smart-proxy/commit/9bee3fca64f756b761076af14f37e99d054cb693. Anonymous

03:16 PM Bug #17506: smart-proxy Error while getting IP from AD DHCP (undefined method `key?')

I guess this is related to #17387 Anonymous

02:59 PM Bug #17506 (Closed): smart-proxy Error while getting IP from AD DHCP (undefined method `key?')

After upgrading Foreman (CentOS 7) and Smart-Proxy (Windows) from 1.13.1 to 1.13.2 I'm getting the following error on... Kevin Carrasco

06:06 AM Feature #17500 (Closed): Introduce providers for realm module

Anonymous

06:03 AM Refactor #17499 (New): Leases by mac and hosts by mac mappings in subnet service do not need to be grouped by subnet

Anonymous

05:58 AM Bug #17373 (Resolved): ISC dhcp provider is unable to handle very big networks

I'm closing this issue: the root cause has been addressed by http://projects.theforeman.org/issues/17387. Anonymous

11/25/2016

12:48 PM Support #17462 (Resolved): failed to import classes

It seems i missread doc, all ok and thank you. Slagian Momirovici

11:45 AM Support #17462: failed to import classes

Dominic, works fine ), thank you very much --> so the issue was ruby version, my fault
I've setup also directory env... Slagian Momirovici

11/24/2016

06:30 AM Support #17462: failed to import classes

Thanks, that does confine the problem to the smart proxy, but doesn't show any likely cause.
Re-reading the descri... Dominic Cleal

06:10 AM Support #17462: failed to import classes

Hi Dominic,
Back with answers:
*Note:* I have left only 2 modules, removed * of them for debugging purposes. ... Slagian Momirovici

04:52 AM Support #17462: failed to import classes

oki, I will test this today and update. Slagian Momirovici

04:00 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Parsing leases file does not provide network info either - all of that comes from smart-proxy parsing of config fil... Anonymous

11/23/2016

07:19 PM Feature #2482 (Ready For Testing): Global search in DHCP reservations/leases using MAC and/or hostname

The Foreman Bot

07:17 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Omapi is limited to lookup/create/update/delete for leases, reservations, and a couple more objects. We need to par... Konstantin Orekhov

04:30 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Before I submit a PR, would you please review https://github.com/korekhov/smart-proxy/commit/ce25338033c4d5bb80c57d... Anonymous

04:17 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> I was asking that question in my original issue, but never got the answer - why smart-proxy needs to read leases fi... Anonymous

08:34 AM Support #17462: failed to import classes

The smart proxy log looks reasonably successful, it's listing manifests that are being parsed for classes. I'd sugges... Dominic Cleal

07:26 AM Support #17462 (Resolved): failed to import classes

Hi,
it seems I am having some problems when importing classes. After a lot of trials I think I should raise this is... Slagian Momirovici

04:43 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

I don't think we ought to discuss degrees of horribleness when we know we can do better.
To reiterate: If we were... Anonymous

03:46 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

Dmitri, external DHCP setup is fully supported feature of Satellite 6.1+ and documented here:
https://access.redha... Lukas Zapletal

11/22/2016

08:18 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> > The second thing is a global search for a MAC, IP or hostname - I went through current DHCP APIs and I don't see ... Konstantin Orekhov

06:24 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

Going back to performance, HA and such.
I was asking that question in my original issue, but never got the answer - ... Konstantin Orekhov

01:30 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Is this with the dataset in the original ticket? Lease record won't get loaded now if it's expired (based on 'ends'... Konstantin Orekhov

12:57 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> I wonder if this would be better implemented on Foreman side, as it would be easier to manage the whitelist from th... Konstantin Orekhov

12:42 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> It is mostly about keeping Foreman itself in a cleaner state - I'd like to have an ability to filter out subnets th... Anonymous

12:27 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

> Is this is a performance or a security concern? If it's the former, filtering out networks won't affect loading/rel... Konstantin Orekhov

08:01 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> There are only a couple of things left - regular expressions in white/blacklist to include/exclude networks. In my ... Anonymous

07:58 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

Konstantin Orekhov wrote:
> There are only a couple of things left - regular expressions in white/blacklist to inclu... Dominic Cleal

07:25 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

> One more thing I just realized - none of the leases were actually loaded, only host reservations. Is this by design... Anonymous

12:04 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

This is not an issue in 1.11.x and prior as far as I can tell, but 1.13.1 has it for sure. Don't have 1.12 anymore, s... Konstantin Orekhov

08:02 AM Bug #17387 (Closed): Speed up Proxy::DHCP::SubnetService#find_subnet

Applied in changeset commit:c16b176dd1e258188e7bc158247b3319c80e6576. Dominic Cleal

07:44 AM Revision c16b176d: fixes #17387 - SubnetService#find_subnet has constant time lookup

find_subnet is now approximately constant with the number of subnets
configured, using hash lookups of possible netwo... Dominic Cleal

11/21/2016

09:06 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

One more thing I just realized - none of the leases were actually loaded, *only* host reservations. Is this by design? Konstantin Orekhov

11/20/2016

05:22 PM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

> While not ideal, NFS is still a very important part of HA strategies, mainly because of its simplicity. As we discu... Anonymous

04:18 PM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

While not ideal, NFS is still a very important part of HA strategies, mainly because of its simplicity. As we discuss... Konstantin Orekhov

11/19/2016

04:08 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

Confirmed - the fix from https://github.com/theforeman/smart-proxy/pull/476 is amazing!
It only takes 30 sec to lo... Konstantin Orekhov

11/18/2016

06:25 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

It looks like the pause is due to ruby VM garbage collector which runs several major and minor garbage collections in... Anonymous

04:50 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

On ruby 2.3.1 with the changes in http://projects.theforeman.org/issues/17387 applied, smart-proxy starts in less tha... Anonymous

11/17/2016

06:18 PM Bug #17373: ISC dhcp provider is unable to handle very big networks

To answer some of your questions, guys:
1. This particular instance has 30K host records and ~82K lease records in... Konstantin Orekhov

07:10 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

Dmitri Dolguikh wrote:
> We should be seeing debug output if leases/reservations are being added/removed, I'd say th... Dominic Cleal

06:51 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

We should be seeing debug output if leases/reservations are being added/removed, I'd say the thread is blocked/hangin... Anonymous

06:08 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

Konstantin, I suspect that the pauses are caused by large numbers of host { } entries being processed. Could you indi... Dominic Cleal

05:31 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

The stack trace from when the USR1 signal arrived is quite telling:... Dominic Cleal

05:12 AM Bug #17373: ISC dhcp provider is unable to handle very big networks

I don't think filtering on networks would help. We need to parse individual records before making a decision whether ... Anonymous

05:05 AM Bug #17373 (New): ISC dhcp provider is unable to handle very big networks

After the upgrade and modifications to dhcp.yml, restart of smart-proxy took 4 hours (based on the debug messages in ... Anonymous

12:04 PM Bug #17387 (Ready For Testing): Speed up Proxy::DHCP::SubnetService#find_subnet

The Foreman Bot

12:02 PM Bug #17387 (Closed): Speed up Proxy::DHCP::SubnetService#find_subnet

Current implementation is O(n), it can be made O(log n). Anonymous

02:59 AM Bug #2687: Performance issues with large ISC dataset (DHCP smart proxy)

Please open a new issue for any further problems, this one is linked to already shipped changes so won't be re-opened. Dominic Cleal

11/16/2016

08:33 PM Bug #2687: Performance issues with large ISC dataset (DHCP smart proxy)

Hello, guys!
Thanks for working on this, but I still have any issue even after upgrading to 1.13.1. After the upgr... Konstantin Orekhov

10:33 AM Bug #14808 (Duplicate): Foreman-Proxy DHCP class doesn't properly read subnets containing if statements

This appears to be a duplicate of http://projects.theforeman.org/issues/16079 which has now been resolved. Anonymous

10:24 AM Bug #1280 (Resolved): MS DHCP ignores address exclusions

This has been fixed in Ms dhcp provider on Windows Server 2012 platform (please note there won't be a fix for Windows... Anonymous

10:21 AM Bug #6916 (Resolved): MS DHCP Timeout

Ms dhcp provider has been re-written and the issue should be resolved now. I'm going to close the ticket, please re-o... Anonymous

10:15 AM Refactor #12394 (Resolved): Netsh Deprecated in Windows 2012R2

The new version of ms dhcp provider doesn't rely on netsh. Closing the ticket. Anonymous

10:14 AM Bug #13420 (Resolved): [Smart proxy] provider dhcp_native_ms DELETE method broken

Ms dhcp provider has been re-written (see http://projects.theforeman.org/issues/7766), and this should be resolved. P... Anonymous

10:08 AM Bug #15451 (Resolved): Occasionally a MS DHCP delete operation fails

Ms dhcp provider has been re-written (see http://projects.theforeman.org/issues/7766), it's unclear if this is applic... Anonymous

10:05 AM Feature #15554: Add lease deletion facility to dhcp providers

Re-wording the ticket, as this is applicable to more than just ms dhcp provider. Anonymous

10:02 AM Bug #15789 (Resolved): In MS DHCP provider general/Vendor options must exist before assigning values to them

This isn't applicable to the rewritten MS DHCP provider (see http://projects.theforeman.org/issues/7766). Closing the... Anonymous

10:02 AM Feature #15874 (Closed): Log X-Request-ID by default

Applied in changeset commit:b39011ebe34b83442bb8c256dd400f494f235389. Anonymous

10:00 AM Bug #16056 (Resolved): Non-english locale casuses "Vendor Class not found" errors when using windows dhcp provider

Anonymous

09:56 AM Bug #16056 (Closed): Non-english locale casuses "Vendor Class not found" errors when using windows dhcp provider

http://projects.theforeman.org/issues/7766 which resolves this issue has now been merged. I'm closing this ticket. Anonymous

09:39 AM Revision b39011eb: Fixes #15874 - added logging of 'X_REQUEST_ID' http headers

Dmitri Dolguikh

08:01 AM Bug #17172 (Closed): SSH BMC provider does not return IP address

Applied in changeset commit:012ccb55ed5a12dedbbf21162c0bc4aab17c2724. Anonymous

08:01 AM Bug #16079 (Closed): ISC DHCP subnets fail to parse with host or class stanzas inside

Applied in changeset commit:7f016ecae66ceb53f2144fb176092a2c034c62bb. Anonymous

07:33 AM Revision 012ccb55: Fixes #17172 - SSH BMC network details implementation

Lukas Zapletal

07:28 AM Revision 7f016eca: Fixes #16079 - subnet recursive parsing

Lukas Zapletal

11/14/2016

09:18 AM Bug #17327: Unable to use multiple FreeIPA servers to enroll to REALM

Should this be filed in the smart proxy if it's meant to fall back to a secondary URL? Or are you implying it should ... Dominic Cleal

09:12 AM Bug #17327 (New): Unable to use multiple FreeIPA servers to enroll to REALM

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1349487
Description of problem:
We have an IDM solution wi... Stephen Benjamin

08:01 AM Refactor #17037 (Closed): Replace Resolv::DNS#getname calls with Resolv::DNS#getresource

Applied in changeset commit:522231bf3e4c918ee821a73acb3f70459ca57791. Anonymous

07:13 AM Revision 522231bf: Fixes #17037 - Resolver::DNS#getresources is used now.

It replaces calls to Resolver::DNS#getaddresses and
Resolver::DNS#getnames in dns module and its providers. Dmitri Dolguikh

07:11 AM Feature #15874 (Ready For Testing): Log X-Request-ID by default

The Foreman Bot

11/11/2016

12:19 PM Feature #17319 (Rejected): Handle kerberos ticket expiration in Gss nsupdate module

When using Windows 2008 AD, kerberos ticket is retained by the module after its expiration. This leads to failures wh... Anonymous

06:54 AM Bug #16079 (Ready For Testing): ISC DHCP subnets fail to parse with host or class stanzas inside

The Foreman Bot

04:20 AM Bug #16079: ISC DHCP subnets fail to parse with host or class stanzas inside

Regression introduced by #12211 Lukas Zapletal

11/10/2016

10:08 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

Please see the description of the problem above. Accessing leases file over NFS isn't going to work with this impleme... Anonymous

10:04 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

> Restarting the smart-proxy after deleting the host does not remove the entry from the leases file.
Restart it be... Dominic Cleal

09:59 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

Yes, we're mounting over NFS. The smart-proxy and DCHP server are in the same cluster and traffic is layer 2 so laten... Eli Landon

04:20 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

Accessing leases file over NFS will absolutely destroy the performance: we no longer will be able to track updates an... Anonymous

04:12 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

I think NFS support is important, as I know multiple users who uses HA solution for dhcp proxy using shared NFS stora... Ohad Levy

04:11 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

does it make sense to check if its mounted as NFS ?
also, should we provide a fall back option? Ohad Levy

03:55 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

> Environment:
> ISC-DHCP-Server version 4.3.3 on Ubuntu 16.04 LTS server
> Smart-Proxy version 1.13.1 on separate ... Anonymous

03:36 AM Bug #17301: ISC DHCP known reservations/leases not updated over NFS

1) Are you using NFS here?
> ISC-DHCP-Server version 4.3.3 on Ubuntu 16.04 LTS server
> Smart-Proxy version 1.13.... Dominic Cleal

09:01 AM Bug #7766 (Closed): ms_native dhcp smart proxy code scales poorly

Applied in changeset commit:14d8e52338553fbf6a54587a4e2459c4ffb86fb8. Anonymous

08:36 AM Revision 14d8e523: Fixes #7766: ms dhcp provider uses native dhcps api now.

Dmitri Dolguikh

11/09/2016

05:49 PM Bug #17301 (Closed): ISC DHCP known reservations/leases not updated over NFS

After upgrading to 1.13.1 the DHCP behavior is not consistent -
* Adding a host successfully retrieves an available ... Eli Landon

11:01 AM Bug #17125 (Closed): Allow windows service name to be specified

Applied in changeset commit:65d1b8de480e7dd911d76271c03014d37b538147. Alex Fisher

10:36 AM Revision 65d1b8de: fixes #17125 - Prompt for Windows service name

Useful if you need to install multiple proxy services on the same
Windows server. Alex Fisher

06:55 AM Feature #17289 (New): Call stop on running async services on proxy shutdown

Anonymous

06:53 AM Feature #15874 (Assigned): Log X-Request-ID by default

Anonymous