joeware - never stop exploring... :)

Ok, let me pull that title back a teeny tiny bit…

Do NOT Use msds-memberTransitive and msds-memberOfTransitive UNLESS you are absolutely positive that the number of values returned is guaranteed to be less than 4501 values.

What are you talking about joe and why are you railing about an attribute that has been around since Windows Server 2012 R2 and you haven’t let out the slightest peep about it before now???

The answer as to why I haven’t gotten the word out on this before is simply that I have been busy. But I digress…

Back to specific tirade for this issue… So while I had previously looked at msds-memberTransitive and msds-memberOfTransitive in a cursory manner and saw that yes, if I use it on administrators it shows me all 10 or less people/groups with that membership no matter how nested I didn’t have an opportunity to perform any real testing on it, much like MSFT apparently.  

[Sun 04/18/2021 16:40:20.90]
E:\DEV\cpp\vs\AdFind\Release>adfind -f name=administrators -dsq | adfind msds-membertransitive -base

AdFind V01.56.00cppBETA Joe Richards (support@joeware.net) April 2021

Using server: LO-DC4.lockout.test.loc:389
Directory: Windows Server 2019 (10.0.17134.1)

dn:CN=Administrators,CN=Builtin,DC=lockout,DC=test,DC=loc
> msds-memberTransitive: CN=$$joe,CN=Users,DC=lockout,DC=test,DC=loc
> msds-memberTransitive: CN=$joe,CN=Users,DC=lockout,DC=test,DC=loc
> msds-memberTransitive: CN=Domain Admins,CN=Users,DC=lockout,DC=test,DC=loc
> msds-memberTransitive: CN=Enterprise Admins,CN=Users,DC=lockout,DC=test,DC=loc
> msds-memberTransitive: CN=Administrator,CN=Users,DC=lockout,DC=test,DC=loc

1 Objects returned

See… That actually IS pretty cool.

Certainly easier than trying to work through using LDAP_MATCHING_RULE_IN_CHAIN like so:

[Sun 04/18/2021 16:50:09.50]
E:\DEV\cpp\vs\AdFind\Release>adfind -f memberof:NEST:=CN=Administrators,CN=Builtin,DC=lockout,DC=test,DC=loc -bit -dn

AdFind V01.56.00cppBETA Joe Richards (support@joeware.net) April 2021

Transformed Filter: memberof:1.2.840.113556.1.4.1941:=CN=Administrators,CN=Builtin,DC=lockout,DC=test,DC=loc
Using server: LO-DC4.lockout.test.loc:389
Directory: Windows Server 2019 (10.0.17134.1)
Base DN: DC=lockout,DC=test,DC=loc

dn:CN=Administrator,CN=Users,DC=lockout,DC=test,DC=loc
dn:CN=Enterprise Admins,CN=Users,DC=lockout,DC=test,DC=loc
dn:CN=Domain Admins,CN=Users,DC=lockout,DC=test,DC=loc
dn:CN=$joe,CN=Users,DC=lockout,DC=test,DC=loc
dn:CN=$$joe,CN=Users,DC=lockout,DC=test,DC=loc

5 Objects returned

So, there you go, I didn’t look at it in any deep manner, just a quick and dirty check. For example I didn’t test (but I assumed based on how AD works) that it had the same issues I have pointed out previously with the Attribute Scoped Query control and Nested Membership control where if a linked value is out of scope for the DSA processing the request it will be missed so you won’t get a complete picture. People with single domains don’t have to worry about it (yet another reason to keep your forests very simple) but people with multiple domains in a forest need to be aware and watching out for it if they are using these advanced cool features.

Note that I *still* haven’t tested that “missing information due to DSA scope use case” because something else has been brought to my attention now that is even more silly. To be blunt, they implemented these constructed attributes completely outside the normal standards used in AD for returning multiple values. I am forced to wonder if the person who wrote the code for this functionality had ever used AD and pulled groups with large memberships before. Problems like this is what you see out of small time vendors who have an AD with 20 groups and 10 people and use that for all of their testing.

So the problem I am raising here with these attributes, msds-memberTransitive and msds-memberOfTransitive, comes into play when it has to leverage value ranging and its complete failure to do it properly.

You should already know what value ranging is, but as a reminder… Value ranging is when the number of values returned for an attribute would exceed the 1500 values limit policy that Active Directory has (MaxValRange) such that it sends back the values in batches of 1500 members. The LDAP client gets a response from the query for the object and attribute with the attribute listed but with no values and another attribute entry of the format attributename;range=0-1499 that contains the first batch of 1500 values (e.g. member;range=0-1499). This functionality is to help protect AD. If you have millions of users in a group you don’t want that all being returned in one single long response as it would tie up too much of the limited domain controller resources.

For example:

Query for all members of the group called biggroup. That group has 5000 members in it as you can see…

[Sun 04/18/2021 17:17:15.12]
E:\DEV\cpp\vs\AdFind\Release>adfind -b CN=biggroup,OU=TestBigGroup,DC=lockout,DC=test,DC=loc -base -csv -cv member
"dn","member"
"CN=biggroup,OU=TestBigGroup,DC=lockout,DC=test,DC=loc","5000"

The query for the actual direct members looks something like:

adfind -b CN=biggroup,OU=TestBigGroup,DC=lockout,DC=test,DC=loc member

On the wire, it looks like:

The response from Active Directory is:

The presence of the “attribute” member;range=0-1499, really the originally requested attribute with the ranging option tacked on, alerts the LDAP client that there were more values available than AD was willing to send in one pass.

The LDAP client processes those 1500 values and then sends another LDAP query to retrieve the next 1500. That next query is similar to the first, it looks like this next query… Don’t worry AdFind does this under the covers for you but you can do it manually too if you like.

adfind -b CN=biggroup,OU=TestBigGroup,DC=lockout,DC=test,DC=loc member;range=1500-*

On the wire that looks like:

and the response looks like:

So then the client processes those 1500 values and asks for member;range=3000-*…

This game of ping pong continues until the domain controller gets to the end of the values to be returned at which point the response from the domain controller changes slightly:

Note that the response with the final result shows member;range=4500-*. The last value is “*” instead of an actual number. That alerts the LDAP client all of the data has been returned for that attribute and it doesn’t need to ask for any more values.

It is all quite simple.

Now to show how the msds-memberTransitive and msds-memberOfTransitive attributes completely fail to do any of this properly. Someone, apparently, in their infinite wisdom decided that you will only ever need to return 4500 values for these attributes and didn’t even do it the normal way. So they are wrong on both counts.

Given the following query:

adfind -b CN=biggroup,OU=TestBigGroup,DC=lockout,DC=test,DC=loc msds-membertransitive -base

On the wire it looks like:

and the response looks like:

As you can see, someone involved in the design or coding decided to return 4500 values instead of 1500 – completely against the policy defined in the LDAP query policy which could break some applications that assume that that policy is being followed. Well ok, that is fine, anyone that hard coded to that is probably asking for troubles anyway and any flexibly written ranging code should work just fine and handle that as it popped up. Instead of asking for msds-memberTransitive;range=1500-* for the next set, the code will ask for msds-memberTransitive;range=4500-*… Which one could reasonably expect to return another 4500 results.

One who assumes that would be completely wrong. Because… Well, some unknown person at MSFT.

Instead you ask for the next set like:

and get a very unexpected result of:

So literally no attributes returned and still a LDAP_SUCCESS result code.

Interestingly, if you try *any* range other than one that starts with 0 (even ;range=1-10) the operation fails with an LDAP_SUCCESS result code.

Note, I did not stumble upon this myself. Sadly. Instead, I was talking to someone in a AD software company that writes some top notch AD software and has the largest number of the best AD folks (and all a bunch of my long time friends) in any single company out there anymore. We were discussing how best to gather certain info and they mentioned this attribute and how AdFind crashes when it tries to enumerate the attribute if there are more than 4500 members. My initial response was “The hell it does!!!”. I knew my ranging code was generic enough and was solid. I wrote that routine 15 years ago and it has worked for millions, if not hundreds of millions or even billions of of queries and never had any issues reported around it.

So I tested it and they were right, <(*&#@%()^#*>, AdFind crashed when trying to enumerate msds-memberTransitive and msds-memberOfTransitive when they contained more than 4500 values. That started me looking at the adfind debug output which looked like:

DEBUG: In ranging… [1][msds-memberTransitive;range=0-4499]
DEBUG: Getting next range – [msds-membertransitive;Range=4500-*]

before the big crash dialog popped up…

First I thought “4500… WTAH? That is weird”. Later when I had a chance to dig more I pulled out WireShark and started watching the sequence and again my response was “WTAH??”.

I was able to find my bug quickly enough. I had never seen a ranging request come back with a STATUS_SUCCESS with no data. And the ranging subroutine was being stupid and just assumed there would be data so never validated that data came back and of course that meant I was trying to read data from memory that wasn’t there and so that landed me a nice unfriendly crash boom bang your app has crashed.

It was an easy fix. Make sure data was returned before trying to use it. Really it should have been there before but I will cut myself some little bit of slack because I barely knew what I was doing with the AD coding 15 years ago.  

BUT, there is still a huge problem here… Incomplete data for this could be disastrous. Part of me actually liked the crash versus just returning data that I knew was incomplete. But another part of me really really hated that crash (and really really really hated not knowing it was crashing), so now what I do is return the data that is returned, *but* at the beginning of the result set it will now say:

ERROR:
ERROR: Failure in Range Retrieval for msds-membertransitive;range=4500-*
ERROR: LDAP_SUCCESS but range attribute not returned.
ERROR: Output for attribute msds-membertransitive is incomplete.
ERROR:

Because I consider this an absolutely critical issue and very possibly reporting dangerous misinformation for security purposes I have that same error message in the output whether or not it is standard output or CSV so it *will* screw up the CSV output. I *want* it to screw up the output.

Looking at the documentation for msds-memberTransitive and msds-memberOfTransitive there is no mention of this limitation or the odd value range handling. It is clearly the intended behaviour because it is so unlike standard ranging that someone had to make this decision purposefully. My expectation, as mentioned above, is that someone figured that 4500 was a larger number of members or memberships than any object would have. It is clearly wrong, I mean absolutely 100% wrong, especially on the msds-memberTransitive side but it is also very possible to be wrong on the msds-memberOfTransitive side. Not all groups are Security Enabled which maybe would have a reason for such a low ceiling. For example, I have seen companies that use non-Security Enabled groups for LDAP based applications and users were in thousands of non-Security enabled groups. I have also seen companies that were very heavy on Exchange Distribution Lists that do not have to be Security Enabled.

Further googling shows I am not the first to find this, though there seems to be some confusion around it and what might be going on so I thought I would spend the time to publish this blog post for it. Worse there are posts saying use this mechanism with no caveat of where it might go sideways on you. I have also found some hits where it has been specified that msds-memberOfTransitive isn’t a problem because authentication will break before you get to that many group memberships for a user. As mentioned above, that is absolutely incorrect. That statement assumes that every group is Security Enabled and that is not a valid assumption.

I thought about bugging this with MSFT but I am conflicted on whether or not it is the worth the time. From what I have seen the last few years, MSFT is letting Active Directory die on the vine. Unless the change drives Azure adoption the likelihood of a change seems to be about 0% so I am not sure it is worth my time to raise it to only be told again that they aren’t spending time working on on-premises AD/ADLDS.

    joe