joeware - never stop exploring... :)

Many moons ago I wrote a post about how a TechNet article on the default tombstone lifetime was wrong. That TechNet article eventually ended up getting corrected at some point though it doesn’t seem to be properly linked anymore to the GUID URL that I had for it – so much for theory of never losing TechNet articles again because they were "unique" GUID links… Regardless, it has been brought to my attention that once again a TechNet article has screwed this simple topic up when updating it for PowerShell[1].

http://technet.microsoft.com/en-us/library/dd392260(WS.10).aspx

Both here

By default, tombstoneLifetime is set to null. When tombstoneLifetime is set to null, the tombstone lifetime defaults to 180 days (hard-coded in the system)

and

In Windows Server 2003 with Service Pack 1 (SP1), Windows Server 2003 with Service Pack 2 (SP2), Windows Server 2008, or Windows Server 2008 R2 operating systems, when tombstoneLifetime is set to null its value defaults to 180 days.

So for now and all eternity, please link to, copy and paste, recite, or whatever you do around this topic from this blog post.

The tombstoneLifetime attribute is very simple. Don’t worry about the operating system as it exists now, don’t worry about the operating system as it existed when the Active Directory was initially built, it is all moot. The one and only way to know what the tombstoneLifetime value for a forest is is to look at it.

1. If the value of tombstoneLifetime is NULL or <NOT SET> the value *IS* 60 days.
2. If the value of tombstoneLifetime is > 0. The value *IS* the integer specified measured in days.

You can easily ascertain the value by using the following command: [2]

adfind -config -f objectclass=ntdsservice tombstonelifetime

Which will look like

C:\>adfind -config -f objectclass=ntdsservice tombstonelifetime

AdFind V01.45.00cpp Joe Richards (joe@joeware.net) March 2011

Using server: K8R2Dom-DC1.k8r2dom.loc:389
Directory: Windows Server 2008 R2
Base DN: CN=Configuration,DC=k8r2dom,DC=loc

dn:CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=k8r2dom,DC=loc
>tombstoneLifetime: 180

1 Objects returned

Or if you really want to make this simple, use the following:

adfind -config -f objectclass=ntdsservice tombstonelifetime -oao 60

which will display the integer value no matter what through some AdFind "magic". That magic being that the -oao switch when specified with a value will populate that value in any empty attributes. So if the value isn’t populated in AD, it will still look like it is when the output is displayed.

For example, same command with one non-existent attribute to help illustrate the point:

C:\>adfind -config -f objectclass=ntdsservice tombstonelifetime hardcodedWindowsADTombstoneLifetimeValue -oao 60

AdFind V01.45.00cpp Joe Richards (joe@joeware.net) March 2011

Using server: K8R2Dom-DC1.k8r2dom.loc:389
Directory: Windows Server 2008 R2
Base DN: CN=Configuration,DC=k8r2dom,DC=loc

dn:CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=k8r2dom,DC=loc
>tombstoneLifetime: 180
>hardcodedWindowsADTombstoneLifetimeValue: 60

1 Objects returned

You can do the same with CSV

adfind -config -f objectclass=ntdsservice tombstonelifetime -nodn -csv 60

which could look something like:

C:\>adfind -config -f objectclass=ntdsservice tombstonelifetime hardcodedWindowsADTombstoneLifetimeValue -csv 60 -nodn
"tombstonelifetime","hardcodedWindowsADTombstoneLifetimeValue"
"180","60"

Simple?

In 2017 when this gets screwed up again in TechNet when they are describing how to figure it out with whatever new fancy tool they have now switched to the above information will STILL be correct and that AdFind command will still work just fine.

 

     joe

 

[1] Further proof that PowerShell isn’t doing anything new, it is simply redoing what was done before with more verbose commands… 😉

[2] Or… even shorter  "adfind -b -pr -f objectclass=ntdsservice tombstonelifetime". That can be an exercise for the class to figure out what is happening there. 

Is anyone out there using 2FA for AD as a best practice or their standard? I am not asking about things like VPN, I am simply meaning basic AD auth once you are already on the internal network.

    joe

If you haven’t noticed, Radio Shack is making an honest attempt at wrangling in all the geeks/nerds that used to go hang out there back in the 80’s. Check out this DIY website they have set up, lots of cool Arduino projects.

http://www.radioshackdiy.com/

If there were only more time in the days for me… I am interested in playing with far too many things. 🙂

 

    joe

I recently had a conversation with an Active Directory engineer who was working on some token bloat issues. As AD has pushed past the decade mark I seem to be seeing/hearing more and more of this bloat problem as many companies continue consolidating groups into and creating new groups in AD without cleanup or normalization.

To be more clear, say a company took whole hog the groups they had before, likely in a couple, but possibly up to hundreds of NT4 domains and collapsed it all into a single forest or they took all of the machine local groups on member servers and sucked them up into AD for “easier management and reporting”. On top of that, all of it has remained with no cleanup and companies moved forward adding more permission and DL groups and merging/consolidating more and more domains (or companies), etc. And then on top of THAT, you get companies who used sIDHistory but never went back and cleaned up what the sIDHistory was initially needed for. What should take place is group/permission normalization, ACL cleanup, etc.

Err ok, I just sidetracked myself with a soapbox speech… back to the AD Engineer patiently waiting for me to shut up and listen to what he wants to accomplish…

So part of the problem this engineer is experiencing is that the Active Directory he is concerned about had been through a double hop AD Consolidation after multiple mergers/acquisitions. I.E. OldOldDomain was migrated/collapsed into OldDomain and later OldDomain was migrated/collapsed into ShinyNewDomain[1]. The migrations involved pulling the objects over whole hog and populating sIDHistory in the new accounts. So what you ended up with was (SIDs shortened for readability):

OldOldDomain – Domain SID S-1-5-12345
OldDomain – Domain SID S-1-5-54321
ShinyNewDomain – Domain SID S-1-5-24680

So a migrated object in ShinyNewDomain has objectSid of S-1-5-24680-2500, and has sIDHistory values of S-1-5-54321-78 and S-1-5-12345-19.

Now let’s say some cleanup had actually occurred and any ACLs with references to SIDs from OldOldDomain were believed to be all gone but there definitely were resources using OldDomain SIDs still. Simple right? Target and remove the OldOldDomain SIDs and leave the OldDomain SIDs in place.

Well unfortunately, the engineer had spoken to another engineer about it and was told it wasn’t possible to perform granular sIDHistory cleanup. He was further told that the process he would have to use to perform this cleanup was to go back to OldDomain, clear the sIDHistory attribute of all objects there, then clear all of the sIDHistory attribute of all matching objects in NewDomain, then re-migrate the SIDs from OldDomain to NewDomain…

I was a bit shocked by this and responded that you could indeed remove individual SIDs from the sIDHistory. And while I don’t know how long it would have taken to perform the other process (assuming OldDomain still existed!), using targeted cleaning of individual SIDs could be done relatively quickly. I would shoot from the hip and say that original process would take a couple of days to get everything all lined up and completed where a granular cleanup could be completed in minutes or hours at most.

I then asked around to some other friends that I know to be good admins, people that aren’t dev guys but generally a big step up from “My name is Peggy”, and a majority of them thought the same thing, it was an all or nothing thing when clearing sIDHistory with one person, who shall remain nameless, actually responding, “Why would you ever want to remove sIDHistory values????” Moving on…

One of the guys sent me a couple of links to MSFT web pages that had VBScript and PowerShell examples and they both removed all values with no indication that you could, if you so desired, remove only a subset of the values. Even a blog entry I had written back in 2006 (http://blog.joeware.net/2006/09/16/621/) only mentioned clearing all values from the attributes on all objects. I did have a later blog post in 2009 (http://blog.joeware.net/2009/06/10/1655/) that explained how to clean up SIDs for a specific domain, looking it over I have to admit it is a bit terse. Plus if you are googling for “sIDHistory cleanup”, the 2009 post doesn’t come up (at least not in the first three pages[2]).

Now I don’t think there is any great conspiracy here, I just think that MSFT didn’t really visualize stacking migrations and thereby stacking up sIDHistory values and I don’t think Google is out to make life hard for AD Admins. That being said, I have now personally seen cases where a single object has 3,4 or even up to 6 sIDHistory values. This is likely unusual for most companies but it is occurring out there and I would hate to think admins in those positions thought they were stuck with keeping all of those entries because they need just one or two of them and can’t find guidance on how to do it.

Granted, if you know enough about the vbscript or the powershell commands or even the AdFind/AdMod commands I used in the 2006 blog you could divine that removal of a single SID from sIDHistory is possible because they all clear the attribute by removing individual values. But low and behold, looking around, I didn’t find it stated anywhere except in my 2009 post and again, that post isn’t popping as early as it could for people. Additionally some people just aren’t all that willing to experiment or don’t have a place to experiment or perhaps even the time to experiment.

So based on the idea that if several smart people I have spoken to don’t know/understand something or, as in this case, have an incorrect understanding of it, I assume that the chances are good that a good number of others don’t know either. This led me to sit down and write this new blog post which will hopefully score higher when you search for “sIDHistory cleanup”. So, if you are still with me, let’s work through the process of “sIDHistory cleanup” in a non-terse manner …

Q: Can you remove a single SID from the sIDHistory attribute of an object? I.E. Can I perform targeted sIDHistory cleanup?

A: ABSOLUTELY! You can indeed remove a single SID or even some subset of SIDs from sIDHistory on an object and in fact complete a targeted sIDHistory cleanup.

Here is a real life example of targeted sidHistory cleanup with AdMod:

C:\temp>adfind -default -f name=$joe sidhistory

AdFind V01.45.00cpp Joe Richards (joe@joeware.net) March 2011

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

dn:CN=$joe,OU=Users,OU=My,DC=test,DC=loc
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6569
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6570
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6571
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6572

1 Objects returned

C:\temp>admod -b CN=$joe,OU=Users,OU=My,DC=test,DC=loc SID##sidhistory:-:S-1-5-21-3641047700-3957557241-2644433309-6571

AdMod V01.13.00cpp Joe Richards (joe@joeware.net) April 2010

DN Count: 1
Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003

Modifying specified objects…
   DN: CN=$joe,OU=Users,OU=My,DC=test,DC=loc…

The command completed successfully

C:\temp>adfind -default -f name=$joe sidhistory

AdFind V01.45.00cpp Joe Richards (joe@joeware.net) March 2011

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

dn:CN=$joe,OU=Users,OU=My,DC=test,DC=loc
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6569
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6570
>sIDHistory: S-1-5-21-3641047700-3957557241-2644433309-6572

1 Objects returned

The key to a targeted sIDHistory cleanup is to specify the actual SID you want removed in the LDAP command. The tricky part of that though is that the SID needs to be specified as a binary SID structure, not the nice friendly string you see above. AdMod is nice in that it allows you to specify the SID## prefix on an attribute to alert AdMod to convert a string SID to a SID structure on your behalf. PowerShell should have something similar (I really don’t know for sure) and vbscript… well you will have to do some digging around on that one. However, to my knowledge, binary structures were never VBScript’s strong point.

Now I did get a follow-up question, “So you can use a wildcard for the SIDs?” I.E. Specify removing any SIDs in sIDHistory that start with the Domain SID of the Domain that is gone? That certainly would be cool but nope, sorry, no such luck. The wildcarding of the SIDs is not possible. If you wanted to clear all of the SIDs from sIDHistory for a specific domain, you would want to dump all sIDHistory values and then filter the strings for the values. This generally just takes a basic string parsing script to perform that filtering for you. Below I walk through some basic steps involved and show how you can quite easily do this with AdFind and AdMod. It is likely that after I post this blog, some PowerShell person somewhere (http://bsonposh.com/) will post a blog on how to do it with PowerShell.

Anyway… now for the interesting stuff…

First… Query the GC with a base of “” for any objects with any value in sIDHistory. Then once that data comes back filter through it for the specific sIDHistory value. Finally mute any records that don’t have any attributes being output (i.e. any objects that don’t have the specified SID listed in sIDHistory).

adfind -gcb -f sidhistory=* sidhistory -mvfilter sidhistory=S-1-5-21-1173133699-2878410131-473346761 -recmute

This generates output like:

AdFind V01.45.00cpp Joe Richards (joe@joeware.net) March 2011

Using server: dc1.sidtest.loc:3268
Directory: Windows Server 2008 R2

dn:CN=group1,OU=TestGroups,DC=sidtest,DC=loc
>sIDHistory: S-1-5-21-1173133699-2878410131-473346761-7912

dn:CN=group2,OU=TestGroups,DC=sidtest,DC=loc
>sIDHistory: S-1-5-21-1173133699-2878410131-473346761-14644

dn:CN=group3,OU=TestGroups,DC=sidtest,DC=loc
>sIDHistory: S-1-5-21-1173133699-2878410131-473346761-15651

dn:CN=group4,OU=TestGroups,DC=sidtest,DC=loc
>sIDHistory: S-1-5-21-1173133699-2878410131-473346761-15662

dn:CN=group5,OU=TestGroups,DC=sidtest,DC=loc
>sIDHistory: S-1-5-21-1173133699-2878410131-473346761-16443

Now visualize hundreds or thousands or tens of thousands of those… You absolutely can parse that by hand if you want but I am lazy, even for five objects so I would go a step further and have AdFind do more of the work by running the following command which is the same as the last but sticks the info into a delimited format using the colon character as the delimiter, oh yeah, don’t output a header…

adfind -gcb -f sidhistory=* sidhistory -mvfilter sidhistory=S-1-5-21-1173133699-2878410131-473346761 -recmute -csv -csvdelim : -nocsvheader

which produces this output

"CN=group1,OU=TestGroups,DC=sidtest,DC=loc":"S-1-5-21-1173133699-2878410131-473346761-7912"
"CN=group2,OU=TestGroups,DC=sidtest,DC=loc":"S-1-5-21-1173133699-2878410131-473346761-14644"
"CN=group3,OU=TestGroups,DC=sidtest,DC=loc":"S-1-5-21-1173133699-2878410131-473346761-15651"
"CN=group4,OU=TestGroups,DC=sidtest,DC=loc":"S-1-5-21-1173133699-2878410131-473346761-15662"
"CN=group5,OU=TestGroups,DC=sidtest,DC=loc":"S-1-5-21-1173133699-2878410131-473346761-16443"

Next you could push that command through a FOR /F loop and generate and write AdMod commands to a text file for your perusal.

C:\qnd\sidtest>for /f "tokens=1,2 delims=:" %i in (‘adfind -gcb -f "sidhistory=*" sidhistory -mvfilter "sidhistory=S-1-5-21-1173133699-2878410131-473346761" -recmute -csv -csvdelim : -nocsvheader’) do @echo admod -b %i SID##sidhistory:-:%j >> SIDHISTORY_CLEANUP.TXT

C:\qnd\sidtest>type SIDHISTORY_CLEANUP.TXT
admod -b "CN=group1,OU=TestGroups,DC=sidtest,DC=loc" SID##sidhistory:-:"S-1-5-21-1173133699-2878410131-473346761-7912"
admod -b "CN=group2,OU=TestGroups,DC=sidtest,DC=loc" SID##sidhistory:-:"S-1-5-21-1173133699-2878410131-473346761-14644"
admod -b "CN=group3,OU=TestGroups,DC=sidtest,DC=loc" SID##sidhistory:-:"S-1-5-21-1173133699-2878410131-473346761-15651"
admod -b "CN=group4,OU=TestGroups,DC=sidtest,DC=loc" SID##sidhistory:-:"S-1-5-21-1173133699-2878410131-473346761-15662"
admod -b "CN=group5,OU=TestGroups,DC=sidtest,DC=loc" SID##sidhistory:-:"S-1-5-21-1173133699-2878410131-473346761-16443"

You can then validate the commands are what you wanted and expected and then batch them to run in phases or through a pilot or run them all at once, just copy the commands to a .CMD file or rename the .TXT file and you are good to go. However… If you just want the commands to directly run you could remove the @echo after the DO and it will execute the commands for you. But in that case… it is simpler to output the information with -ADCSV format and push straight into AdMod like the example in http://blog.joeware.net/2006/09/16/621/

The search command for this case would be adfind -gcb -f "sidhistory=*" sidhistory -mvfilter "sidhistory=S-1-5-21-1173133699-2878410131-473346761" -recmute -adcsv

Which would generate the following output:

~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~~~~ADCSV~~~
"dn","sidhistory"
"CN=group1,OU=TestGroups,DC=sidtest,DC=loc","S-1-5-21-1173133699-2878410131-473346761-7912"
"CN=group2,OU=TestGroups,DC=sidtest,DC=loc","S-1-5-21-1173133699-2878410131-473346761-14644"
"CN=group3,OU=TestGroups,DC=sidtest,DC=loc","S-1-5-21-1173133699-2878410131-473346761-15651"
"CN=group4,OU=TestGroups,DC=sidtest,DC=loc","S-1-5-21-1173133699-2878410131-473346761-15662"
"CN=group5,OU=TestGroups,DC=sidtest,DC=loc","S-1-5-21-1173133699-2878410131-473346761-16443"

The whole command to cleanup would then be

adfind -gcb -f "sidhistory=*" sidhistory -mvfilter "sidhistory=S-1-5-21-1173133699-2878410131-473346761" -recmute -adcsv | admod sidhistory:-:{{sidhistory}} -unsafe

Not bad for 30 seconds worth of “scripting” eh?

joe

[1] Shiny here refers to the Firefly definition and if you don’t know what Firefly is… You have gotten your nerd/geek card revoked. 😉

[2] I have a three page rule with search engines I tend to follow, if it isn’t on the first three pages, I am not likely to see it because I stop looking.