joeware - never stop exploring...

Why is Facebook doing this…

On the newsfeed I see…

on my wall I see…

So I post a test comment and then on my wall I see

So then I look at the newsfeed and I see…

Anyone?

If you spend your time living your life, you don’t have time to waste talking about others’ lives.

   -me

You may have read my other post on using the SID/GUID alternate formats for DNs for queries and my post last year on using alternate DN formats for binding and searching as well… Here is something else I recently learned that I thought was interesting.

There are actually more alternate DN formats…

All of these formats can be used for the bind string…

http://msdn.microsoft.com/en-us/library/ms676245(VS.85).aspx

Dmitri posted them to AD Org as such

ACTIVE DIRECTORY FORMATS

DS_FQDN_1779_NAME,
DS_USER_PRINCIPAL_NAME,
DS_NT4_ACCOUNT_NAME,
DS_CANONICAL_NAME,
DS_UNIQUE_ID_NAME,
DS_DISPLAY_NAME,
DS_SERVICE_PRINCIPAL_NAME,
DS_SID_OR_SID_HISTORY_NAME,
DS_CANONICAL_NAME_EX

and

ADAM FORMATS

DS_FQDN_1779_NAME,
DS_CANONICAL_NAME,
DS_UNIQUE_ID_NAME,
DS_DISPLAY_NAME,
DS_SERVICE_PRINCIPAL_NAME,
DS_SID_OR_SID_HISTORY_NAME,
DS_CANONICAL_NAME_EX,
DS_USER_PRINCIPAL_NAME

So yes… You can even use displayname as the bind DN string but man I would recommend being careful there because there is NOTHING implementing uniqueness and obviously if AD can’t uniquely identify the bind DN it won’t let you authenticate with it.

    joe

About a year ago I wrote a popular blog post about DN formats available in Active Directory. The article is here –> http://blog.joeware.net/2008/05/03/1226/. Great article if I can believe the feedback because it helps people set up their environment and apps in a more generic way that avoids some of the pitfalls of hardcoding DNs which is an evil UNIXy kind of thing to do… <eg>

In that article I talk about how you can use the various DN formats available in Active Directory for the bind string (obviously for Binding) and for the search base (just as obviously for searching). What I had no idea about that I recently learned after a discussion with a user who emailed me and another discussion with a friend of mine at MSFT is that the special SID/GUID methods of specifying a DN can also be used in the LDAP query filters (need I say also obviously for searching)…

You may be thinking… ah yeah… so? Big deal? How about posting some more pictures of your house and what you have gotten done in the last year or so ya git???

Well it can be a big deal for some folks… Think of the following scenario… You have some application or device that needs to locate people based on some group membership or possibly the manager or some other DN based field (specifically attribute syntaxes 2.5.5.1, 2.5.5.7, and 2.5.5.14) but the first part of the query is pretty much hardcoded … For example… They tell you the query is

(&(objectcategory=person)(objectclass=user)(memberof=%GROUPDN%))[1]

so you have to specify a group DN like cn=somegroup,cn=users,dc=test,dc=local and then think, crap, I can never change that group DN now without fixing this hardcoded piece… Or worse, you are specifying a DN for a user… Or even worse, later on down the road your predecessor changes it and all hell breaks loose and they haven’t the foggiest clue how to fix it and tell everyone the previous admin was an idiot… or maybe the idio… err person you replaced did it and you change it and bam you break! That sucks… Seriously.

So how can you get around that… By specifying the group (or person or whatever) by its GUID… You could also do it by its SID, but if you specify by GUID, you can even move the group/user to another domain without keeping sIDHistory and you will be good…

So instead of

(memberof=cn=somegroup,cn=users,dc=test,dc=local)

or would instead specify

(memberof=<GUID=ADCC0217-5535-4E17-85CB-67DAD4635C3F>)

Or maybe your app is looking for everyone under a certain manager…

(manager=<GUID=ADCC0217-5535-4E17-85CB-67DAD4635C3F>)

So for a full blown example with all of the DN coolness in Active Directory used…

So you have to bind with some user X to search some ou Y for all users with manager Z and the application has been kind enough to make it so you only enter the bind DN, the bind password, the search Base DN, and the manager’s DN and it does the rest… This would normally look like

• Bind DN: cn=sillyapp,ou=serviceids,dc=test,dc=local
• Bind Password: ThisAppDoesNotRock011235813213455
• Search Base DN: ou=mi,ou=us,ou=grunts,dc=test,dc=local
• Manager DN: ou=fatexec,ou=easystreet,dc=test,dc=local

So then you have multiple ways this can break and require reconfiguration, some of which include…

• The sillyapp ID gets renamed
• The sillyapp ID gets moved to ou=legacycrap,ou=appids,dc=test,dc=local
• The search base gets changed to ou=mi,ou=us,ou=cherishedpartners,dc=test,dc=local[2]
• The fatexec ID parent DN gets changed to ou=wereallydowork,dc=test,dc=local
• The domain gets renamed
• And others…

So how could you configure this…

• Bind DN: <GUID=AB9C4D61-8F4E-4257-B46E-733CCA31EA4C>
• Bind Password: ThisAppDoesNotRock011235813213455
• Search Base DN: <GUID=79EBB6AE-A181-4D44-9E23-8A941E4117C8>
• Manager DN: <GUID=4EC2E83C-DEFD-4C57-B8D0-4725B3790088>

Now not a single one of the events above or others of a similar vein can break that configuration.

The things that could break you… someone deleting one or more of the IDs or the OU and then recreating with the same name.

     joe

[1] Ok more realistically and what I have seen more than once is if the developers don’t let you modify the entire query it is unfortunately likely that the query looks like (&(objectclass=user)(memberof=%GROUPDN%)) though now with Windows Server 2008 objectclass is indexed so it doesn’t matter.

[2] This would be after the economy gets better and companies realize they may not have been as nice as they could have been with the people who produce, ship, or sell their goods.

I received an email a couple of months ago from someone looking to remove SIDs from a specific domain from the sIDHistory attribute of all of their users. Here is the response I sent

So the ability to remove SID’s from sIDHistory is very simple and basic, you basically have to supply the specific SIDs you want removed. The -sc csh shortcut in AdMod relies on the SIDs to be cleared being passed to it from AdFind. So all you need to do is to filter the output from AdFind to the specific SIDs you want cleared versus all SIDs. You do that with the -mvfilter switch… You would specify the domain portion of the SID of the domain that you want to clean the SIDs out for… So say you have a user with a SID of S-1-5-21-1757981266-299502267-1801674531-19235, the domain portion of that SID is S-1-5-21-1757981266-299502267-1801674531. So to filter and just output objects with that SID you will want to add -mvfilter sidhistory=S-1-5-21-1757981266-299502267-1801674531 combined with -recmute to filter out objects that had a sIDHistory value but not from that domain.

You send that info across to AdMod with -adcsv and specify -sc csh and you should be good. I would recommend just running the AdFind command initially to output the objects and the SIDs so you can validate the info prior to clearing it.

Note that clearing the values from sIDHistory is basically one way… Once cleared, the only way to get it back is auth restore of the objects. So be sure you want to clear them. There is no way to arbitrarily stick values in that attribute.

So the whole adfind command for users would look something like

adfind -b search_base -f "&(objectcategory=person)(sidhistory=*)" sidhistory -adcsv -mvfilter sidhistory=domainSID

then combined with AdMod

adfind -b search_base -f "&(objectcategory=person)(sidhistory=*)" sidhistory -adcsv -mvfilter sidhistory=domainSID | admod -sc csh -unsafe

   joe

At least it appears so…

On a conference call today my manager John Tanner was explaining who I was and said I had “Ridiculously Deep AD knowledge”…

I couldn’t help it, I had to LOL when I heard it as I have never heard that said before.

You too can have a joeware t-shirt just like that by going to http://www.cafepress.com/joewarenet

http://support.microsoft.com/kb/970789/en-us

A folder that is created under the root of the system drive is missing entries in its security descriptor, which may cause some application failures on the English version of Windows 7 Release Candidate 32-bit Ultimate

In the English version of Windows 7 Release Candidate (build 7100) 32-bit Ultimate, the folder that is created as the root folder of the system drive (%SystemDrive%) is missing entries in its security descriptor. One effect of this problem is that standard users such as non-administrators cannot perform all operations to subfolders that are created directly under the root. Therefore, applications that reference folders under the root may not install successfully or may not uninstall successfully. Additionally, operations or applications that reference these folders may fail.
For example, if a folder is created under the root of the system drive from an elevated command prompt, this folder will not correctly inherit permissions from the root of the drive. Therefore, some specific operations, such as deleting the folder, will fail when they are performed from a non-elevated command prompt. Additionally, the following error message appears when the operation fails:

Access is denied.

Furthermore, the missing security descriptor entries protect non-admin file operations directly under the root.

http://msdn.microsoft.com/en-us/magazine/dd695919.aspx

..

Man, That C Is Sharp

For the majority of my fifteen year career at Microsoft, I’ve been a systems and drivers developer. My language of choice and necessity has been a fairly bare bones C++. I rarely get to use runtimes like MFC and the Microsoft .NET Framework. Until recently, I couldn’t have even spelled STL much less made use of it.

That’s my day job. But in my off time I love playing around with C#. Mostly I write little apps for my Windows Mobile devices and occasionally for my PC. C++ doesn’t treat me as badly as it does many native developers, but I still I get a little giddy when I write in C#. You can get a lot done with very few lines of code. I swear, C# is so much fun they should make it a controlled substance.

In the fun category, XNA Game Studio is like C# on steroids. The team over there has done an amazing job of making game development easy. The framework is straightforward, the hard stuff is largely handled for you, and they’ve released a ton of samples aimed at teaching you the various aspects of game development.

Start at creators.xna.com. From there you can download the free XNA Game Studio 3.0. If you already use one of the various incarnations of Visual Studio 2008, XNA Game Studio will integrate with it. If you don’t have Visual Studio 2008, don’t fret. XNA Game Studio also works with the free Visual C# Express Edition. (In other words, although I mention Visual Studio, you can substitute Visual C# Express Edition if that’s what you’re using.)

The creators.xna.com Web site is also full of great information to get you going. Click the Education link at the top of the page to find beginner’s guides, samples, and how-to’s. The “Beginner’s Guide to 2D Games” is especially good, as is the documentation that gets installed with XNA Game Studio. In some cases, the installation documentation has information that’s not on the Web. In Visual studio, you can get to that documentation by selecting Help | Contents and setting the filter to XNA Game Studio 3.0.

XNA Game Studio lets you write one code base and deploy it to Xbox, PC, and Zune. Everything I do here will work on all three platforms. The free downloads are all you need to develop for PC or Zune, but Xbox development requires a Premium membership that costs a yearly fee.

..

Nuff said right???

Ok may be a little more… Windows 7 blows Windows Vista out of the water for performance and stability. Period.

Oh here is something funny if you don’t much like SQL Server or if you were trying to download Windows 7 this morning….

Database glitch causes Windows 7 download server meltdown

…

This morning at 6AM PDT, when Windows 7 Release Candidate downloads were officially made available for MSDN and TechNet subscribers, it looked like a sequel to that botched release. After 20 minutes or so of smooth downloads, both sites began bogging down, and the situation deteriorated rapidly as the minutes passed. For several hours after the official launch, most subscribers who tried to log on found themselves unable to reach the download pages.

This time, though, the problem wasn’t capacity. Instead, a source tells me, the glitch was caused by a SQL Server database that reached excessive fragmentation levels because of the tremendous surge of queries. How massive was the demand surge? The number of requests to the MSDN and TechNet databases in less than an hour was equal to more than a week’s traffic under normal circumstances.

…

Very humorous.

I got this from Tony and I wanted to post it, looking at the site and trying to send email to the list it appears their ISP already chopped the lines…

 

Hi all

The ISP that we have been using to host ActiveDir.org for the past 8 years is shutting down and we are preparing to transfer the site to another hosting service.  The mailing list subscription and site user account information will remain the same and no action is required on your part.

A potential fly-in-the-ointment is that we understood the hosting termination notice period from our ISP was 30 days from Monday 27^th April.  Now we understand that the ISP has requested the termination from their upstream provider AT&T and that cut-off is expected “within 30 days”.  Needless to say, this has put us in some difficulty and we are doing our best to transfer the site to our new provider asap.

There may be some disruption to service during this time.  We ask for your patience and apologise for any loss of service.

Cheers

Tony and Matty

A man’s ethical behavior should be based effectually on sympathy, education, and social ties; no religious basis is necessary. Man would indeed be in a poor way if he had to be restrained by fear of punishment and hope of reward after death.

  -Albert Einstein

A human being is a part of a whole, called by us ‘universe’, a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest… a kind of optical delusion of his consciousness. This delusion is a kind of prison for us, restricting us to our personal desires and to affection for a few persons nearest to us. Our task must be to free ourselves from this prison by widening our circle of compassion to embrace all living creatures and the whole of nature in its beauty.

   - Albert Einstein

Imagination is more important than knowledge.

   - Albert Einstein

Weakness of attitude becomes weakness of character.

    - Albert Einstein

Another discussion recently popped up on AD Org about indexing the objectClass attribute. Don Hatcherl (previously mentioned on this blog multiple times - short and sweet, when he talks about AD, I and everyone else with any sense listens…) said the following about indexing objectClass…

…An early version of ESE (the one shipped in Exchange 4.0, and that never shipped with AD) had perf problems indexing highly non-unique values.  We handled this during development by inventing the ObjectCategory concept and creating a climate of fear to prevent people from indexing ObjectClass.  Both of these were, in retrospect, stupid ideas.  By the time Windows 2000 shipped ESE’s perf problem (ok, ok, this particular ESE perf problem) had been fixed, but by then there was too much organizational inertia around ObjectCategory to get rid of it.  It took until Windows 2008 to do the correct thing.

You should do two things:

1) index ObjectClass

2) accept my apology for the confusion

DonH

Thanks Don!

I recently had a blow up of a server at work and unfortunately it was a server I adopted rather recently and so hadn’t been able to put the time into the discovery work on it that needed to be done. Well another group that the server was apparently shared with decided to do some updates and unfortunately blew the box out of the water. At this time I find out that the backup process in this customer was data only, not bare metal. Oi.

online casino

So one of the most painful aspects of this whole thing is around scheduled tasks. The JOB files are in intact, however, the creds (or even the SID of the runas ID) aren’t actually stored in the file and in fact there is NO WAY to ascertain what creds used to be used. This is amazingly annoying and painful as I sit here trying to work out creative ways to figure this out. I can think of multiple ways to get his info when the scheduler service is up and running ok but if all you have are JOB files there is nothing, or at least nothing I can find.

In looking this over I found that there isn’t even a way to properly export tasks. This is just plain silly. Does MSFT think that the one machine you deploy a dozen or more tasks to will always be up and running and survive forever?

Ugh.

I have spoken about this tool before. The old version I have is hands down the best file copy app out there. I expect this newer version that is now publicly available is just as good… Thanks to Scott C for letting me know this was now out there.

http://technet.microsoft.com/en-us/magazine/2009.04.utilityspotlight.aspx

   joe

PS C:\Documents and Settings\$joe.TEST> cd\
The term ‘cd\’ is not recognized as a cmdlet, function, operable program, or script file. Verify the term and try again
.
At line:1 char:3
+ cd\ <<<<

That will be the last time I say that… I promise. Yes I hate the name TEC… There I said it… I HATE THE NAME. I put this name change right there with ADLDS instead of ADAM… But I will stop.

So this was the first Experts Conference put on by Quest since they bought NetPro. Who here has dealt with Quest in the past, specifically the Quest Sales Team members? Raise your hand. One fear that I had was along the lines of Quest turning what has always been an awesome technical conference and all around great get together of techies, nerds, and geeks who were friends into a big giant marketing junket.

I expect this was a common fear because the whole shebang started off with an opening video on the big screens that was generally about Quest and world domination, etc and had Gil stepping in and say whoa whoa wait a minute… That isn’t what this is about. I loved that they made fun of that fear showing they recognized it and acknowledged it and wanted to put it to bed as fast as possible.

But the question remained, was it ‘for real’? Well I can say… It was. And that seriously surprised me. As I walked around D… TEC, not once was I accosted by a salesperson from Quest. No one jumped out from around a corner, no one tried to trap me, no one said, hey how about I give you a free pass to a Vegas show and you sit in a room as we try to sell you something for a couple of hours (at least no one at the conference did that – the Strip was another story…) 

Now let me tell you, I have been talking to Quest sales folks for many years, probably seven or so? This is in the context as the lead tech resource for several very large companies while Quest was trying to sell them something. While the technical resources I have met with on the sales calls[1] have all been a pleasure to talk to[2], the sales guys were all a bit over the top if you know what I mean. I know that there had to be some serious wrangling somewhere inside of Quest before TEC came along to get the sales sharks to not be their normal piranha like selves. I don’t know, maybe they didn’t invite the sales people, I am not sure, but whatever they did was perfect. If I wanted to talk about products Quest had for sale there was a special room I could walk into with all of the other vendors and I could talk about whatever I wanted. But if I wanted to avoid them, I could do that as well.

I realize I just burned up a large chunk of the post so far but that was a big concern of mine when I decided to attend. Again, I have dealt with several members of the Quest Sales team in the past, they aren’t a calm accepting bunch. Anyway, on to other topics…

Vegas!!! Or at least Henderson anyway… What can you say about that? I can say low 70’s and a boat load of sun beats out snow and freezing temperatures in Chicago. I think they learned their lesson and we won’t see Chicago or likely anywhere else cold again for TEC. I know many companies don’t want to send their employees on Vegas boondoggles (why spend on that when they can send the Execs on Hawaiian boondoggles instead…) however the funny thing about Vegas and nerds is that they still show up at the sessions, they still sit and chat with the other nerds.

This isn’t a bunch of realtors or insurance salespeople who go to Vegas for a conference and then never even know where the conference is at because they are drunk and falling over in the strip clubs the whole time. These are hard core nerds who would rather sit in a chat about replication issues than at a card table. They would rather try to understand Brett talking about the intricacies of JET/ESE than sit at a slot machine. They would rather listen to Dmitri talking about the various fields of Access Control Entry Structure than see the Blue Man Group banging on drums. Why? Because the info you can get at TEC isn’t info you are going to get anywhere or anywhen else[3] and you can play cards, play slots, or see the Blue Man Group anytime. I don’t care how many other conferences you go to, I don’t care which ones they are, you will not get the quality of info concerning AD and Windows Identity that you do at TEC. You won’t get the friendly feeling, you won’t get the speakers and MVPs and Microsoft employees sitting around in bean bags chatting comfortably as peers with the attendees. Pamela Dingle wrote up a great experience post here - http://eternallyoptimistic.com/2009/03/26/tec-and-the-targeted-conference-value-proposition/ which I completely agree with as evidenced by my comment to her post.

Obviously, due to the financial conditions around the business world, the attendance was down but it was still great attendance. Microsoft wasn’t able to send as many people but the Softies that did attend were some of the crème de la crème and the folks the DS MVPs really enjoy seeing at the MVP summits. You had Dean formerly of the Dean and joe Show, you had James McColl who “owns” the DS Power Shell extensions, you had Nathan “Mr. RODC” Muggli, you had Dmitri Gavrilov who has touched so many parts of the AD and ADAM source that it is impossible for anyone to manage the directories without using something Dmitri worked on, you had Brett Shirley who is literally one of a few people who can read, let alone write the ESE code which is the underpinning Database for AD and Exchange as well as DHCP and WINS and so many things on the Vista or Windows 7 machine[4] you are reading this posting on that you would be shocked. Of course we had Stuart Kwan with his keynote and the challenges he issues every TEC. You may recall last year’s challenge was to produce a movie poster that incorporated various MVPs and Microsoft folks about AD Forest Recovery… the response was

This year the challenge was to produce a song based on an Elvis song that had lyrics exploring our top requests for the next releases of product from Microsoft. These are the lyrics…

(sung to the tune of Elvis Presley’s “Blue Suede Shoes”)

v.1 was the money
v.2 for the show
v.3 got you ready, now go go go

So don’t you
Recycle my new OU
You can do anything
But can you RTM ILM 2?

Can you give us some prune and
Can you give us some graft?
You better make us happy
So don’t give us the shaft.
took a purdy name and you made it a mess
Leave it as ADAM, not LDS

So don’t you
Recycle my new OU
You can do anything
But can you RTM ILM 2?

Now that you’ve got – an Identity Suite
Gimme some lovin on schema delete,
I need multi-domain on a single DC
I’m talkin’ ‘bout writable not a GC

So don’t you
Recycle my new OU
You can do anything
But can you RTM ILM 2?

Who killed LDAP, and didn’t tell me.
What the hell’s a web-service doin’ on a DC.
Give me a GUI, not command-line hell
I need a management solution, not Power—-WHAT?!?

….

So don’t you
Recycle my new OU
You can do anything
But can you RTM ILM 2?

Several folks got together at Midnight on Tuesday night to sing this in front of a camera, I can’t be sure but I think alcohol played a role and I distinctly recall seeing Dean with a pink scarf wrapped around his head in it when I watched it. I think someone (Sean??) will post it some time in the near future so you can see this work of art as sung by the creators.

Because I love my buddy Dean I want to call out his session in particular. The Dean and James presentation was good, I won’t say it was as good as the Dean and joe Show but admittedly Dean was working with far more constrained material than we had to work with. Certainly I think James is a far more polished presenter than I am so that was good. Their presentation seemed to go over well but I am not sure everyone understood everything about the implementation and how far down the road the stuff they were talking about was. Hopefully everyone saw in some way shape or form the Joe Kaplan and Brandon Shell presentation on what you can do with Power Shell against Active Directory right now today. Also unfortunately no one asked the question I really wanted to hear asked and answered that I mentioned previously… The question about bytes on the wire when retrieving the description or even better userAccountControl or security descriptor of 50,000 or so users… Anyway the stuff that JoeK and Bwandon were showing off was good stuff and available right this second and uses LDAP which I consider a huge win over the stuff coming in Windows Server 2008 R2 (sorry Dennis, I still don’t feel there is anything compelling in my environments). As JoeK said, the AD PS stuff is Vaporware and as Bwandon said, they aren’t even delivering what Quest delivered two years ago. Its a strategic decision…

I also wanted to call out Brett’s sessions. I only got to see one of his sessions in its entirety but it was very good. Brett is fighting a couple of major issues with his presentations. The first is that he is really very smart and has forgotten more about ESE/JET than any of the rest of us will likely every know. It is difficult to slow down and speak to people who don’t have seriously deep experience with a product if you live deeply in that product, at least IMO. Coupled with that is the second point which is that ESE/JET has pretty purposely been kept as a black box all of these years. It is literally shown as a little box at the bottom of AD architecture docs and you simply see “ESE” and nothing else. So when Brett started talking about ESE and really started getting into it, it was a topic that most people didn’t even have a casual background on other than some occasional key words here and there like ISAM or B-Tree or Version Store. I admit that I was listening and often thinking “wow I don’t have the slightest clue what he is talking about”. I hope we see Brett again next year presenting again and hopefully more people will be more informed with what ESE is and the terms he is using. Also maybe he can come down just a little out of the deeper parts of the box and talk more specifically about the pieces of ESE/JET where we feel pain and that can be directly translated to ESE/JET components such as maybe the Version Store issues we had with Active Directory prior to LVR replication. I think a lot of people could use to hear more about the actual physical layout of AD in the database as well such as the relationships between DNTs, PDNTs, NCDNTs, etc. Brett’s presentation was my favorite and I wish it could have been about 4-5 hours because learning more about ESE/JET is something I would like to do.

Unfortunately I do have a complaint about TEC this year that I think heard enough times outside of myself that it should be mentioned here. The problem was around session scheduling. There was a lot of collisions in the schedule that I didn’t think were very good. Obviously scheduling something like this has to be an incredible pain and very difficult at a core level but it needs more work. I think possibly something that would be good to do is to have everyone select the sessions they are interested in seeing and then put popular sessions up against less popular sessions. What we had this year was multiple well known personalities going up against each other in the same time slots and people complaining that they wanted to see both (or more) but obviously could only attend one. One that really irked me was Brett and Dmitri both presenting at the same time. I absolutely wanted to be in both sessions but obviously couldn’t so tried to split my time between them which was wholly unsatisfactory. So I would have an hour and half period where I couldn’t figure out which session I wanted to see because I wanted to see several and then I would have an hour and half that I didn’t want to see any of the sessions. Again, I understand how difficult this problem could be to try and solve and make everyone happy but I think putting all of the well known folks in the same track at least may be a good start so you don’t have to choose Guido or Dean, Brian Puhl or Brian Desmond, Dmitri or Brett, Joe Kaplan or Darren, etc.

I was talking with some folks at TEC on a couple of occasions and I started speaking out about comments that had popped up in my head at some point that TEC is almost the Microsoft MVP Summit Part Deux. I get a very similar feeling from it. Most of the folks reading this won’t really understand what that means because they aren’t MVPs but trust me that this is a very good thing. The summit is open honest direct content sharing between Microsoft and the MVPs and feedback about what was shared – both ways. When the summit is before TEC, then the MVPs who present are bringing that information straight to TEC and presenting it to the non-MVPs which is great for both Microsoft and for the attendees. That allows the Microsoft folks to watch the MVPs and see how well their messages got through from the summit which I expect gives them the opportunity to tweak the messages they are trying to put out there as well as correct things that were misunderstood. But also, since you have so many MVPs and the same Microsoft people there, the same feeling of the MVP summit and the easy open communication between Microsoft and the summit attendees rolls over into TEC which I think has a tangible positive impact on the conference as a whole. People are in a better mood and more willing to talk and share and actually “communicate” when they are comfortable and the Softies and the MVPs are very comfortable with each other and that translates to a better experience for everyone at TEC. 

Thanks have to go out to Christine and Gil and Stella and everyone else that was involved with putting on such an important function. They need to pat themselves on the back and kick their feet up and let out a sigh of happiness. Bravo.

Well to wrap this post up, as always, I very much enjoyed TEC and while I may get “ho hum” or “can’t possibly do it” about going every year because of my obligations and other issues I am always end up being very glad that I come. I wish I could bring my entire Active Directory Staff I work with in my day job because I think there is something for all of them and it will just make them better at their jobs. It comes down to the fact that I love the people, plain and simple. Everyone is always extremely polite and respectful and for the most part interesting. There is such a wide variety of people using AD in such a wide variety of ways it is interesting to hear the various viewpoints. Don’t let anyone tell you anything different… TEC is about the people. It is about the interconnections and the hallway discussions. As I tell people, as a general rule the learning just starts in the conference rooms, it really gets going in the hallways and after the official sessions when people really start talking about how what they saw in the sessions personally impacted them and how they worked around it. I had people asking me questions on stuff from the Dean and joe Show from 2006 and I believe they felt fully comfortable to do it and who knows how long they were thinking about doing that… What other conference out there now has such history and continuity that that could happen?

     joe

[1] In fact the first time I met Darren Mar-Elia he was the CTO of Quest and had flown out to see me and some others at the Widget factory I used to work at and to this day I still consider Darren a good friend even though he has an unnatural tendency towards all things GPO…

[2] And Darren isn’t the only great technical type person I have spoken with from Quest, there are lots. That company has some serious brain power in it and some very good people that I am always happy to see.

[3] Unless you are a Microsoft MVP and you get to go to the Summit at Microsoft which is an invitation only NDA event.

[4] Search your machine for edb.chk files, every one of those is part of an ESE Database.

As some of you were witness at TEC 2009[1], I have put down my work on LDAP (since its a dead protocol and all…) and decided to enter the exciting field of Cybernetics, Robotics, and Artificial Intelligence Constructs. I was secretly testing my latest secret beta of the Tracy-bot artificial intelligence construct while attending TEC 2009.

While the sheer presence of the Tracy-bot in the technical sessions with a bunch of Nerds, Geeks, and other technical folks was enough to cause some folks to guess that it was an artificial construct, others needed to see the Tracy-bot getting drinks and snacks for myself and my friends as well as refusing to take money to go gamble to fully understand it was an artificial device. Still others were just standing around completely unaware and/or confused by the fact that the Tracy-bot was not human.

For those of you who didn’t realize that an artificial product was running around the conference, here is a picture of the secret beta product and possibly you will recall the artificial life form’s presence in several technical sessions at TEC 2009, including the Active Directory ESE (Extensible Storage Engine) session put on by none other than #2 of 6… Brett Shirley up to and including shouting out comments to have the presentation continue so Brett could show off his cool ESE Data Commit in Action Slide.

The Tracy-bot kernel is code complete. The product will not be released until there is a full Software Development Kit for complete customization via Perl scripting as well as many pluggable modules such as the Chess Expert Module, Dungeons and Dragons module, Face Book module, French Maid Module, Catholic School Girl Module, Lawn Mowing Module, Pole Dancing module, Back Scratching Module, as well as the very difficult to produce “Enjoying Watching Science Fiction Shows” module. I have completed several modules but have been having issues with the Sci-Fi shows module as it is still producing crashes and system hangs in the kernel software that result in a snore like sound output as well as unintended hand gestures that result in channel changing.

Please note that neither a .NET Framework interface nor a Power Shell provider will ever be built or supported for the Tracy-bot. The goal is to make sure the Tracy-bot doesn’t become bloated and non-performant and that requires careful programming with native code. The engine is based on the Windows Server 2008 R2 Server Core OS though a fully functional Microsoft Surface GUI not based on any .NET framework components nor Explorer has been produced and is going through intense acceptance testing. Virtual versions of the Tracy-bot are being considered for the Nintendo Wii and iPod Touch.

No requests for beta products will be accepted. This is a closed beta testing program.

  joe

[1] Note that this is NOT my wrap-up on DEC/TEC/Whatever, simply a quick comment on what was going on as I have received some email on the subject and people are concerned that good non-NET based tools will no longer be produced.

So who is going to TEC? Excited yet? You should be, it will be fun. Personally, I can’t wait to hear Dmitri, Brett, and yes, even Dean, speak. Those guys are incredibly bright and just plain know a lot of stuff. The weather in Vegas is slated to be in the low 70’s and sunny. We missed the 80’s by a week unfortunately.

If you are sitting in the sessions and wondering what kind of questions you should be asking…

In the sessions on PowerShell and ADAC (or ADMUX if you prefer) ask why the PowerShell cmdlets aren’t using LDAP[1] which is already present on every single domain controller… But instead a brand new “Web” Service that runs on a Windows Server 2008 R2 Domain Controller. See

http://technet.microsoft.com/en-us/library/dd378937.aspx

http://technet.microsoft.com/en-us/library/dd391908.aspx

http://msdn.microsoft.com/en-us/library/dd303965(PROT.10).aspx

http://msdn.microsoft.com/en-us/library/dd304395(PROT.10).aspx

No, this doesn’t mean you need to load IIS on the DCs. It is another binary. And in fact it really doesn’t have any HTTP involvement and doesn’t run over port 80, it is just XML. Just to get that easy question out of the way… But maybe questions along the amount of network traffic may be good ones to ask about since XML is such a ’sparse’ protocol compared to LDAP. Maybe if we are lucky we could get a demo of (including network traffic comparison) of say dumping the email addresses for all users in a 50k user forest to a text file as done via LDAP and done through PowerShell using the AD Cmdlets. That is actually a test I have been wanting to do but haven’t had the opportunity to set up a 2008R2 DC to do the tests. If I could only just install the PowerShell AD cmdlets to test… ;)  Another interesting test of something that I just had to do at work yesterday in fact would be to dump the replication metadata value for the legacyExchangeDN attribute for all person objects in a 75k user forest[3]. I needed to see if the LEDN had gone through some mass change at some point so was interested in version numbers and originating write dates. I can’t say for sure what the network impact delta would be between these two types of requests but I could hazard a guess.

Don’t worry, one of the presenters of this info is my good friend Dean Wells formerly the Dean of the Dean and joe Show before he joined the dark side (<insert heavy breathing>I’m your father Lucas</heavy breathing>) and he actually enjoys audience participation versus just everyone sitting there dead silent. If you weren’t planning on attending that session… “What are you INSANE!”. You know Dean, he is easily, hands down, the best presenter I have ever seen. It may not be the most informative presentation of the whole conference (though it very well likely could be - I would say it would be but it isn’t the Dean and joe Show where we could pick any topics we wanted, the topic is constrained) but it will certainly be the most entertaining and energetic and fun presentation at the whole conference as Dean could present the phone book and have everyone trying to get him to do a second session[4]. If you can make him stutter or be unable to respond to a question, kudos to you because I just haven’t seen it happen. No fair asking him questions like “Do you still beat your maid that you snuck into the country illegally from Romania?” as that just isn’t nice and really, how do you answer that question without stuttering? No, err yes err… Err Romania?? Where’s that? Oh Maid?? I don’t have a maid! ;o)

Note that I am aware that Active Directory 4th Edition is now out on the book shelves and at Amazon. While I won’t be bringing any copies I think Brian may be bringing copies for purchase. I don’t believe he will have any to give out for free because unlike when AD3E came out, O’Reilly didn’t seem interested in giving copies out at DEC err darnit TEC[5].  I don’t mind signing the books however you probably really want Brian’s signature as he is the author for this version. He took what I did and extended it and chopped out a bunch of stuff that I wanted to chop out but wasn’t allowed to chop out. 

So if you are going to TEC/DEC/Whatever… Have a safe flight/drive/train trip/whatever. If you aren’t going… I’m sorry that sucks. I know the company I work for wasn’t interested in paying my way and I didn’t want to speak to have NetPro/Quest pay to get me out there so I took it as a vacation time and am paying for it. On reflection, it probably is silly of me not to speak but in all honesty, I don’t much like to do it unless I have something specific I need to say and I never have anything I need to say at DEC. I just go to meet the people and talk to folks about the issues they encounter to give me ideas on ways I can try to help. If you folks who attend DEC/TEC/Whatever think that had value, tell every NetPro/Quest person you run into that and maybe they will bring me out there just to chat with people like I usually do. Especially do that if you have in the past told the NetPro/Quest people that you wanted to meet me or came because of me or my blog.

I think though if next year I am taking my vacation at this time and I am paying for it, I should take it in the Bahamas and if someone happens to be there that is into Active Directory stuff then I can chat with them there. :) 

    joe

[1] Don’t worry, the Quest cmdlets for AD still use it though. So they will work against the various versions of AD without the need of a K8R2 DC with the Web Service loaded.

[2] That may or may not be a sarcastic comment.

[3] adfind -gcb -f “&(objectcategory=person)(legacyexchangedn=*)” msDS-ReplAttributeMetaData;binary -mvfilter msDS-ReplAttributeMetaData=legacyexchangedn -csv

[4] Serously, my opinion of Dean’s speaking ability is that high. Trust me on this, I had to try and speak after him in the same presentation and generally I consider myself to be a humourous interesting person to chat with, just not after Dean has been up there prancing around in his glory. ;o)

[5] Had NetPro/Quest/Whatever not changed the name of the conference, who knows, O’Reilly might have gave out some free copies… Now instead they hear, free books for TEC? Why? Now DEC, that we would give out free books at… (Man I am on a roll today!)

Its St. Patrick’s day and you know what that means… Buy and deploy a Carbon Monoxide Sensor Day or change the batteries in your existing CO Detector.

Many, if not all of you will recall, last year I was poisoned by Carbon Monoxide. I am doing well now. I am strongly of the opinion that this did impact my overall health as this last year I have been sick more times than I have been since I was a little kid. Obviously it could be that I am getting older but I don’t think it is because I am in worse shape because I am in better shape than I have been in for years. Physically active, not drinking hardly any soda, eating overall better, attending running events (albeit as a spectator LOL) etc etc ad nauseum.

I have to say I am extremely pleased with the number of people I have heard from who have deployed Carbon Monoxide detectors since I started talking about the issues I encountered. I heard lots of stories of people who were in a similar position that I was with CO detectors that were 5, 7, 10, 12 years old and never knew they went bad as they would do the alarm tests just fine. The fact that you usually don’t or won’t know when a CO detector has gone bad is why I will buy a new one every year and put it in my house. The idea being that there will always be at least one detector in the house that absolutely should be working since it will be less than a year old.

Recently I heard of two issues with CO, one was a family that had installed a fireplace or something incorrectly back near the town I grew up in. The other was my own brother in law who was being poisoned by it at work. It put a few of his coworkers in the hospital I guess.

Carbon Monoxide is serious stuff. Take it very seriously. Its one of those things that once it starts getting you, you are less and less likely to do anything about it until you are dead. And then you totally won’t do anything about it…

One of the things I love about the MVP summit is getting together with really smart people and discussing various deployment architectures.

One of the topics of conversation during a get together at the Experience Music Project social event was a discussion about how to make a high visibility public web site based on Windows Servers in an Active Directory domain highly available.

First you need to discuss what highly available AD means…

Ability to logon due to a single DC or infrastructure failure is only one aspect of a highly available environment. What about being available through

• security compromise
• administrative "oh shit!" (OS!) events
• updates that just didn’t go right

While AD naturally has a fault tolerant distributed deployment model, that does nothing to help with those types of issues. In fact, depending on how it is all managed, a distributed deployment model could contribute to the possibility of these issues as well as the overall impact.

Security compromise… For the most part, *most* companies *probably* don’t have to worry about someone outright attacking their AD environment. However that doesn’t mean no one has to worry about it. In those companies where these concerns are real, security needs to be in the front seat for the high availability discussions… Think of the military, think of the government, think of the NSA, think of NASA, think of Microsoft, think of Apple, think of very large companies that are likely targets for corporate theft/espionage, think of companies using AD in a DMZ or similar for internet facing applications. The directory is in an exposed position and it is pretty much a certainty that there is someone who knows more about how to compromise things than the person running the AD knows at some point. Don’t take that as an insult… In the battle of good versus evil in the security world, you as the good guy have to be on the ball and right 100% of the time, the bad guy only has to be right once. Due to the nature of AD, if you have compromised one DC, it is a short step to compromise all DCs in a forest.

"OS!"… Everyone needs to be concerned about "OS!" events. PERIOD. We are, I believe, all humans, humans make mistakes, failure to take that into account in the first place is just one more failure to add onto the list of items you are reviewing when performing the failure analysis. These types of mistakes made to the directory will quickly (you wanted low convergence times right?) replicate around your entire domain/forest. You accidently delete all users in an OU and soon they will be gone from all DCs.

Good updates going bad… I think many of us, especially those of us have been in this business a long while, have seen this happen. Something worked great in the lab and out in production something goes left instead of right and you are standing there going WTF[1]? And those without a production environment at all… Well they really are likely to have an issue. What do I mean when I say you don’t have a production environment??? Let me quote something Don Hatcherl[2] said on ActiveDir.Org when someone said they just had a production environment and no lab environment…

I have to make a comment here, as I’ve heard this too many times.  You do, in fact, have a lab environment.  What you do not have is a production environment.

DonH

I have a great story about updates going bad when I was working for a Fortune 5 and Microsoft Consulting guys were testing Schema updates in the lab (yes we had an official lab) and everything looked great to them and the testing went months so you would think any issues in there, they would have found. Well it comes to production and ugh… we have mangled attribute names on several attributes. This is just one example of something that can go wrong. Fortunately that was pretty easy to fix but some other updates that go bad aren’t quite as easy to identify and fix. Anyone who ran into tcp chimney issues[3][4][5][6] with Windows Server 2003 SP2 can probably attest to that as it usually took some time to work out what was going on. That issue hit all DCs as well, but thankfully it wasn’t damaging, not like say… applying a kiosk GPO at the domain level and locking all machines down to kiosk mode or mucking with the machine certs of every machine in the domain or changing other security settings. All of which will replicate with lightning speed to the whole environment.

If you aren’t protected from these types of issues, can you protect yourself enough to build something where high availability AD means taking care of these items as well? It depends. It depends on how available is highly available to you and your company. This answer will vary and the resulting work and architecture that you need to put into place to cover for this will also vary. Like security this is a sliding scale that you need to slide to your sweet spot – or at least the spot you can deal with. For most companies, there is going to be a "good enough" point where they stop worrying about it because the concerns over money and resources to account for the problem exceed their concerns about the problem.

Back to the public web site…

In this environment, all three of these issues are very realistic and likely… in fact, even expected. These absolutely would be on the table as issues every single day of the admin’s life that had to run it. This environment must be absolutely available all of the time. Down time runs in the thousands of dollars per minute or perhaps even thousands of dollars per second. The environment absolutely would be a target for hackers and couldn’t afford downtime due to administrative OS! or update failures.

The first thing a configuration like this needs, which really isn’t about AD, is physical location redundancy. You do this by putting the web servers and domain controllers in multiple data centers. Say 4 data centers in North America, 4 data centers in Europe, 4 data centers in Asia. Regional failure/capacity planning says that you can lose a single data center and maintain standard performance, if you lose two in the same region, the site will still work but with reduced performance maybe costs you only a couple tens of thousands of dollars per hour which maybe is acceptable for short periods versus the cost of beefing up even more.

The next thing you need, which again isn’t about AD, is web/app server redundancy. You can throw as many web/app servers into every data center as you feel is needed to maintain availability. Also with the cool virtualization failover and resource management scenarios with VMWARE like VMotion / VMWARE HA / VMWARE DRS you don’t need quite as many web/app servers at any given moment to still have good redundancy.

Now we come to Active Directory. What is the best way to set AD up for this environment. The default thinking would be to set up a single domain forest with lots of DCs in each site. This might work out but I think it is wrong and you can’t properly address the three issues previously mentioned. AD is not isolated and a single forest cannot be isolated no matter how you try to break things up – staggering replication, OU security separation, whatever -  it is still all connected. Any security issues, changes, or mistakes that impact the whole forest impact every single data center. Obviously that would be silly to do after taking the time and money to break up the web site into different physical data centers in the first place. So what do we do? IMO… You have a single domain forest dedicated to *each* *individual* data center. No trusts, no interconnections, firewalled off from each other, completely free standing in each case. Updates only occur in one data center at a time and don’t move on to another data center until everything is validated as working 100%. But the costs of separate forests… Oh my! Oh wait, you are paying for separate data centers, what is the small cost of the extra forests compared to that? Seriously.

Thoughts?

  joe

[1] For those who don’t know this term… We shall say it is the NetBIOS name of the Windows Forest called "WindowsTestForest.loc".

[2] AD God

[3] http://msexchangeteam.com/archive/2007/07/18/446400.aspx

[4] http://blogs.msdn.com/jamesche/archive/2007/12/19/having-network-problems-on-win2003-sp2.aspx

[5] http://www.cisco.com/en/US/ts/fn/620/fn62761.html

[6] http://support.microsoft.com/kb/945977

A cowboy named Bud was overseeing his herd in a remote mountainous pasture in Idaho when suddenly a brand-new BMW advanced out of a dust cloud towards him.

The driver, a young man in a Brioni suit, Gucci shoes, RayBan sunglasses and YSL tie, leans out the window and asks the cowboy, ‘If I tell you exactly how many cows and calves you have in your herd, Will you give me a calf?’

Bud looks at the man, obviously a yuppie, then looks at his peace fully grazing herd and calmly answers, ‘Sure, Why not?’

The yuppie parks his car, whips out his Dell notebook computer, connects it to his Cingular RAZR V3 cell phone, and surfs to a NASA page on the Internet, where he calls up a GPS satellite to get an exact fix on his location which he then feeds to another NASA satellite that scans the area in an ultra-high-resolution photo. The young man then opens the digital photo in Adobe Photoshop and exports it to an image processing facility in Hamburg , Germany. Within seconds, he receives an email on his Palm Pilot that the image has been processed and the data stored. He then accesses an MS-SQL database through an ODBC connected Excel spreadsheet with email on his Blackberry and, after a few minutes, receives a response. Finally, he prints out a full-color, 150-page report on his hi-tech, miniaturized HP LaserJet printer and finally turns to the cowboy and says, ‘You have exactly 1,586 cows and calves.’

‘That’s right. Well, I guess you can take one of my calves,’ says Bud.

He watches the young man select one of the animals and looks on amused as the young man stuffs it into the trunk of his car.

Then Bud says to the young man, ‘Hey, if I can tell you exactly what your business is, will you give me back my calf?’

The young man thinks about it for a second and then says, ‘Okay, why not?’

‘You’re a Congressman for the U.S.Government’, says Bud.

‘Wow! That’s correct,’ says the yuppie, ‘but how did you guess that?’

‘No guessing required.’ answered the cowboy. ‘You showed up here even though nobody called you; you want to get paid for an answer I already knew, to a question I never asked.  You tried to show me how much smarter than I you are; and you don’t know a thing about cows….this is a herd of sheep.

Now give me back my dog.

I just wanted to take a moment to send out a general thanks to the Active Directory Product Group at Microsoft. I always enjoy seeing the team as a whole but really enjoy the individual talk time with the PM’s and Devs even more. My favorite time for this summit was the open session where we got to sit and discuss the Recycle Bin feature stuff with Tim and Stephanie. Not because I think it is the only cool feature or anything, but because I really enjoyed the open discussion and it was nice to learn the technical details behind a feature that has been so long in coming. 

Uday and Moon, your presence was missed. Also I will miss seeing a large portion of the DS Team at DEC/TEC… Unfortunately it seems only a few of the DS Team will be able to make it to DEC/TEC due to budget cutbacks at MSFT. This is unfortunate because I often hear from the attendees that they really appreciate the availability of the Microsoft DS Team resources at DEC to respond to questions and hear feedback. Since this is the only conference with solid real deep focus on the DS pieces this is the best conference for Microsoft to send those resources but someone at some level doesn’t seem to understand that.

It was odd having my good friend Dean over on the other side of the fence as a Microsoft employee but I think it was good and he did Microsoft proud. I had a lovely time visiting with him and his wife (hmmm good meatballs) and awesome son (Hello Lucas… volcano!!!).

Just a quick FYI for those who kept asking me… The fact that Dean is working on PowerShell a lot now is not going to sway my opinion on it. Again, sorry to all those who asked me that specific direct question… but no. Not even for Dean. I will or will not use PowerShell when it becomes compelling specifically for me to use. This isn’t to say it isn’t the right answer for others, but that is for each individual to decide. I doubt there is anything that can be presented or said in a single presentation that could get any serious folks to just jump whole hog, this will be something admins choose to do or not based on their own thoughts, needs, and preferences. I found it odd when asked by some of the PM’s if I was just all of a sudden going to change my mind on it because of a good presentation. Does anyone make up their mind on whether they will change directions based on a good presentation? Once I need to do something that isn’t truly feasible for me to do (by my definition, no one else’s) in perl or command line tools that I have or can write, I will maybe look in that direction. More on the PowerShell stuff and specifically the DS Team’s AD and ADAM cmdlets for PowerShell in a later blog post.

Also a general thanks to the overall MVP program for having the Global Summit. It is always useful to me and this year by moving the Executive presentations to the end it really made me feel like my time was used well.

   joe

I’m sure that sounded like a good plan in the beginning…
I’ve had lots of those…
Like when I tried to breed wiener dogs with cats to get wiener cats…
Well you don’t get long skinny cats…
You get one dead cat and a dog with lots of scratches.

      - Randy (TV Sitcom – My Name is Earl)

The real Eleanor if you don’t know who she is…

The replacement? (http://www.thalondesign.com/files/smart.htm)

Recently needed to pull some QFE info to validate some things… Found some fun commands.

wmic qfe list full /format:htable

wmic /node:”servername” qfe list full /format:csv

Today I ran into an issue where a syncing tool blew up because it encountered an object that had multiple values in an attribute that “normally” has a single value in it and it didn’t know how to properly sync that object. Well obviously whomever set up the syncing tool kind of made a mistake because the attribute in AD was multivalued so there was always a chance this would happen so they should have accounted for it in the configuration of the syncing tool. But they didn’t and so I had to go figure out what was going on. Looking at it I quickly realized the problem and was able to point out that an object had multiple values for destinationIndicator and the company standard was to only have a single value. So this raised the question for me, are there any other objects out there in the same boat? I.E. They had multiple values for that attribute and would eventually cause me pain in the syncing tool…

So the generic question was, how do I quickly ascertain which objects had multiple values for a given attribute… Initially I thought, this will be a bit of a pain, I will have to output all objects with that attribute populated and then parse the attribute out with a script… Then I thought… Well I don’t actually care about the various values, just whether or not there are multiple values at all… Then a flash of EUREKA! AdFind to the rescue… (again).

Side Bar: I love when I come up with new ways to use my utilities that I never thought of before. It just encourages me more and more to have flexibility at the core of the utilities because I never know what I or someone else may end up doing with them.

I realized that all I had to do was be able to pick out objects that had multiple values for a single attribute so I typed up a command line like

adfind -h domainname -default -f “&(objectcategory=person)(destinationindicator=*)” destinationindicator -csv

and then added one simple switch that made all the difference in my ability to quickly scan through and find multiple values…

-csvmvdelim ####

So the whole command looked like

adfind -h domainname -default -f “&(objectcategory=person)(destinationindicator=*)” destinationindicator -csv -csvmvdelim ####

Now you may be sitting there thinking… “Well how the heck does that help anything???”. It doesn’t by itself, but when you combine that with grep or find or findstr or whatever and you pipe the output of that command into one of those search tools looking for the string #### it will only output those lines of the CSV output that match… I.E. Objects with multiple values for that attribute. So in its entirety it looks like

adfind -h domainname -default -f “&(objectcategory=person)(destinationindicator=*)” destinationindicator -csv -csvmvdelim #### | grep ####

I don’t know about anyone else but I just think that is pretty cool.

    joe

AdFind V01.40.00 has emerged from the cocoon… The Butterfly Release is now available.

After I released V01.39.00 I intended to go work on AdMod and get that updated and released, however I started hearing reports of typos and other comments on the functionality in AdFind V01.39.00. So I decided to work on AdFind a little more. As mentioned previously I am glad I did because while poking around in the source I figured out how to get adfind piping to adfind working which is a big win.

You can get AdFind V01.40.00 here —> http://www.joeware.net/freetools/tools/adfind/index.htm

You can get the new usage screens by typing adfind /? or looking here —> http://www.joeware.net/freetools/tools/adfind/usage.htm

Fixes for V01.40.00

o Added multi-DN piping capability (AdFind will read DNs from STDIN)

o The -alldc+ switch I added to V01.39.00 wasn’t enabled for use. I have enabled it now in V01.40.00

o Fixed typos in usage

o Added Windows Server 2008 R2 (Windows 7) Decodes

o Added –csvqesc to specify the character use to escape quotes inside of CSV fields. Default is \

o Documented –extsrvinfo, I added this in V01.39.00 but forgot to add it to the usage.

o Added –srvctls which allows you to specify arbitrary server controls

o Added –showdelobjlinks for Windows Server  2008 R2

o Added –showrecycled for Windows Server  2008 R2

o Added -showdel+ for Windows Server  2008 R2

o Added –tdcdshort which gives a secondary shorter format for –tdcd like (xxxd:xxh:xxm:xxs)

o Added –ic (intermediate count for multi-DN piping)

o Added –ictsv (intermediate count with TSV output for multi-DN piping)

o Added –db (display base for multi-DN piping)

o Added –stdinsort to sort multi-DN piping

o Added –subset to output a subset of the returned objects (every second, every tenth, etc)

o Added default environment variable and environment file reading

o Added shortcut –sc ou:xx to find OU’s

o Added shortcut –sc trustdmp to output trusts in a domain

Yesterday I added a new member to the domain admins group in a company I do work for. Off the cuff I typed up a few comments to send along with the note indicating the access has been granted, thought I would share.

1. Do not share your credentials with anyone. Period.

2. Be scared when using the ID, you can do a lot of damage with it.

3. Have second and third thoughts before changing things.

4. When in GUI applications, if you don’t mean to change things, use CANCEL, not OK to exit dialogs.

5. You can do a lot from your normal userid, prefer to use it over the Domain Admin ID.

6. Try to solve issues without logging interactively into Domain Controllers.

7.  If you aren’t sure about something, don’t do it.

8. Ask questions. I much rather hear “I don’t know” than someone try to guess.

http://en.wikipedia.org/wiki/Charles_Darwin

This is a pretty common question folks ask about on the internet. They bought a new machine with 4GB of RAM and Windows is only telling them it has some lesser amount between 3GB - 4GB available so they think their machines are broken or that Windows is broken.

Here is an excellent article discussing this topic

http://www.hardforum.com/showthread.php?t=1035670

Recently I was searching for some specific OU’s in Active Directory that had a specific substring within the name. i.e. it didn’t start with a specific substring, the substring I wanted was buried in the main string like for example the substring Crab in the string Joe’s Crab Shack which would be represented as *crab* or in this specific case the LDAP filter looked like OU=*something.  This is called a medial search.

I knew that although I had more than a half a million objects in the directory I only had about 25 OU’s to search through so even though medial searches can be slower than normal searches, with only a handful of OU’s this really shouldn’t be too much of an issue. Surprise… It was. The query took a considerable amount of time to run and it really really shocked me. I could have output the relatively few OU’s and manually found the ones I wanted faster as it took over a minute to find the OU’s I needed.

This really caught my curiosity so I started digging into it. The very first step is to validate that the OU attribute was indexed, I was pretty confident it was but you need to start somewhere:

G:>adfind -sc s:ou searchflags

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: CN=Schema,CN=Configuration,DC=test,DC=loc

dn:CN=Organizational-Unit-Name,CN=Schema,CN=Configuration,DC=test,DC=loc
>searchFlags: 1 [INDEX(1)]

1 Objects returned

So yes, it was indexed. Next step is to look at the STATS of the query and make sure the index was being used…

G:\>adfind -default -f ou=*something -stats+only

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

Statistics
=================================
Elapsed Time: 89679 (ms)
Returned 0 entries of 714377 visited - (0.00%)

Used Filter:
(ou=*something)

Used Indices:
DNT_index:485112:N

Analysis
———————————
Hit Rate of 0.00% is Inefficient

No dedicated indices used for search, this is inefficient.

Indices used:

Index Name  : DNT_index
Record Count: 485112  (estimate)
Index Type  : Normal Attribute Index

Filter Breakdown:

(
(ou=*something)
)

WHOA! It isn’t using the OU attribute index, it is using the DNT index which is everything… Look at that 714377 entries visited… Now that doesn’t make much sense… Let’s take a step back, what is used if we use OU=*?

G:\>adfind -default -f ou=* -stats+only

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

Statistics
=================================
Elapsed Time: 150 (ms)
Returned 22 entries of 23 visited - (95.65%)

Used Filter:
(ou=*)

Used Indices:
idx_ou:23:N

Analysis
———————————
Hit Rate of 95.65% is Efficient

Indices used:

Index Name  : idx_ou
Record Count: 23  (estimate)
Index Type  : Normal Attribute Index

Filter Breakdown:

(
(ou=*)
)

Now that is more like it… 23 entries.

But what is happening, that makes seriously no sense. The logic of the Query Processor should be such that if you are searching for an attribute, even via a medial search, if there is a regular index… Use it, don’t use the DNT index. Even if it only cut it down by 10% that is still a nice reduction, in this case it would cut it from more than half a million objects to 23… That is a serious reduction.

So I tested some more and it seems that any time I tried to run a medial search against an attribute that had a normal index (searchFlags & 1) the Query Processor would fail to utilize the index even if the number of choices that had to be looked at was substantially reduced from the DNT index. The only time this worked ok was when an actual tuple (or medial) index was defined for the attribute like so.

G:\>adfind -default -f ou=*something -stats+only

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

Statistics
=================================
Elapsed Time: 60 (ms)
Returned 0 entries of 0 visited - (0.00%)

Used Filter:
(ou=*something)

Used Indices:
idx_ou:0:T

Analysis
———————————
Hit Rate of 0.00% is Inefficient

Indices used:

Index Name  : idx_ou
Record Count: 0  (estimate)
Index Type  : Tuple Index (medial substring index)

Filter Breakdown:

(
(ou=*something)
)

Note the index type… tuple. This of course is a workaround for this problem but you don’t generally want to go around making a bunch of tuple indexes as it breaks up the strings into chunks of 3 characters for comparison which could rather dramatically impact DIT size not to mention insertion of new instances of values for that attribute.

I tested this problem against Windows Server 2003 and Windows Server 2008 and decided to ping some friends of mine in the production groups at Microsoft in Redmond to see what they thought. Rather quickly Dmitri came back to me and said it looked like this was an area of improvement and to get it into the change request system to get it looked at. He gave me specific details and pointed out where I could find it in the source code to see what it was doing as well. Not only that, he gave me a second workaround that can used until such a time that Microsoft corrects the issue.

There are two solutions to this issue.

1. The first is as we saw above, define a tuple index. That requires a schema change and if you want to practice in your lab you can quickly do this like so…

adfind -sc s:ou searchflags -adcsv | admod searchflags::{{.:SET:32}}

and to clear it you use

adfind -sc s:ou searchflags -adcsv | admod searchflags::{{.:CLR:32}}

Then the attribute will look like

G:\>adfind -sc s:ou searchflags

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: CN=Schema,CN=Configuration,DC=test,DC=loc

dn:CN=Organizational-Unit-Name,CN=Schema,CN=Configuration,DC=test,DC=loc
>searchFlags: 33 [INDEX(1);TUPLE INDEX(32)]

1 Objects returned

2. The second workaround is what Dmitri clued me on to… Using what he termed as an “Index Hint”. Basically you add one more piece to the query that forces the Query Processor to look at the indexes again and actually chose correctly. It kind of sucks that it needs to be done but it is nice that it can be done at least. Changing the query to be “&(ou=*something)(ou=*)” will perform as you would expect (ou=*something) should perform all on its own. The (ou=*) is the hint that the Query Processor needs…

G:\>adfind -default -f “&(ou=*something)(ou=*)” -stats+only

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

Statistics
=================================
Elapsed Time: 20 (ms)
Returned 0 entries of 23 visited - (0.00%)

Used Filter:
( &  (ou=*something)  (ou=*) )

Used Indices:
idx_ou:23:N

Analysis
———————————
Hit Rate of 0.00% is Inefficient

Indices used:

Index Name  : idx_ou
Record Count: 23  (estimate)
Index Type  : Normal Attribute Index

Filter Breakdown:

(
(&
   (ou=*something)
   (ou=*)
)
)

Still not as efficient as a Tuple index but it doesn’t have the overhead.

I have been working with AD for almost 10 years now, I had no clue about this until just this week when I ran into it. I have always known medial searches to be a bit more slow than your normal searches but never had the time to really dig into it like this and I am glad I did. Definitely keep an eye open on this and if it is something that impacts your company, please be sure to do what I am going to do, report this as an issue to Microsoft for correction. The more people who report it, the more likely it will be fixed in some timely fashion. So call your TAMs up and say, hey I have a bug I want fixed… Here it is… Oh and at the same time, don’t forget to order your TAM (male or female) a nice new Joeware Thong (http://www.cafepress.com/joewarenet). They love em!!! I have had several TAMs email me laughing about having received them. I wouldn’t mind to see it happen more and more.

BTW, Dmitri Gavrilov is one of the speakers (http://www.tec2009.com/vegas/agenda/directory/speaker_bios.php) at The Experts Conference in Henderson (Green Valley Resort) this year and he will be talking about Query Performance. I intend to be in the room when he is presenting because I am sure there are other things I will learn from him as well. That has been my relationship with Dmitri from the beginning, he speaks and I learn things.

   joe

Back on Jan 21 I tipped my hand on a new capability in AdFind which for many people is extremely exciting based on the feedback in my inbox. To refresh your memory, that was the ability for AdFind to take in a list of Base DN’s to execute queries against. For short we will say piping AdFind into AdFind though you could, if for whatever reason gripped you, pipe dsquery into AdFind.

Well in a follow up conversation with my friend and co-Author Brian Desmond, he asked “How do I pipe DNs from AdFind into AdFind and then get counts for the number of users under each of those DN’s?” My response was… well you can’t. The whole counting mechanism is based on the number of objects AdFind returns period… But then I thought, I hacked in CSV when I didn’t think I could… then I hacked in the piping in multiple DNs when I didn’t think I could, let me give this a try before totally saying no… So voila, a new switch because you just know there aren’t enough switches in AdFind yet…. The new switch is -ic… where -c stands for count, -ic stands for intermediate count… That lets you do something like….

G:\>adfind -default -f ou=* -dsq | adfind -sc adobjcnt:user

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) February 2009

Using server: r2dc1.test.loc:3268
Directory: Windows Server 2003

BaseDN: OU=CharTests,OU=TestOU,DC=test,DC=loc
1 intermediate objects returned

BaseDN: OU=createtest,OU=TestOU,DC=test,DC=loc
10 intermediate objects returned

BaseDN: OU=Deleted,OU=XXXTest,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=XXXTest,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=Domain Controllers,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=Email,OU=My,DC=test,DC=loc
1 intermediate objects returned

BaseDN: OU=GPOTest,OU=TestOU,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=Groups,OU=My,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=Groups,OU=TestOU,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=HideTest,OU=TestOU,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=joeperm,OU=TestOU,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=My,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=oneleveldown,OU=createtest,OU=TestOU,DC=test,DC=loc
1 intermediate objects returned

BaseDN: OU=Outlook,OU=TestOU,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=permtest,OU=TestOU,DC=test,DC=loc
2 intermediate objects returned

BaseDN: OU=PoSTest,DC=test,DC=loc
700001 intermediate objects returned

BaseDN: OU=Skip,OU=TestXXX,OU=XXXTest,DC=test,DC=loc
1 intermediate objects returned

BaseDN: OU=TestXXX,OU=XXXTest,DC=test,DC=loc
4 intermediate objects returned

BaseDN: OU=TestDisable,OU=XXXTest,DC=test,DC=loc
0 intermediate objects returned

BaseDN: OU=TestOU,DC=test,DC=loc
6 intermediate objects returned

BaseDN: OU=Users,OU=My,DC=test,DC=loc
2 intermediate objects returned

BaseDN: OU=Users,OU=TestOU,DC=test,DC=loc
10 intermediate objects returned

700039 Objects returned

Anyone think that is pretty handy??? If so, you can thank Brian, he asked the right question at the right time… I modified the adobjcnt shortcut such that when it detects it is in multi-DN mode it will also insert the -ic switch as well as set the search scope to one-level. If you wanted counts of all of the users in each OU but you wanted the counts to roll up to the higher OU’s as well you would simply add the -s sub or -s subtree switch to your command.

But then I thought, while that is useful, it would be even more useful if I could somehow get that in a CSV format so I could use this more easily from scripts when trying to get a snapshot of an environment… I looked and there was just no way I could get it into the CSV code path. When you do CSV the whole counting section isn’t used and even if it were, it is outside of the location where the CSV code is and in order to try and get it in there would cause me to use some wholly unnatural global variables and other things that just made me go, no, I will not go there, that is too ugly, too inelegant… I know I do some bad things in code, but I don’t want to do THAT bad of things in code.

All hope is not lost however. I decided to add… yes… another switch. I know I know, another one truly isn’t needed but I wanted this functionality and if you don’t like it, just ignore the fact that it is there. This switch is not a very flexible switch, there are no modifiers for it. It is called -ictsv and it simply takes the -ic output and makes it into a TAB Delimited format output. This isn’t going to be tweaked to allow different delimiters or anything like that. It is a hack completely outside the normal CSV routines which have all that flexibility. I chose tab delimited because DNs have commas and it is unlikely (impossible? I don’t know, didn’t test) to see a tab in a DN and to be honest, I like tab delimited output. I usually use TABs for my delimiters for CSV output. Anyway that output looks like…

G:\>adfind -default -f ou=* -dsq | adfind -sc adobjcnt:user -ictsv
OU=CharTests,OU=TestOU,DC=test,DC=loc   1
OU=createtest,OU=TestOU,DC=test,DC=loc  10
OU=Deleted,OU=XXXTest,DC=test,DC=loc 0
OU=XXXTest,DC=test,DC=loc    0
OU=Domain Controllers,DC=test,DC=loc    0
OU=Email,OU=My,DC=test,DC=loc   1
OU=GPOTest,OU=TestOU,DC=test,DC=loc     0
OU=Groups,OU=My,DC=test,DC=loc  0
OU=Groups,OU=TestOU,DC=test,DC=loc      0
OU=HideTest,OU=TestOU,DC=test,DC=loc    0
OU=joeperm,OU=TestOU,DC=test,DC=loc     0
OU=My,DC=test,DC=loc    0
OU=oneleveldown,OU=createtest,OU=TestOU,DC=test,DC=loc  1
OU=Outlook,OU=TestOU,DC=test,DC=loc     0
OU=permtest,OU=TestOU,DC=test,DC=loc    2
OU=PoSTest,DC=test,DC=loc       700001
OU=Skip,OU=TestXXX,OU=XXXTest,DC=test,DC=loc      1
OU=TestXXX,OU=XXXTest,DC=test,DC=loc      4
OU=TestDisable,OU=XXXTest,DC=test,DC=loc     0
OU=TestOU,DC=test,DC=loc        7
OU=Users,OU=My,DC=test,DC=loc   2
OU=Users,OU=TestOU,DC=test,DC=loc       10

Pretty cool huh… Anyone think that is handy? If so… well you are welcome… But something still bothers me about that output… Anyone else bothered by it? It could be just me but I kind of like seeing things that normally have a hierarchical form to be displayed that way. I don’t mean in the white space, but instead, I mean I don’t want to see something like

OU=Outlook,OU=TestOU,DC=test,DC=loc     0
OU=permtest,OU=TestOU,DC=test,DC=loc    2
OU=TestOU,DC=test,DC=loc        7

I want to see the TestOU first and then its sub-OU’s after… I tried modifying my search to see if I could force AD to return the info in that order but quite frankly, AD truly isn’t hierarchical, it just appears that way. It is actually a flat database. The idea of hierarchy is imposed on it for LDAP purposes. So I thought, I really need to do something about this… This will drive me nuts. However, trying to retrieve all of the information and maintain it in memory so I can then sort it is ridiculous, might as well just write this thing in PowerShell or .NET… (count it…) if I do it in such a silly way… Then I thought, wait, I already have what I need in memory to sort it hierarchically after the DNs have been piped in so I added, yes thankyou, another switch called -stdinsort. Why did I do this with a switch? Because I didn’t want to assume someone would want it sorted like I would and I didn’t want to assume I would always want it sorted. Also I wanted to give myself the ability to sort it hierarchically as well as alphabetically both case sensitive and case insensitive, so it is a switch with a default sort order of hierarchical but you can add cialpha or csalpha to get the other types of sort… Now that output looks like

G:\>adfind -default -f ou=* -dsq | adfind -sc adobjcnt:user -ictsv -stdinsort
OU=Domain Controllers,DC=test,DC=loc    0
OU=My,DC=test,DC=loc    0
OU=Email,OU=My,DC=test,DC=loc   1
OU=Groups,OU=My,DC=test,DC=loc  0
OU=Users,OU=My,DC=test,DC=loc   2
OU=PoSTest,DC=test,DC=loc       700001
OU=TestOU,DC=test,DC=loc        7
OU=CharTests,OU=TestOU,DC=test,DC=loc   1
OU=createtest,OU=TestOU,DC=test,DC=loc  10
OU=oneleveldown,OU=createtest,OU=TestOU,DC=test,DC=loc  1
OU=GPOTest,OU=TestOU,DC=test,DC=loc     0
OU=Groups,OU=TestOU,DC=test,DC=loc      0
OU=HideTest,OU=TestOU,DC=test,DC=loc    0
OU=joeperm,OU=TestOU,DC=test,DC=loc     0
OU=Outlook,OU=TestOU,DC=test,DC=loc     0
OU=permtest,OU=TestOU,DC=test,DC=loc    2
OU=Users,OU=TestOU,DC=test,DC=loc       10
OU=XXXTest,DC=test,DC=loc       0
OU=Deleted,OU=XXXTest,DC=test,DC=loc    0
OU=TestDisable,OU=XXXTest,DC=test,DC=loc        0
OU=TestXXX,OU=XXXTest,DC=test,DC=loc    4
OU=Skip,OU=TestXXX,OU=XXXTest,DC=test,DC=loc    1

Much better… ;o)

This and more in the new version of AdFind… AdFind V01.40.00 is expected to emerge from the cocoon on Feb 13, 2009.

     joe

So wow, looked at this list of recalled Peanut Butter products for the first time today…

http://www.accessdata.fda.gov/scripts/peanutbutterrecall/index.cfm

Random super bowl thoughts…

1. Boring Springsteen put on a poor half time show.

2. Not sure why he wouldn’t have closed with Born in the USA with red white and blue fireworks. Likely would have been huge right now.

3. Cardinals did not deserve to win.

4. Cardinals offense and defense both were explosive at times but mostly they were inconsistent.

5. The 100 yard dash was just hilarious to me. That poor guy was at the 50 yard line thinking, why hasn’t anyone tackled me yet, I have to keep running????

6. The penalties were just getting stupid. Especially the unnecessary roughness / unsportsmanlike conduct

7. This is the first super bowl I remember that I didn’t come away thinking, wow the commercials were amazing, most just made me sort of chuckle.

8. First super bowl that I remember, including the one that I actually attended in Pasadena back in the 90’s (I won the tickets) that I actually sort of tracked what happened in the game.

9. Again, the 100 yard dash was just hilarious to me.

10. Some Comcast customers in Arizona got some porn instead of part of the 4th quarter - http://news.bbc.co.uk/2/hi/entertainment/7864733.stm

11. I am a Comcast customer and I noticed they were inserting in their own local commercials for themselves and companies I don’t think could ever afford a superbowl spot.

12. The bud light skiing commercial was pretty funny.

13. The bud Clydesdale fetch commercial was pretty funny.

14. A very funny super bowl commercial I didn’t get to see (I think local Comcast programming overwrite it), found it on a super bowl commercial web site was the e-Trade baby out takes commercial - http://www.superbowl-commercials.org/299.html

15. The Pedigree Dog commercial made me chuckle. This was the one with the random wild animals as pets instead of dogs.

16. The Bridgestone tire commercial with the guys hot-rodding around some moon somewhere and coming back to their car on blocks was pretty funny.

17. The insects stealing the Coca Cola was probably the only one that got close to the normal quality of super bowl commercials for me.

18. The Audi car chase through the decades was probably the next best in terms of quality. I was impressed with how they showed each decade with the appropriate filming quality and style.

19. The Pepsi “I’m Good” commercial made me chuckle.

20. The bud light meeting made me chuckle slightly.

21. The bud Clydesdale lover boy horse commercial was pretty good.

22. The bud Clydesdale immigrant commercial had me talking in a poor Scottish brogue for a bit…

23. This is probably my favorite commercial of the night because I was laughing all the way through it, Career Builder If you hate going to work - http://www.superbowl-commercials.org/276.html

I received an email this evening that I swore I had previously wrote something up on the blog for and didn’t, so I will share…

The email

From: xxx@domain.gov
Sent: Wednesday, January 28, 2009 7:28 PM
To: ‘joe@joeware.net’
Subject: Adfind piped into admod question

Good evening Joe,

First I’d like to commend you on your books, website, and apps, I use many of them almost on a daily basis. 

So here it is:

We have users who’s home folders are being moved from \\xxx-home to \\abc-xxx
now \\abc-xxx and \\xxx-home aren’t the only home servers so it’s not like i can replace them all so I’ve written this script to test it out. In the servers OU I placed a user account and have populated the home directory as \\xxx-home2\joesmith$…again this account is just for testing.

run this script

adfind -b “ou=xxx,dc=domain,dc=gov” -f “(&(objectcategory=person)(objectclass=user)(homedirectory=\\xxx-home*))” samaccountname -adcsv | admod -unsafe homedirectory::\\abc-xxx.domain.gov\%username%$

the script runs just fine and updates joesmith’s home directory to \\abc-xxx.domain.gov\joesmith$ which was anticipated.
So this works all fine and dandy, where the problem is, is if down the road we want to utilize this again it doesn’t work. so for instance:
user account joesmith who’s home directory is now \\abc-xxx.domain.gov\joesmith$ (home directory was changed with utilizing the above script)
if \\abc-xxx was being retired and we were putting \\abc-home10 in it’s place. If you adjust the script find homedirectory=\\abc-xxx* it wont find anything to alter. The only way to make it find something is to tell it to find homedirectory=*

At first I thought it was because of AD replication (that is all DC’s aren’t up to date with the correct information), but I checked all our DC and it was updated before I ran the 2nd update.  Then I thought well what if there was a space before the \\ so I opened up ADUC and ADSedit and verified there wasn’t s space in the homedirectory attribute.  
Any clue why it wouldn’t update the 2^nd time around?  Also what does the -adcsv –dsq switch do? Tried to find some documentation on it but there was little.

Any information you could provide would be much appreciated.

Thanks again,

     xxx

I get questions like this pretty regularly which is why I guess I thought I blogged about it before, but I couldn’t find a blog entry for it so here was my response

Hey xxx, glad you like the tools, book, et al.

First off, I wanted to point something out… This first command

adfind -b “ou=xxx,dc=domain,dc=gov” -f “(&(objectcategory=person)(objectclass=user)(homedirectory=\\xxx-home*))” samaccountname -adcsv | admod -unsafe homedirectory::\\abc-xxx.domain.gov\%username%$

has an issue with the admod portion. Specifically the %username% won’t work. That will decode to the current environment variable for username which should be your current logon account. That little shortcut in ADUC is just that, a shortcut in ADUC, it isn’t a global AD thing. I think what you would really want would be

adfind -b “ou=xxx,dc=domain,dc=gov” -f “(&(objectcategory=person)(objectclass=user)(homedirectory=\\xxx-home*))” samaccountname -adcsv | admod -unsafe homedirectory::\\abc-xxx.domain.gov\{{samaccountname}}$

That will take the samaccountname that is returned from the query and piped across to admod and use that to populate the username piece of the share name so it would be specific to every user queried, it wouldn’t set the value to your current username environment variable for every account returned.

Second, the issue you see with the \\abc-xxx… is doing something you probably don’t expect… In an LDAP query, the backslash (\) character is an escape character. It tells LDAP that something special is coming - specifically a hex character sequence like \2a for example. This is generally not a problem and it all sorts itself out so you likely never have an issue using \\someserver… However if you follow a backslash with character that is a valid HEX character the escape and the HEX character are taken together… In that case, the proper way to search for a value that has embedded back slash literals is to use \5c for each backslash, so it would be \5c\5cabc-xxx. You can check out http://msdn.microsoft.com/en-us/library/ms675768(VS.85).aspx which discusses this and more about creating query filters.

    joe

And then I realized I didn’t answer his last two questions so I added

Sorry, didn’t answer the other questions:

-adcsv : As the adfind usage indicates, it is a special CSV mode that embeds some extra info in the CSV for AdMod or whatever tool being piped to to better understand the request, for example some things that can get embedded would be hostname, userid, password, etc used in the adfind query.

   -adcsv xxx    Special CSV mode for interacting with other joeware tools.
                 xxx is an optional string that specifies value to use for
                 use for empty attribs.

-dsq : This is simply quoted DN output like DSQuery outputs.

  -dsq          DSQuery style quoted DN output

   joe

So a little more discussion on the embedded backslashes, as the MSDN article above discusses the following characters all need to be escaped with “need” being a little more loosely defined for the backslash character itself than the other characters.

So now I know at least one or two of you are thinking… wow… so joe, why don’t you just help the guy out and likely help me out too and just make AdFind smart enough to do that embedding work for me… Well I could try… But I really really try hard not to mess with the actual LDAP query that is entered. The minimal amount I do for -bit to do a find/replace on some specific strings is fired off by the actual -bit switch. If people have to specify a switch to encode backslashes why don’t they just instead encode the backslashes themselves since it isn’t that much more and I don’t have to try and guess which backslashes should and shouldn’t be encoded. But joe… \5c is a pain in the butt to remember… Yes, for me too… The way I remember it when I need to is to query for the object I know that has one of the troublesome homedirectories in it and then tell AdFind to return the homedirectory value in binary so I can quickly see the HEX characters I need to enter. Like so

[Thu 01/29/2009  3:32:04.61]
G:\new1\Dev\Current\CPP\AdFind\Release>adfind -default -f “&(name=someuser)(homedirectory=*)” homedirectory

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

dn:CN=someuser,OU=Users,OU=TestOU,DC=test,DC=loc
>homeDirectory: \\abc-xxx\someuser$

1 Objects returned

[Thu 01/29/2009  3:32:06.72]
G:\new1\Dev\Current\CPP\AdFind\Release>adfind -default -f “&(name=someuser)(homedirectory=*)” homedirectory;binary

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003
Base DN: DC=test,DC=loc

dn:CN=someuser,OU=Users,OU=TestOU,DC=test,DC=loc
>homeDirectory;binary: 5C5C 6162 632D 7878 785C 736F 6D65 7573 6572 24

1 Objects returned

Note the 5C5C right there at the front…

    joe

The conference formerly known as The Directory Experts Conference (DEC) and now known as The Experts Conference (TEC) For Directory & Identity is now on my itinerary for March. I really didn’t think I was going to be able to pull it off this year and by some spit, luck, and a miracle I was able to do so. This will set me back a pretty penny since this isn’t a work related trip at all (You know how companies are cutting back) but I do want to touch base with everyone as this is my main chance to see the “joeware” fans and the global AD crowd once a year.

So if you are booked and going, make sure you look me up when you are there. If you aren’t booked to go… why not? Hands down this is the best general public event for Active Directory and this year, Brett Shirley and Dmitri Gavrilov are speaking and seriously, I can’t wait to sit in on their sessions to learn from them and I have no doubt I will. I believe Dean will be speaking as well in his new Microsoft PM role. I will certainly be there to heckle him and cheer him on to success in his new Evil Empire role….  What was it again… Emperor Palpa-Deano?

    joe

Example 1: Get tokenGroups attribute for every direct member of the domain admins group.

F:\Dev\Current\CPP\AdFind\Release>adfind -default -f name="domain admins" member -list | adfind -s base tokengroups -resolvesids

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003

dn:CN=TestAdminClone,OU=Users,OU=TestOU,DC=test,DC=loc
>tokenGroups: BUILTIN\Users
>tokenGroups: BUILTIN\Administrators
>tokenGroups: TEST\DnsAdmins
>tokenGroups: TEST\Domain Users
>tokenGroups: TEST\Domain Admins

dn:CN=$joe,OU=Users,OU=My,DC=test,DC=loc
>tokenGroups: BUILTIN\Users
>tokenGroups: BUILTIN\Administrators
>tokenGroups: TEST\DnsAdmins
>tokenGroups: TEST\Schema Admins
>tokenGroups: TEST\Enterprise Admins
>tokenGroups: TEST\Domain Users
>tokenGroups: TEST\Domain Admins

dn:CN=Administrator,CN=Users,DC=test,DC=loc
>tokenGroups: BUILTIN\Users
>tokenGroups: BUILTIN\Administrators
>tokenGroups: TEST\DnsAdmins
>tokenGroups: TEST\Schema Admins
>tokenGroups: TEST\Enterprise Admins
>tokenGroups: TEST\Domain Users
>tokenGroups: TEST\Group Policy Creator Owners
>tokenGroups: TEST\Domain Admins

3 Objects returned

Example 2a: Get sAMAccountName for all direct members of Domain Admins group

F:\Dev\Current\CPP\AdFind\Release>adfind -default -f name="domain admins" member -list | adfind -s base samaccountname

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003

dn:CN=TestAdminClone,OU=Users,OU=TestOU,DC=test,DC=loc
>sAMAccountName: TestAdminClone

dn:CN=$joe,OU=Users,OU=My,DC=test,DC=loc
>sAMAccountName: $joe

dn:CN=Administrator,CN=Users,DC=test,DC=loc
>sAMAccountName: Administrator

3 Objects returned

Example 2b: Get sAMAccountName for all direct members of Domain Admins group (alternate method using attribute scoped query)

F:\Dev\Current\CPP\AdFind\Release>adfind -default -f name="domain admins" -dsq | adfind -s base -asq member samaccountname

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003

dn:CN=Administrator,CN=Users,DC=test,DC=loc
>sAMAccountName: Administrator

dn:CN=$joe,OU=Users,OU=My,DC=test,DC=loc
>sAMAccountName: $joe

dn:CN=TestAdminClone,OU=Users,OU=TestOU,DC=test,DC=loc
>sAMAccountName: TestAdminClone

3 Objects returned

Example 3: Dump the serviceBindingInformation attribute for any ADAM serviceConnectionPoint objects that belong to Windows XP Professional machines. And no, this isn’t contrived, this is something I have wanted to do a few times to find people playing with ADAM on workstation class machines as well as something I have been asked how to do in the past… Normally this would take a script.

F:\Dev\Current\CPP\AdFind\Release>adfind -default -f "&(objectcategory=computer)(operatingsystem=Windows XP Professional)" -dsq | adfind -s one -f "&(objectcategory=serviceconnectionpoint)(serviceclassname=LDAP)" servicebindinginformation

AdFind V01.40.00cpp **BETA** Joe Richards (joe@joeware.net) January 2009

Using server: TEST-DC1.test.loc:389
Directory: Windows Server 2003

dn:CN={cb226d48-e9a7-420a-bce0-98edcc6b8cf2},CN=SFMXP64,CN=Computers,DC=test,DC=loc
>serviceBindingInformation: ldaps://sfmxp64.test.loc:636
>serviceBindingInformation: ldap://sfmxp64.test.loc:389

dn:CN={e4756748-fc8b-4166-8b7b-498bd97c9949},CN=SFMXP64,CN=Computers,DC=test,DC=loc
>serviceBindingInformation: ldaps://sfmxp64.test.loc:636
>serviceBindingInformation: ldap://sfmxp64.test.loc:389

dn:CN={3991aa1d-c87d-46cd-a08c-c52af9ae168e},CN=SFMXP32,CN=Computers,DC=test,DC=loc
>serviceBindingInformation: ldaps://sfmxp32.test.loc:636
>serviceBindingInformation: ldap://sfmxp32.test.loc:389

dn:CN={d3c0e3ee-d888-496d-acf0-6e4078052476},CN=HOMEXP,CN=Computers,DC=test,DC=loc
>serviceBindingInformation: ldaps://homexp.test.loc:636
>serviceBindingInformation: ldap://homexp.test.loc:389

4 Objects returned

When working on fixing several typos and mistakes in V01.39.00 I realized I had figured out a way to handle the piping from one instance of AdFind to another. I am still looking for holes but it looks good thus far. This is something I have wanted for some time but couldn’t see how to handle it with the current framework. Well out of the blue while looking at part of the code it struck me how to pull it off so I added it. Aside from that are the correction of some typos folks reported to me as well as adding some Windows 7 support (i.e. decoding some of the Windows 7 attribute values and support for the new object deletion/recycling capabilities).

   joe

Ok so how about we let the future decide on what is and isn’t history and how about we get to work on getting our collective asses out of the hole we put ourselves in. I would much rather put my time and focus and energy on that versus getting sprained elbows patting ourselves on the back talking about how we are making history. Failure to do so will have a future that records history as “…and then they all constantly celebrated about how they made history and quickly plunged the country into an even darker place because they spent all their time talking about how they made history instead of working on the problems they needed to solve for the future….”

Someone on ActiveDir Org list compared what happened with the presidency to Landing on the Moon. Fine, great. What does that do for us today? We haven’t been back to the moon in how many years? The 40th anniversary of the first manned moon landing is coming up on July 20… Six visits total. Last visit was Apollo 17  on December 14, 1972. That’s how long…

All the moon landings did was prove it *could be* done, they didn’t make it a daily reality. In fact, we still have a percentage of the population of the country who don’t even believe we went to the moon. So, all in all, how successful and historic and life changing was our landing on the moon? Really?

Apparently a new Sonic Drive-In… This last weekend we had 6-9 inches of snow in the Mt. Clemens area of Michigan. At the same time a new Sonic has opened up here (probably only 4-5 inches by the point these pics were taken). The traffic to this place is unbelievable. The nearest Sonic to this one is in SouthGate which is quite a ways away… But still the roads were horrendous.

The Sonic is Open!

Proceed to Staging Area????

Seriously???

Is Sonic really that good? Anyone?

   joe

AdFind V01.39.00 is now released. This is the first public release of AdFind since V01.37.00 in June 2007. V01.38.00 was never publicly released.

One of the biggest changes with this version is that I moved to a new compiler. V01.37.00 was compiled with Borland C++ Builder 6 (released in 2002).  V01.39.00 was compiled with Code Gear C++ Builder 2009 (released in 2008). I haven’t done extensive testing but the new executable appears to be faster and it is substantially smaller, about 40% smaller to be exact.

It feels good to finally get this version out there for you all to use. Please let me know any thoughts you have or issues that you encounter. I have been using this version with this compiler for some time internally and haven’t run into anything odd but there are so many of you out there using it now, there is always a possibility you will hit something that I haven’t. 

You can get AdFind V01.39.00 here —> http://www.joeware.net/freetools/tools/adfind/index.htm

You can get the new usage screens by typing adfind /? or looking here —> http://www.joeware.net/freetools/tools/adfind/usage.htm

List of the specific changes for AdFind, to get more info on the switches, shortcuts, etc, check out the usage screens.

o Changed longhorn references to Windows Server 2008
o Updated decoded attributes to account for Windows Server 2008 values
o More decoded attributes
o Documented decoded attributes in usage
o Multiple bug fixes
o Multiple usage screen fixes
o Multiple shortcut fixes
o -csv now also sets -noctl
o -sc sdump sorts multivalue attributes
o Arbitrary text mode for -csv which allows an attribute of header:value
o -rawsddl no longer requires -sddl
o Auto Ranging disabled for any attributes that are specified with a specific range.
o If no base is specified, assume -default
o mvfilter matching will be made without any modifier on the attribute
o All multiple values for a single attribute for -mvfilter/-mvnotfilter
o -binenc now does CURRENT encoding
o Added more attribs to -fullrootdse

o New Switches
   * rootdseanon
   * nirs
   * nirsx
   * writeable
   * sslignoresrvcert
   * mvsort
   * mvrsort
   * filterbreakdown
   * enccurrent
   * tdcd
   * inputdn

o New Shortcuts
   * admincountdmp
   * xrdump
   * dcdmp
   * adobjcnt
   * adamobjcnt
   * alldc+
   * users_disabled
   * users_nonexpiring
   * users_pwdnotreqd
   * users_accexpired
   * computers_disabled
   * computers_pwdnotreqd
   * computers_active
   * computers_inactive
   * schver
   * spn:xx
   * email:xx
   * site:xx
   * subnet:xx
   * syscrit
   * rodc_cachable
   * policies

Not really an issue or at least it shouldn’t be, if something is generating more than 50 concurrent NSPI connections, it likely needs to be checked out anyway…

NSPI connections from Microsoft Outlook to a Windows Server 2008-based domain controller may fail with an error code: “MAPI_E_LOGON_FAILED”

This behavior occurs because Windows Server 2008 only allows for a default maximum of 50 concurrent NSPI connections per user to any domain controller. Additional NSPI connections are rejected with a MAPI_E_LOGON_FAILED error code.

See the whole support KB here —> http://support.microsoft.com/kb/949469

My Grandma was shooting from the hip when I asked for this recipe

http://blog.joeware.net/2008/12/23/1554/

She found the “real” recipe and sent it along…

1 stick of oleo or butter
1/2 cup cocoa powder
2 cups of sugar
1 large can of carnation milk

Melt together oleo (or butter) & cocoa powder.
Add 2 cups sugar & a large can of carnation milk.
Cook at a slow boil about 10 min. or until thick. store in frig.
If you save jam or jelly jars & lids you can boil them turn upside down on towel to dry.
Pour hot fudge in & tighten lids will keep quite awhile.

My recent post about getting Active Directory replication Meta Data has spawned a considerable number of emails. I wanted to take a moment and remind everyone of the help switch specifically talking about the replication metadata info. This usage provided in AdFind tells you your sort options available for each metadata attribute and what info is contained in each of the columns.

You can get that help by typing

adfind /meta?

Here is the latest version of that output…

F:\Dev\Current\CPP\AdFind\Debug>adfind /meta?

AdFind V01.39.00cpp ***BETA*** Joe Richards (joe@joeware.net) January 2009

-help         Basic help.
-?            Basic help.
-??           Advanced/Expert help.
-????         Shortcut help.
-sc?          Shortcut help.
-meta?        Metadata help.

Usage:
AdFind [switches] [-b basedn] [-f filter] [attr list]

   basedn        RFC 2253 DN to base search from.
                 If no base specified, defaults to default NC.
   filter        RFC 2254 LDAP filter.
                 If no filter specified, defaults to objectclass=*.
   attr list     List of specific attributes to return, if nothing specified
                 returns ‘default’ attributes, aka * set.

  Switches: (designated by - or /)

   MetaData Help
   =============
   AdFind has the ability to decode various metadata type attributes. These
   attributes can give information about replication status of the server
   itself or replication metadata for individual objects.

   These special attributes are normally returned from Active Directory in
   XML format. This is a bit bulky and can be tough to read without cleanup
   so I have added the ability decode the attributes and cut down the amount
   of data passed over the wire. Using the ;binary option when specifying an
   attribute causes AD to reformat certain attributes and send them across as
   binary blocks of data. When requesting the meta attributes outlined below
   if you do not specify the ;binary option, they will be returned in the
   native format, if you add the ;binary option, they will be returned in the
   alternate format and AdFind will decode the strings to its format.

   To further assist the ease of retrieving this information, see the shortcut
   usage menu via adfind /sc?

   MetaData Attributes
   ——————-
   msDS-ReplQueueStatistics  - RootDSE attribute
       Replication queue statistics. Output is labeled. No sort options.

   msDS-ReplPendingOps - RootDSE attribute
       Any replications operations currently in progress. Output is labeled.
       Default sort order is server return order. Sort options - dsa,date

   msDS-ReplConnectionFailures - RootDSE attribute
   msDS-ReplLinkFailures - RootDSE attribute
       Replication failure information. Output is labeled. Default sort order
       is by DSA. Sort options - dsa,date

   msDS-ReplAllInboundNeighbors - RootDSE attribute
   msDS-ReplAllOutboundNeighbors - RootDSE attribute
       Replication info for all direct neighbors. Output is labeled. Default
       sort order is by DSA. Sort options - dsa,date,nc,err

   msDS-TopQuotaUsage - RootDSE attribute
       Indicates the top object owners on a given server. Output is labeled.
       Default sort order is server return order. Sort options - nc,owner.

   msDS-NCReplInboundNeighbors - Naming Context attribute
   msDS-NCReplOutboundNeighbors - Naming Context attribute
       Replication for all direct neighbors for the specific NC. Output is
       labeled. Default sort order is by DSA. Sort options - dsa,date,nc,err

   msDS-NCReplCursors - Naming Context attribute
       Replication cursors by DSA by context. Output format:
            HighestUSN LastSyncTime DSA
       Default sort order is last sync time. Sort options - lastsync,dsa

   msDS-ReplAttributeMetaData - Object Level attribute
       Replication metadata for object. Output format:
             USNLocal DSA USNOrig Date/Time Version Attribute
       Default sort order is attribute. Sort options - DSA,date,usnloc,usnorig,ver

   msDS-ReplValueMetaData - Object Level attribute (FFL2+ only - i.e. LVR Replication)
       Replication value metadata for object. Output format:
             attribute USNLocal DSA USNOrig Date/Time Version State ObjectDN
       Default sort order is date. Sort options - attrib,obj,DSA,state,date,usnloc,usnorig,ver

   Sort Options
   ————
   The decoded output for most of the metadata attributes can be sorted to various
   fields in the output. The specific fields for each attribute are listed with
   the description of the attributes. In order to change the sort field, use the
   -metasort switch. Specify the switch combined with the options specified above
   to change the sort order. If value has a dash (-) appended, the search order
   is reversed.

  Ex1:
    adfind -rootdse msDS-TopQuotaUsage;binary
      Get top 10 quota users in decoded format

  Ex2:
    adfind -b cn=someobject,ou=someou,dc=test,dc=loc -s base msDS-ReplAttributeMetaData;binary
      Get attribute metadata for specified object in decoded format

This software is Freeware. Use at your own risk.
I do not warrant this software to be fit for any purpose or use and
I do not guarantee that it will not damage or destroy your system.
Contact joe@joeware.net via email for licensing information to package
this utility in commercial products.

See full Warranty documentation on www.joeware.net.

If you have improvement ideas, bugs, or just wish to say Hi, I
receive email 24×7 and read it in a semi-regular timeframe.
You can usually find me at joe@joeware.net