Follow @RoyOsherove on Twitter

Learning from mistakes: the Don Quixote pattern

Experience is built on mistakes. in the near future I'll try to describes some of the biggest mistakes I've made during my (short)career and the lessons that I learned from them.

In this post I'll tell you about one of the most embarrassing moments in my career. A moment which holds a lot of frustration for me, even today, but one which I have learned a good lesson from for the future.

Don Quixote Vs. ADSI & WMI

I had just started at my new job (this was quite some time ago and a few jobs back) as one of the .Net developers in a small team of developers at a medium sized company. I was enthusiastic and excited to start there and was just waiting for the moment in which I would be asked to perform my first "real" task, at which point I would "prove myself" to the others. My team leader was very nice and soon the time came for me to get a task.

My task was to create a component that searches through specific machines on the network and enumerates all the physical files under all the virtual directories in IIS for a specific machine. They already had some .Net code that used the System.DirectoryServices namespace but they had not figured out how to enumerate all the directories, as it has been discovered that some directories, although physically available, are not discoverable using DirectoryServices under certain conditions. My mission was to come up with a way to find those directories that the initial "crawling" had not found.

This was not an area I had any knowledge in. Very quickly I started to dive into the various articles that explained DirectoryServices and finding out the reasons why some directories were not showing up. It turned out there was no way to expose these directories using DirectoryServices. I had to find another way to accomplish this task.

If I remember correctly no initial time estimation was expected of me. After the couple of days were over I went to my team leader and said that I am still investigating the various ways in which this would be possible to do, but that "All in all, it seems to be a fairly complex solution". I then went back and sank knee deep in a mountain of papers and articles on Active directory(ADSI). All the while my teammates were working on other stuff. When asked "what's up" by them I would usually put up a long face expressing "If you knew what I was going up against you would understand why I'm shutting up most of the time" but overall I got to know them and became friends with them.

Initially I had a small thought in my head "I wish you could use the DirectoryInfo class to search remote machines" but I had very quickly dropped this thought and never even bothered to check whether this was possible. It just didn't seem possible. I continued on to other areas. During my quest into ADSI I also discovered some articles on WMI- windows management instrumentation, and the capabilities of this technology. As the days went by it became clear to me that WMI was the way I was looking for. It had the ability to query a remote machine for a physical directory structure, and combining a search using DirectoryServices and WMI seemed to be the way out of this mess.

During these days (about a week and a half) I went a couple of times to my team leader to update on the current situation. I was told "not right now, I have other things but I'll get to it later"  and I didn't really fight to make those meetings happen too much. I was secretly glad at the opportunity to be left alone with my thoughts with no one to bother me with various questions about how I'm doing. After all, I was not doing so good. I was actually stuck and I didn't feel so good about my WMI solution. It felt out of place and too awkward but i didn't see any other way and felt that I was stuck with it. 

I was nearing the middle of my second week working on this problem when I let my team leader know of my proposed solution using WMI and ADSI. I got the go ahead to continue and went ahead and implemented my solution. It took another week or so for my solution to be ready. It worked slowly and made terrible hard drive noises when searching remote paths on network machines (Up to now I don't know how that noise was generated but the hard drives were going crazy). It was taking about 2 minutes to enumerate directories on a remote machine and it actually sounded painful.

My teammates, who were aware of my efforts and were part of my "testing group" were there all along trying to keep out of my way as I plowed through various technology boundaries to create what was at the end,one of the most horrible sounding physical searches in application history. Remember those old pin head printers that would squeal with each line they printed on large papers with little holes on both sides of it? It sounded a lot like that.

After showing this to my team leader and getting a fairly lukewarm response I headed home for the weekend. When the start of next week came I was seek and when I came back two days into the week my team leader asked to see me in her office.After I sat down she said:

"While you were away Me and another person from the team were talking about your work and we were wondering if there wasn't a better way to solve the problem than the way you solved it. So, when the week started we sat together and found a different way. It took us about 2 hours. It is simpler, it takes about one tenth of the time your solution takes and it's totally quiet." 

Needless to say I was shocked to hear this. There I was, slaving away at a solution for almost 3 weeks, and they go and find a much better solution in 2 hours without even telling me. When I asked for the details it was indeed a much more elegant solution than then one I had created. It used the System.IO.DirectoryInfo class (The one I disregarded at first) to iterate over the directories over the remote machine using a public $C share on each machine. It was so simple yet I had missed it and went on to battle windmills.

Less than a month later I was fired from that job. The cause for the firing wasn't stated but it would be a good guess that I didn't "prove myself", but rather, I "proved them wrong" about my abilities. It was horrible. I was devastated. How terrible a blow can your ego suffer? I was brought there as a knowledgeable person in .Net, and I was able, in one fell swoop, to wipe off any good impression they had of me.

It took me quite a while to get over this, but there is a very important lesson to be learned here:

  • Lack of communication can lead to a devastating failure.

I did not communicate with my team leader.

I did not share my problems and my continually bad estimations. Had I shared my problems from the beginning in a daily fashion ,together we would probably have come to the same solution that was achieved when i was not there. My fear of exposing my weaknesses had left me even more vulnerable. Had I just shared my problem I might have even come to the conclusion that using the DirectoryInfo class was something that should be tested before disregarded as impossible.

My team leader did not communicate with me

Over 2 and a half weeks, not once did my team leader come in and check in on the actual state of the task, asked questions, tried to help in any way, or shown any sign of monitoring and risk management. Had I been asked daily what problems was I facing in my task, it might have been obvious that I had simply "missed" a small turn on the way to the solution, which lead to a much longer way. Sometimes you just need a second perspective and another clear mind next to you to make sure you don't lose your way. Also, if my team leader was so convinced that my solution could be done better, why not approach me with this questions? why go to other team mates? It almost felt as if  was not being trusted and it hurt. A team leader should be open with her subordinates. Why did it take 3 weeks for this question to come up?


In summery

As developers, we all make mistakes. It's the natural and human thing you can always count on. As a developer, it is your responsibility to make sure that at any given time, your team lead knows what you are facing and if you're having any problems. If you don't communicate this well using scheduled short meetings or even hall small talk, you're not giving your team lead a chance to help you. You're also setting yourself up to be the scapegoat if anything goes wrong since you are the only person that could have known and fixed any problems you are encountering.

As a team lead it is your responsibility to make sure that at any given time you know what every person on your team is working on ,if they are having any problems doing it and how long it is supposed to take. You should do at least a daily morning meeting with your team to review today's objective for each person, to leverage any problems anyone might have and to basically make sure that everyone is on the same page. Forget to do this basic thing and you're setting yourself up for big surprises down the line. Doing code reviews is also a great way to keep tabs on what's going and the quality of it. Harder to do, this should be done on weekly basis, or milestone basis (some even do it before any code check-in of any developer). forget to do this and you could wind up with code who's quality you are unsure of.

Learning from mistakes: the Don Quixote pattern

Using SOA on existing N-tier applications