Friday, January 2, 2009

How I Didn't Get in Trouble

A few months ago I noticed a significant performance problem during the application startup of the really big, web system I work on.  It was literally taking up to a minute and a half for IIS to load up the application.

After some digging I found out that the application was loading about a million records or so from the database into the application's cache.  Regardless of the decision to implement this, I did not want this happening locally.  I couldn't test any of my work.  I couldn't develop.

Where I work, even though we all have a local development environment, we all share the same development database server.  And these million or so records were being loaded by a stored procedure aptly named spGetAll[Things]. 

I notified my boss and the developer responsible for creating this stored procedure about my problem and told them that I was practically unable to work and that we had to change this behavior in our development environment.  They said that it would be done...  And I waited... and waited...

And after a day and a half, I modified the stored procedure to return only the top 20 rows - problem solved!  But I knew that this could cause problems for someone else who might be debugging a problem related to this data and find that the data was missing. 

So I waited, expecting someone to yell about the missing data and then at me.  But I could work again.  Other's could work again.  Adding "Top 20" to a stored procedure on a development database saved countless minutes and probably a lot of money for the company. 

Finally the email arrived.  "There is a problem with the data in such and such table and we're going to overwrite it with what's on production.  Is this OK?"  So I humbly sent out an email saying that I modified the spGetAll[Things] and that maybe this is the problem.

Here I am, thinking that I'm going to get in trouble, that some poor developer spent hours or even days trying to track down a problem that I intentionally created.  And what happens?  They apologize to ME.  OMG.  They said that they were sorry but they'd have to fix the stored procedure. 

"Ok" by me.  They fixed the stored procedure.  And I quickly add a configuration variable to use a new stored procedure spGetAll[Things]Truncated.  I probably should have done this in the first place but adding "Top 20" is just so much simpler.  Especially when it's done under the radar.

The moral:  if I didn't fess-up to what I'd done I could have gotten in real trouble... and maybe even fired.  Don't be ashamed of your work.  Even when it's bad.  There is no way to work in a team if you hide what you've done.  And you'll see that people are very forgiving when you admit to making a mistake.