IT Engineers and How FBL Works

In a previous post I briefly discussed FBL.  After that post went up I received quite a few emails asking me to write some more about the subject, so here it is.

FBL is an acronym used by IT engineers that stands for Flashing and Blinking Lights.  To engineers, that’s all it means.  Small, different colored lights that may or may not blink and may or may not indicate anything.  To management, however, it represents the opportunity to demonstrate to others the awesomeness of your data center and where this year's budget went.

IT people use a lot of acronyms.  They have one for just about anything.  RAM stands for Random Access Memory.  CPU stands for Central Processor Unit.  SRM stands for System Recovery manager.  RPO stands for Recovery Point Objective. BRCM stands for Break Room Coffee Machine.  LAN stands for Local Area Network.  LBGD stands for Large Box of Glazed Doughnuts. WAN stands for Wide Area Network.  And of course you already know about FBL.  You get the picture.  There’s an acronym for almost everything.

You might ask why IT people have so many acronyms.  No one really knows for sure, but here’s my theory:  In the early days of IT, making a computer work was like a black art.  No one really understood it fully, but there were a select few who could actually do it and they were soon considered the most valuable employees you could have on your staff.  If you had some IT people working for you, your company was progressive and was destined for fame and fortune.  We all enjoyed the status of being in IT because IT was every company’s secret weapon.  If your company had an IT department then it’s success was almost guaranteed.  To protect our status as gods of technology and keep interlopers from dethroning us, we had to develop our own exclusive and mysterious language.  From that moment on new acronyms were born every day.  That’s enough on the origin of acronyms.  Now back to FBL.

There are basically three different kinds of FBL.  Green FBL means life is good and everything’s probably working as it should.  If all your FBL is green, you probably won’t find your boss standing in your cube or calling you at home unless he/she wants you to work on your day off.  Your yearly performance appraisal will probably go well and you might get some extra money.  Green FBL is good.

Red FBL is a different story because even a small amount of FBL will make management absolutely crazy.  It usually (but not always) means something is not working right.  You will have a visit from your boss or maybe, if you’re lucky, just an email.  Either way, red FBL brings bad things into your life.  If you work for a large organization, FBL can result in more meetings.  You may have to go to meetings and explain why you have red FBL and what you’re going to do to get rid of it.  After the red FBL is gone, you may be summoned to more meetings to present your plan to make sure the red FBL doesn’t come back.  You will have to explain very technical things to people who will not understand anything you say.  It’s not that they aren’t smart enough to get it because they are smart people.  It’s because you are speaking Engineer and they can only understand Management.  Sometimes you can find a translator who can make them understand, but usually it’s just easier to say that the red FBL was caused by a transient error or that you have applied a software patch to make sure the red FBL doesn’t come back. Your goal is to end the meeting before any additional follow-up meetings are scheduled or new projects for you are identified. As a point of reference, these meetings remind me of a bunch of angry badgers fighting over a Cure 81 ham.

As engineers, we’re not too concerned about red FBL, but there are problems with that approach. Equipment manufacturers usually follow the "standard" definitions for FBL, but not always. Some manufacturers build their devices so that red FBL means everything is working properly. You don't want those in your data center and you should do everything in your power to keep them out. The reason for this is that, if management sees red FBL in the data center, you've got some "splainin" to do. You will get emails, text messages, phone calls, meeting requests, Twitter messages, and maybe even graffiti spray painted on your house asking why they see red lights. No amount of reassuring them will convince them that red FBL is normal. If you resist, they may hire outside consultants to come in and do an expensive and thorough evaluation of your data center and this can result in problems during your yearly performance appraisal.. It's much easier to just avoid buying anything that has red FBL when operating normally, no matter how badly you need it. You'll be much happier in the long run.

You might be asking "What about yellow FBL?" Yellow FBL is an entirely different story. As engineers, when we see yellow FBL, we know that we need to do something to prevent it from turning into red FBL. Yellow FBL almost always eventually turns into red FBL. I think this has something to do with red and yellow's relative positions on the color wheel, but I'm not really sure.

Management sees yellow FBL as a personal challenge. The risk takers see yellow FBL as a dare from a higher being to delay spending any money on correctional measures until the next budget cycle. They may ask engineers to take a guess at the odds of yellow FBL turning into red FBL before the next fiscal year, especially if they have a bonus riding on their ability to follow this year's budget. Don't get involved! If you are wrong, you will be pay a hefty price for your good intentions. This is the time to provide vague and mostly meaningless advice that no one can hold you accountable for. If your management is risk-averse, they will immediately redirect the entire staff's efforts to anything that will turn the yellow FBL back to green. Problem solved.

I mentioned earlier that as engineers we usually aren't overly concerned with FBL as we go about our daily activities. Despite what management thinks, it is possible to have red FBL when everything is working perfectly. As engineers we know this to be true. Sometimes red FBL doesn't necessarily mean we have to fix anything. It sometimes means that we just need to figure out how to make the device change its red FBL to green FBL. This can be as simple as issuing a software command that acknowledges the red FBL. Boom. Problem solved. Be careful with this, however. If there is actually something wrong, the device may turn back to red FBL after you've left for the day and you'll have to go back to work. It is usually better to just deal with red FBL and get it over with.

So that's how FBL works.