Tom Peruzzi's thoughts on digital, innovation, IT and operations

the knowing-doing gap

Posted in general failures, organizational by opstakes on February 8, 2013

Bildschirmfoto 2013-02-08 um 08.40.36There is a great book out there from Pfeffer and Sutton writing about the difference in knowing and doing. Even if you know it will not become reality you often start doing something in that direction. Even if you know this brings you into a dangerous situation, you keep on running. It is scary, honestly but human too.

The same often happens when talking about operations, QA and development. Some examples: Even if developers know that untested code often results in bad behaviour, stress and pain, the code gets deployed. Even if operators know that they should provide a service (the infrastructure and related daemons) they much more try to build barriers to “ensure stability”. Even if QA knows that controlling isn’t enough it is much easier to tell somebody that he did not reach the line instead of helping him/her coming there. We are all obliged to support the company we work for. If the company is run by idiots, please leave otherwise do your work as you promised with your signature.

We all live in a time where time becomes an evident issue, pressure is high, likelihood to fail too, but we are no longer allowed to fail even if a lot of enterpreneural books write about fail-learn-improve cycles. Often this happens as we react not like we know we should react. Business pressure, politics, friendships and sometimes the missing willing to change make us to some sort of organisational animal, no longer reacting rational in a human manner, much more rational in a corporate manner.

But as we are all obliged to Quality, as it is our wish to product high quality products/services, as achieving mastery is one if the fundamental personal goals of “running” we should try to bring in more evidence, react on what we know and not what best fits know and stop doing if we recognize the difference in knowing and doing. Another reason why this is that hard: the organisational culture has to support such behaviour. Saying “no” or “we must change …” has nothing to do with blaming, it’s a positive escalation showing new ways of working and thinking and propably (hopefully) better ones.  This has nothing to do with basic democracy, everyone should bring in his/her expertise and evidence to allow the company grow and prosper – knowing without doing isn’t enough!


Operations Strategy

Posted in general failures by opstakes on December 29, 2012

to operate or not to operate. If you believe in Henry Mintzberg (2003) saying that strategy has a lot to do with craftmanship and either deliberates or  emerges from the bottom then you potentially should use operaitons as one source of inspiration!

I do not want to talk about DevOps, NoOps or whatever Ops, the serious question is whether operations can participate in collective and corporate learning and generate benefit for an organisation or not, whether it creates some strategic extra or it is only a matter of costs. In the past a lot of people wanted us believe that operations is dying, becoming part of a museum, being exchanged with smart scripts, recipes, a lt of logic and automation and and and. The funny thing on the story: a lot of those ideas came from developers who – in the past – alsways had severe problems with operations for different reasons. And as development is a creative process being very near to products it often has a better internal voice than operations. How often did I hear that operations does not produce any strategic value to product X or servicy Y?! I cannot count that number, it is pretty to large I guess.

But this brought me to my story. Is ops really just about money? Is it the department socialiszing the developers ( 😛 )? Is it the steady continuuum of delivering a service? Or is it the bow before me for I am root fraction not worth sitting in an upper floor?

It is a strategic question which has to be answered. If you think in raw numbers operations can be outsourced, if you think in ressoruces, it can be outsources too, if you think in value and operations as a service than it potentially should stay in. The reason why? The more operations offers an infrastructure + processes as a service to its customers, the more it concentrates on its specific know-how and capabilities, the more it becomes a strategic asset. Yes, this is a long way to go and no, you cannot keep on acting the way you did the last 10 years. A good Ops manager must become a product manager of the corporate infrastructure, he/she must sell it to its customers, he must have passion in serving those needs. If so – I bet  – there will be no discussion for outsourcing or not, because that discussion will be driven by the ops manager to be able to deliver the best infrastructure available!

So please, dear Mr. Ops Manager, make you live easy and become a major part of the service chain. Go that way, even if it is the harder one, but this will keep you, your team and operations vivid and valid for the next period of time.

Ops Predictions 2011

Posted in general failures, ITSM, organizational, startup failures by opstakes on December 21, 2010

End of year is coming, time for review and predictions …

What we have seen this year is the emerging trend to try to move to the cloud. Why say try? Cause a lot of different lacks did delay decisions: lack of experience, lack of manageability, lack of security, lack of commodity, lack of portability and much more but the train cannot be stopped anymore. We will continue to see different diverse ways to the cloud, the aggressive one (we just do it), the one’s moving via private virtualization, the one’s doing outtasking to the cloud and the one’s not knowing that they are already in the cloud.

So what’s next? According to the analysts cloud is directly on the way to the phase of desilusion. Sounds bad but isn’t so. We now reach the working scene, the marketing whow is over and we can start working on a deep and permanent way. So think about it: cloud will become commodity in 2011, we will stop talking about who’s in the cloud or not, we will start just using it.

This leads to another trend for 2011: cloud operations. We did central operations, decentral operations, virtual operations, outsourced operations, outtasking and whatever, next is cloud operations. Maybe you will not take care on it but potentially you will have to think about how to operate your IT then parts of your IT are somewhere (you do not even know exactly the location, just the name/identifier of the cloud).

This leads to tons of aspects in terms of all ITSM processes, especially change mgmt (do you still own your cloud virtual environment … how to combine those releases …), incident, event and problem mgmt. (who manages what?), SL management and all others, with special focus on IT financials.

Next trend, partly invoked by ideas like DevOps is agile operations. The more agile the company, the more agile development the more event driven the IT. This leads to agile operations for the IT ops department. So how to do so?

Agility means being very flexible and self responsible within a certain frame/border. Agile operations mean being very reactive, fast and flexible within a fixed set of frameworks/standards to deliver prompt IT resources on a very $$based approach.

So agile operations relies on cloud operations and vice versa. In my understanding and strong believe the trend per se for 2011 should then be called

agile Ops operations

So what does this mean for you? Think about strong boundaries and frameworks married with a high level of ops automation. This superset is then offered to your company / development enabling them to use ops resources on demand and cost sensitive. You as the ops entity do all the cloud stuff either private, hybrid or public within your defined subset to deliver on a regular and flexible bases predictable IT.
For me this sounds reasonable good. Remember, I’m an ops man … doing agile ops operations even means you create your ops platform (DevOps), you keep the releases within your responsibility but you stop from reacting and being the holy grail nobody knows about within your company. Ops get’s public, viable and business enabling to the company! This is our all time goal and this must be the goal for all of us.

We will see what happens exactly in 2011, hopefully my predictions comes to truth by 80 %.

all time cloud

Posted in general failures, KnowHow, startup failures, technical by opstakes on July 20, 2010

I stop writing about operational failures only, potentially this blog will go on to write about “hot” topics within IT operations. It will still stay focused on operations, as I am an ops guy.

Good reason for telling, why I stopped writing for the last weeks: I wondered where cloud computing will go to!

We do quite a lot of different cloud projects and right now it seems that either there is no space left to deal with clouds or on the other hand side there is still a lack of experience out there on all sides of business. This is just a short draft about my ongoing thoughts, discussion welcome!

Cloud topics on business side:

why do we still believe, that it is as easy as writtin in the prospect? Haven’t we learned from all former proposed functionality? Yepp there is high potential to get it done and delivered in a smarter and more cost sensitive way, but at what risk and cost? And how does operations look like afterwards?

Cloud topics on IT side:

Clouds is nothing we can pass by. Clouds have to be worked with, IT has to understand pros and cons of clouds and how to live with them for the next decade. Clouds are neither friends nor enemies, they are a new way of delivering services to customers, more service based than ever before. Clouds are not VMWare and are not xen or kvm, clouds are a business case thou IT has to understand business and business methodology otherwise they will deliver virtualization. Not bad at all but only a few percent of cloud power.

As IT I would strongly recommend not to put to much pressure on compliance, legal and data security. There exist several organisations covering that topic and it is max. a question of weeks or months to get it fully done. Secondly there are already SAS70 ready solutions out there and other standards are met too, if you cover that topic it is OK, cause it is a risk, but nothing more. Using compliance as IT against cloud will mark you as the one securing your own office place …

Cloud topics on operations side:

clouds mean to no longer be the prime operations partner. To be honest, then thinking about all the complexity getting more and more clouds can even help you reducing YOUR complexity and getting things done. Yeah, number of systems will potentially go down, lot will be delivered out of cloud, partly you will act as an cloud offer. BUT, this is good news, you can transform yourself from an 24/7 operator to a platform architect handling tons of tons of tons of different systems without dealing with the day2day problems, they are within the cloud handeled by others 🙂

Tagged with: ,

Our own datacenter is the best

Posted in financial, general failures, technical by opstakes on December 8, 2009

Potentially no, but there is still a sub reality out there showing IT Operators or Facility Managers that owning their own datacenter is much cheaper, cooler, greaner, leaner, and whatever even if they just want to run 50 servers.

To be honest it could make sense, depending on your location, some geographic stuff and depending on the growth. But even google did not start by building their own datacenters. They rented before and now step by step they migrate to their owns – because they have reached a size of interest for having their own ones!

And if you think about your datacenter think about:

  • who’s operating the facility
  • who keeps care on USV, Diesel …
  • who keeps care on getting all licence stuff done?
  • who cleans the filters
  • who is responsible for CCU & friends
  • who is the cabling expert?
  • who is the power expert?
  • who is responsible for rack planing and provisioning …

and all of that in a twenty-four-seven environment …. ? Do you really want to be the Facility guy PLUS the Ops Leader? Can you combine those two or run them in a professional manner? Will you get headcounts for an electric expert as an IT Ops Leader? Tons of questions and nearly all – especially if you think about associated risks – should lead to the decision to NOT rund your own DC before you reached critical size.

The next interesting topic which always happens: If we still do not have the critical size, why don’t we do some shared hosting too? Because you do not have the skillset to do this? Because you are an internal service unit and you are not set up to offer on market prize on the external market? Because your SLAs are not that strong? Not really, potentially no, the main answer is: Because it is not your business!

So when do you reach the critical size? It does not depend on the number of systems, it depends on strategic and economic questions

  • is running own datacenters a potential USP and can I run it cheaper/equal market?
  • Sum of mass discount smaller than savings by own DC (economic value)
  • extra flexibility needed (keep care! flex against price, and flex. against coolness)

The how and when will vary and potentially we will have to work on a new definition of critical size with regards to cloud computing, the price modell and the new datacenters (generation IV) which should reduce costs too.

To cloud or not to cloud …

Posted in startup failures, technical by opstakes on November 12, 2009

…. that’s the question

I am not intended to talk about the general facts about clouds, cost topics and the different types (PaaS, IaaS, …) And I’m too not interested in referring about whether an offer is SaaS or Cloud or not. My topic is much more operational and what I want to show you is that a cloud as a private or public cloud does not solve all your problems as you potentially intended to do so.

If talking about PaaS/IaaS a cloud is nothing more than another type of infrastructure provisioning, nothing more, nothing less. You still lack support for your application and in terms of public clouds you will not be enabled that easy to run a loadbalancer or other resilient stuff that easy.

Despite the fact that all major cloud computing offers try to declare their cloud as save, unlimited, borderless resilient and (d)dos-attack safe, we still know that Murphy exists. Talking to vendors today talking about resilience mostly ends up by them telling ungecky sentences like “a cloud can’t fail …”

So if you are aware of potential risks and if you already know how to deploy your app(s) in your favorite inhouse or external cloud you should still think about how operations changes by using a cloud? Be aware of topics like:

  • how about backup/archiv?
  • am I still able to fulfill all my current and foreseeable compliance stuff and how?
  • how will my release process, my associated toolbox and my service support process change?
  • Is it a strategic or an economic value and how to live with?
  • Is my ops platform able to run in the cloud?
  • Is there any benefit from using a cloud or is my app still missing some major advantages?
  • Do my vendors and their license support their app in the cloud? Is the licence cloud-enabled?
  • Do I need special hardware?

The main topic I see right now is that we all talk about how cool it would be and how nice and easy everything should look like in the cloud but I will only talk about Operations and there are still a bunch of unanswered questions. So please don’t say yes or no to a cloud because of style or your personal relation to the vendor, think about the questions above and if you can easily answer all those you should really think about running a cloud.

2 people are a NOC

Posted in organizational, startup failures by opstakes on November 10, 2009

Despite the fact that people cost money you will never be able to run a commercial 24/7 site with just 2 people in a secure and safe 24/7 way. On the one hand side you burn your employeees on the other hand side you will potentially break local law regarding workers rights.

And, to be honest, If you want to run your application you have to think about what the work of an NOC will be, is it?

  • monitoring, remediation
  • reporting, escalations

or even?

  • application support
  • ops tool maintainance
  • tasks like backup, rollback
  • qa topics?

the more you think about the more you will come to the conclusion that an active NOC can be a major advantage for your organisation and business. So, if it is not built as a technical Call Center but moch more as the name claims an operations center, than you will gain major advantages. But his means, that you need a structure and the right people, not 2 potentially not hundreds. And you need time, a working NOC is not a matter of a bunch of definitions and nice mission statements. You need role separation (dev, sys engineering, ops, noc), technical clarification and setup of the NOC itself, including processes, space, people and resources.

So what does this mean for startups?

You should potentially think about a shared NOC or think about when the right point will be for thinking about a NOC. And believe me, there will be long time no potential need for getting such an or structure up and running. Try to work based on OnCall procedures as long as possible. NOC costs money, even if there is major benefit. And a NOC requires working structures and procedures. So only start building a NOC if you are already aware of processes.

%d bloggers like this: