Breaking Performance Silos
Author by Alex Podelko
It is amazing that, while performance is the result of every aspect of the system, performance as a discipline consists of multiple silos and different groups of performance-related specialists have very limited communication between themselves.
You hardly find any book or community covering performance end-to-end. Performance-related responsibilities are usually spread across several groups and rarely is there a person responsible for end-to-end performance throughout the whole lifecycle of the system. Ian Molyneaux’s post The Case for the CPO has the point – but how far it is from the reality. Many performance experts I talked to do believe that it should be a person responsible for performance – but you hardly see it anywhere, not to mention CPO.
Moreover, the current devops trend should amplify that need in the unified view of performance – devops, in essence, is about breaking down silos. And performance is the first area where silos should be broken down – but it looks like performance somehow was left out. Not much happened with performance silos and it looks like the devops process happens somehow in parallel.
Development of performance as a discipline definitely have cycles - and it is a pity that next cycles start from a scratch and forget about what was done before. Yes, traditionally performance engineering was mainly concentrated on the back-end because the front-end was trivial and didn't impact scalability. Yes, now the front-end is far from trivial and deserves the attention it has now. But the back-end and scalability issues haven't disappeared and they deserve their share of attention, too. I like Andy Hawkes' post When 80/20 Becomes 20/80 bringing back some balance. Every performance specialist should understand Performance vs. Scalability and, even being a specialist in a specific area, know about other performance-related areas and main issues and methods associated with them.
Application Performance Management (or Monitoring, APM) tools are, in a way, supposed to break these silos down providing end-to-end visibility into the system. Even leaving aside the discussion how these tools live up to their promise to provide full visibility across all tiers with minimal overhead, most silos still remain. We still may have different environments (development, test, production) and APM views of each one may be not completely compatible due to differences in configuration. APM also assumes that we have people who may work with the information provided by the tools across all tiers and work with all teams to ensure a system's performance. Whatever issues were identified by (or with help of) APM, with the exception of trivial cases when they may be handled automatically (for example, auto-scaling), they need to be addressed by professionals as part of the performance engineering efforts (even if they are not officially named so). Meanwhile, we practically have no across-the-silos performance professionals, nor do we have positions in organization org charts for such professionals.
Moreover, with APM, we base our decisions on the current information. While theoretically it is possible to do some modeling and what-if scenarios based on the current situation (although, to my surprise, it doesn't look like any APM vendor has it beyond trivial or has much interest in it), modeling has its limitations and provides rather the best case scenario. Only load testing allows one to assure that the system would handle the expected load (assuming, of course, that it is done properly).
Velocity is probably the most popular performance-related conference for the moment, but it actually limits itself to a few performance areas. It is centered around front-end and web operations. Some scalable architecture patterns are discussed (mainly when backend processing may be parallelized). Other topics, such as load testing, APM, capacity planning, are mainly ignored. All these topics still look relevant to web operations, especially if we speak about e-commerce. And it looks like traditional corporations with all their performance problems don't exist anymore. Well, I suppose focusing on web operations is intentional – but it isn't a comprehensive view of performance. What I am concerned about here is that with all Velocity’s popularity, it may make the impression that the topics that are not covered are not important – and I believe that it may harm the industry.
For those who are really interested in performance, I'd suggest to attend Performance and Capacity 2015 conference by CMG, which is probably the only vendor-neutral conference that is covering most aspects of performance and this year’s program is taking shape and looks great so far. Unfortunately, it is rather small now and its impact on the performance community is rather limited. Unfortunately for the performance community – for attendees it is probably rather an advantage when you may easily reach out to renowned performance gurus in a cozy environment. I don't position it as an alternative to Velocity – front-end coverage is rather light for those who specialize in it – but rather as a perfect complement and perhaps a better choice for those who do not specialize in front-end performance.