Anup Surendran's blog

This is a follow up post to my previous one about Performance.

Scaling in technology lingo is - " The ability for your service to continue to serve at the same level of  service quality to multiple clients or requests as demands increase". That might be a simplistic explanation. Wikipedia explains it like this - The ability for a business or technology to accept increased volume without impacting the contribution margin (= revenue - variable costs). For example, a given piece of equipment may have capacity from 1-1000 users, and beyond 1000 users, additional equipment is needed or performance will decline (variable costs will increase and reduce contribution margin).

It is interesting to note some technology lessons learned from Google's technical architecture . Here are the highlights :

- Infrastructure can be a competitive advantage.

- Spanning multiple data centers is still an unsolved problem.

- Take a look at Hadoop

- Synergy isn't always crap. By making all parts of a system work together an improvement in one helps them all. Improve the file system and everyone benefits immediately and transparently. If every project uses a different file system then there's no continual incremental improvement across the entire stack.

- Build self-managing systems that work without having to take the system down. This allows you to more easily rebalance resources across servers, add more capacity dynamically, bring machines off line, and gracefully handle upgrades.

- Create a Darwinian infrastructure. Perform time consuming operation in parallel and take the winner.

- Don't ignore the Academy. Academia has a lot of good ideas that don't get translated into production environments. Most of what Google has done has prior art, just not prior large scale deployment.

-Consider compression. Compression is a good option when you have a lot of CPU to throw around and limited IO.

All these will show up in some form or fashion when you consider scaling your technology services.

Now, let us talk about 10 things to keep in mind when scaling your business. These are the three things you have to keep in mind

1. Scale your most profitable business processes
2. If your business processes deal with customers, then find fanatics who can add that one bit extra to help your customers.
3. Assign the best people to your most profitable processes
4. When new people come on board make sure they learn from the best
5. Make sure the best communicators are kept in place for intra business process communications
6. Scaling processes requires documentation and measurement. KPI's should be reviewed and measured periodically
7. Find the optimal productivity of your key personnel. You want them to run a marathon, not a 100 metre dash every day
8. Ask people in operations for feedback to make their job better.  Ask customers for feedback. See if the feedback can help you scale better.
9. Create a Darwinian approach to scaling business processes. Let the best business process win
10. Consider packaging work items differently for efficient human processing
11. Make sure your first and last interaction with the client has the fanatics involved