“Software Companies need an Off-Shoring Strategy”
(or so they were saying)

In the start-up I co-founded, our CEO (and my partner) was pushing hard for an off-shoring strategy. This was the late ’90s/early 2000’s and the buzz in the software business was that the way to grow quickly and cheaply was using the Indian offshore companies. He would regularly send me emails saying “5 skilled programmers for $15/hour” whereas compared to the rates we were paying was a significant reduction. Reading further, there were additional management costs but but still seemed impressive. In addition, this was still during the dot.com boom and local talent was hard to find. (I’d remedied our lack of hires by partnering with a local Indian contracting firm who was providing us with developers where the contracting firm held their H-1 Visas).

Researching

I felt inherently that it would be difficult to build a team off-shore but had no concrete reasons for my reluctance so I attended a couple of seminars around offshoring.

• Many presenters gave facts and figures on the great cost savings. The Indian consulting firms described how cheaply they could provide CMMI Level 5 qualified work. (Since in my early years I was involved in the development and use of SEI Assessments, the predecessor of CMMI, I seriously doubted their Level 5 claims and believe it was more like ISO 9000 where everyone is trained about the steps – how to check in code, how to do code reviews – but not that the development and quality is superior).
  
  
• The large software firms spoke highly of their cost savings. Most had large facilities they owned, operated, and managed where they sent software for final testing or had customer support centers.
  
  
• One presenter discussed how to set up your own facility in India which included numerous lengthy trips to India and the mandatory bribes and dinners with various officials in order to get anything done.
  
  
• Another presentation covered how to set up a “lease-to-buy” program in India, where you’d partner with an Indian firm who would set up a facility, hire resources including management, work with you to train them with the goal of transferring the facility and resources to become employees of the company. That sounded the best but still involved a great deal of up-front negotiations and travel. Until the team is turned-over to your company, there is an overhead fee paid to the services company including equipment and facility costs plus service company fees.
  
  
• I met one person from a small company who swore that offshoring was the way to go. Digging deeper I found that the product expert for their company was located in Russia and he had hired, trained, and was managed their offshore development team from the company’s inception. Their main difficulty was how to train the U.S. Sales and Marketing people but solved it by having Sales and Marketing travel to Russia periodically. An entirely different situation than I was faced with.
  
  
• All presenters from small-to-mid-size companies discussed the high turn-over rates and large yearly increases the region was currently experiencing due to the big corporations, like Oracle, that paid more and that Indian engineers felt gave them better prestige.
  
  
• In addition, the presenters all had similar statistics regarding the number of managers needed per developer, additional cost for up-front design, and percent slower ramp-up of skills resulting in less productivity for longer duration.

I documented my findings regarding what it would cost our company to set up a reasonable off-shoring arrangement versus hiring locally. Being a start-up with very few people, none of us would have time to travel to India and do the full-time effort required. At our early start-up stage with only Angel Investors we did not have the capital to initially buy a building so I analyzed the “Lease-to-Buy” option with a three-year turn-over. Besides the actual developer hourly rates, costs include:

• A new local manager position (part-time)
• Significant travel budget
• Service company overhead and fees
• Local manager
• Average turn-over rates for India resulting in additional new hire travel, training and resulting loss of productivity during ramp-up
• Yearly increases
• Loss of productivity of the local team’s time that would need to be spent with remote conferencing, training, and travel
• Average productivity and quality figures

Needless to say, the yearly costs were nearly equivalent to hiring less developers locally and didn’t improve significantly after the group became employees, although at that time the potential to lower turn-over rates due to stock options and other benefits increased.

Conclusion

My CEO agreed. It was an invalid business proposition. We hired locally.

New Company – New Strategy

When our start-up was acquired, the acquiring company had an outsourcing strategy in-place for its existing product line. Their offshore team performed testing only. Because of the size of that product’s software platform, each remote tester required local hardware and software requiring extensive Build & Release personnel and policies and high-speed dedicated networking from Corporate to India. The company had decided that the city they’d chosen was not a good choice, suffering from high turnover but lack of local talent pool. Plus the infrastructure there was not sufficient.

They chose a new partner in Hyderabad for a Lease-to-Buy plan. Their goal was to move all development and maintenance to India with the exception of new technology development. I was given directions to develop an offshore team for our product line as well.

Prep Work

I hired our company’s prior VP Engineering as a consultant to help me with the initial 3 months effort. Together we put together the hiring plan. She coordinated updating and enhancing the Developer’s Handbook and other on-line documentation and development guidelines. She updated our software process to incorporate the offshore efforts and put together the training program and travel requirements. Fortunately we had a very good process (many of the tenants of “Agile” but not Scrum). We had good tools to aid with the process: Our proprietary tracking software (SD Tracker), PerForce for code check-in, and Telelogic’s DOORs Requirements Management system (since purchased by IBM). All three had web-based interfaces which was key for remote development. SD Tracker had ongoing comments section which greatly aided on-line “discussions”, capturing what was discussed and decisions made.

On-Boarding

We would start with only two developers, one tester and one manager who would also need to do hands-on development part-time. The other product line was much larger and had a significant hiring plan. The two product line teams, though, would remain independent since the code was very different. The hardest position to fill was the overseas manager. He/she would be key to the success of the project since he/she needed to be technically skilled to lead and train the new hires plus a good manager. I quickly learned that in India, the engineers do hands-on development work for only a few years then have the goal of becoming a manager. Managers there do not do hands-on development work. Finding someone willing to do both took a lot of convincing, but we were finally successful. The developers and tester were also hired and all four came to California for two months training. The manager had family in India and ended up having to return after six weeks. We were fortunate that he was extremely bright, picked up the basics and the deeper requirements, and although he still was a bit reluctant to go back to development, agreed to try for a year at least.

After the team returned to India, there was a focus on continued communication. One of the lead developers was given responsibility for managing the Indian team half-time (coding the other half). She traveled there and spent a month on-site. Upon her return she regularly scheduled nightly conference calls weekly or more as needed.

End-of-year Statistics

Regardless, ramp-up was slow for the developers. After one year I found the developers were only 20% to 30% up-to-speed whereas in one year local new hires would have been at least 50%. (The software was complex). I also tracked the time I was spending providing additional design documentation versus what would be required locally but also many questions that seemed to require a simple written answer ended up looping several days before it became clear or a conference call was needed.

After a year, one of the developers was deemed unable to perform and was moved to our services team to work on implementation code and a new developer hired and both developers were brought to Corporate for training/re-training. The manager was unable to travel due to personal reasons but had proven himself to have been a valuable hire. A few months earlier, at the end of our release cycle, his team was not able to complete some of their assignments and he jumped in and completed not only his own but any outstanding work. His code was reliable and high quality which is the only reason the release was successful. At that time I discovered that the services company was paying the manager the same low rate as the two developers (since he was also developing) instead of the manager rate I’d agreed to. I was furious and demanded they immediately provide him the raise and feared losing him so also spent time talking to him and making sure he was happy and letting him know how pleased we were with his efforts. Over the next few years the team was expanded slightly (3-4 developers, 2 testers) with the average amount of turn-over and salary increases. The manager remained.

Ongoing Travel, Communications, Travel, Communications

People were continually going to/from India. Any of our Indian engineers who traveled home yearly would take an extra few weeks to go to the site to work/train. All managers (including Directors, VPs) went there at least yearly. The CTO moved to Israel where he was responsible for all Indian operations. Plus yearly we would bring everyone to Corporate to work for a month or so.

The company implemented it’s “Buy” option and all of the offshore team became employees. Plus they have been raming up the size of the offshore team.

Changes Needed with Agile

Then the company introduced Agile. Initially they tried to use the DOORs Requirement System to capture Stories in a separate spec but that was not a good process. First, DOORs doesn’t have an easy way to prioritize and sort requirements so the Product Marketing team (who performed as Agile Product Owners (POs)) exported the Story doc to Excel then went back to update DOORs with the Story Points and other changes agreed to in team meetings. They still needed to update the actual product specs in DOORs and now the tasks in SD Tracker were linked to stories and specs. But the biggest issue was the difficulty in working with the offshore team starting with Stories. They were used to working from detailed specs and the added interaction between them and the POs caused significant time delay. The company smartly decided to hire POs in India to work locally with those teams. For the next release, the company moved to Rally for Agile Story Management. This was much better than DOORs and Spreadsheets since Rally had a prioritization tool and the features one would want to manage Agile Stories. But, unlike real product documents, the Agile Stories, though grouped in Epics, don’t maintain a specific logical readable order – they remain distinct stories. In order to still communicate the business basis, the POs first wrote extensive PRDs using Microsoft Word. Those would be attached to the Epic. The features within the PRDs became Stories in Agile. In order to provide better clarity, extensive Conditions of Satisfaction (COS) were added as validations. To accommodate this, a design sprint was added as Sprint 0. Of course, quickly after the project started, those stagnant Word documents became out-of-date and useless.

Both product line’s Product Management and Engineering Management and the VPs of both organizations spent several weeks in India to kick-off both new releases Sprint 0’s and to train/re-train. Having a local PO improved (but did not eliminate) communication issues since the Indian PO while very intelligent and eager, had a significant amount of information to learn before he could be effective. Local Product Marketing managers typically took 1-2 years to become effective and still didn’t know the breadth of the product after that time but had local product experts to interface with which the offshore PO did not. While I believe a local team could produce deliverable code in a sprint, the use of locally disperse teams requires the addiction of 3+ “Test Sprints” after code complete. This then is more of a hybrid model than strict Agile but seems to be working fairly well. The Product Line has two separate Agile teams including PO(s) and Scrum Master – one at Corporate; one in India.

Same Conclusion

Bottom line – I think this offshore team is as effective as anyone could hope an offshore team to be. A form of “Agile” is being performed, although it is currently not as effective as our prior Practical Software Methodology (an Agile-like process) with a single local team.

There is substantially no cost savings overall.

There is more work for the POs and development manager than interfacing daily with a local team plus ongoing travel costs and interruptions. Particularly when “Offshore” is half-way around the world and trips there require weeks due to travel time.

More effort to complete, more miscommunications, less product delivered.