Varghese Abraham, Director - Project Management Office, New York City Department of Transportation
Everyone wants to hop on the bandwagon of Open Source application development for the most obvious reason, eliminating commercial software licensing costs. However, an organization’s decision-makers need to consider a much more complicated set of Open Source advantages and disadvantages before making a commitment on its development strategy.
At the start, it seems there are more questions than answers. What are the constraints local government agencies have in embracing the Open Source culture? How do you prepare an organization to move from proprietary, vendor-supported software development practices to a non-commercial, community-based environment? Why should a municipal government agency invest in a new technology practice that many consider risky, and that needs significant upfront investment in new computing environments and training? What are the right Open Source frameworks, code, and systems supported by a robust development community?
This discussion tries to answer these questions by describing one initiative undertaken by the New York City Department of Transportation (NYC DOT). We successfully ventured into Open Source development practices for a major IT system replacement moving from mainframe to web.
Business and Technical Transformation
NYC DOT’s mission is to provide for the safe, efficient, and environmentally responsible movement of people and goods in the City of New York. Traffic signs on the City’s street blocks and intersections collectively define the traffic regulations, and provide information about speed limits, driving direction, parking, and other guidance to motorists and pedestrians. NYC DOT recently replaced its long-serving mainframe traffic sign order management system after more than three decades of use. We replaced it with a new custom developed, Open Source web application: the Sign Information Management System (SIMS). Nearly 11.5 million records were migrated to SIMS, and 100 Android tablets were deployed to field staff for real-time visibility and tracking of work progress. This eliminates paper-based work order tracking and delayed data entry.
This was the first major enterprise-level Open Source IT system implemented at NYC DOT.
Change is hard. First, you need a champion who will stand by you to support change, who is willing to take risks for the greater good of the organization, and who is determined to push the enterprise beyond the comfort zone of proprietary systems and vendor lock-ins. Cordell Schachter, the Chief Technology Officer of NYC DOT, was this project’s champion. With experience in government and corporate IT project development, Cordell has charged our IT organization to diversify technology investments, look for opportunities to use Open Source technology, all to avoid vendor lock-ins to increase leverage in contract negotiations. Taxpayers are better served when government organizations have a wide list of technology options for larger projects federated across multiple technology stacks.
In our case, what started as implementing a common Open Source asset management platform ended up in a new branch of codebase specifically customized for our client workflows
Second, you need a team ready to take responsibility for systems built on new technology. A key to our project’s success has been the oversight by the staff on the in-house Open Source DevOps team who supported the most challenging aspects of project execution and now supports the system in production. The DevOps team was formed by the IT & Telecom’s Project Management Office (PMO) that I lead at NYC DOT.
There was skepticism within our organization about using the new technology. Like most municipal government agencies, NYC DOT has significant IT investments involving proprietary vendor technologies. The majority of our customer-facing transactional applications utilize those platforms. Our IT team is trained in those technologies. Introducing new Open Source application frameworks meant that we needed to build in-house expertise to support them and not rely just on vendors. We added Open Source development team members, and provided training opportunities. All were ready to take responsibility for the operation of the system at launch, and were important participants in the Go-Live readiness process.
Third, your government purchasing process needs to be open to procuring Open Source and other non-traditional technologies. We conducted a Request for Proposal (RFP) process. The evaluation team selected an established vendor, Cambridge Systematics, with expertise in building Open Source asset management and work order tracking solutions. We executed a Fixed Firm Price (FFP) contract and used Agile product development.
Look before you Leap
NYC DOT began considering Open Source development a few years back, creating a handful of public-facing information sharing websites. We learned through that effort that it’s difficult to re-use another organization’s Open Source code.
In addition to being hard, implementing change is costly. Open Source doesn’t mean the code or the development effort is free of cost, or that there is a significant cost advantage over proprietary software development. The overall development cost for the first-time implementation of most Open Source application stacks could be higher than similar conventional development.
It’s costly to be an Open Source good contributor. Giving back to the community with high quality reusable code components can be very difficult for project teams working on tight budgets and deadlines. Preparing, polishing, sanitizing, and maintaining a codebase to publish to the community takes time and effort. In our case, what started as implementing a common Open Source asset management platform ended up needing a new branch of codebase specifically customized for our operational workflows.
Open Source communities may not be able to provide support when you need it the most. For complex development scenarios and unique support needs, the community might not be able to respond as required by your schedule. This is where you need to plan for established support services, in the areas where you lack in-house expertise.
Start Strong and Finish Well
Introducing Open Source with a major project will help to get the organization’s attention. You need to be open and flexible to accept changes throughout the development lifecycle. We started the process that lead to our selection of an Open Source solution in 2012, planning for cloud-based hosting of the application. As we gradually built in-house Open Source expertise for application development and infrastructure configurations, we moved to on-premise hosting, supported by our DevOps team.
Looking back, our Open Source investments and the careful project planning paid-off. On a recent Monday morning, NYC DOT clients were greeted with the announcement of the successful implementation of the web-based, smartphone compatible, map-centric Sign Information Management System, completely replacing a mainframe green screen application they were using for 30+ years until the previous Friday.
Where do we go from here? With a scalable enterprise-level Open Source platform in place, NYC DOT can undertake other major initiatives leveraging it. A strong interest continues in using Open Source technologies in government projects and academic communities. New York City has access to both pools of resources, Open Source and commercial. By avoiding lock-in to any technology, we can use our diversity of platforms to choose the best applications for the need while attracting and retaining outstanding staff.
The future looks bright for Open Source software development everywhere, and especially at NYC DOT.