Why software development is different now4 Oct, 2013
Not too long ago, when Web applications dominated the software-development world, a developer's primary decision was choosing a framework from the multitude of Web frameworks fine-tuned in the 2000s. Three forces have been changing that status quo in recent years: new devices and client delivery mediums, fragmentation of frameworks and platforms, and a renewed focus on design driven by catering to end users – not just IT buyers.
This Spotlight describes the effect these forces are having on the software-development ecosystem for developers, vendors and others who participate in the application lifecycle value chain – from idea, to coding, to deploying and back again. These forces are not only important to developers, but also to others in the developer ecosystem, such as those who sell developer tools, provide systems management suites and run the actual applications – like service and cloud providers.
These three forces aren't exclusive; there are many others affecting software development, such as the opening up of vast troves of 'dark data' that had been previously inaccessible (through 'big data' technologies and a renewal in 'composite application' patterns through public-facing APIs). However, these three forces are the primary ones actively shaping the software-development landscape.
Devices, fragmentation and design – three forces affecting the AppDev landscape
New devices and client delivery mediums
Since the release of the iPhone in 2007, new client devices have been stealing developer attention away from the traditional Web and desktop. Developers care a great deal about what types of devices end users are selecting. 'Should I, the developer, target traditional PC desktops, or do I need to instead target mobile and tablets? Do I need to do both?' Thus, when considering software development, tracking traditional PCs and new devices as one market ensures you're seeing the forest, not just the trees. By some counts, if you bundle traditional PCs with these new devices, the share between the two clusters is about 50/50. In both consumer and enterprise markets, we still see a tremendous amount of Web applications alongside mobile apps; in fact, applications are often delivered both as native mobile apps and Web applications. The result is an explosion of end-user devices to support – where the Web was just interested in which browser version housed the Web application, the combination of the mobile Web and native mobile apps creates an exhausting array of end-user form factors, most driven by touch instead of mouse and keyboard. Add to this support for 'traditional' Web applications, and now you have two problems.
While the Apple ecosystem has limited configurations across iPads and iPhones, the Android ecosystem has a seemingly uncountable number of variations. As hopefuls like Microsoft enter this market, the combinations will grow further. While it would be easy to just jam the same UI and app behavior into each of these combinations, companies like Evernote and The Omni Group have shown that custom-fitting the UI to each form factor (across desktop, phone and tablet) is key to perfecting user experience and happiness, which drive revenue. Combining phones and tablets across two or three different operating systems (as Microsoft/Nokia) creates a deadly matrix of support.
Not only are there the obvious new devices like smartphones and tablets, but early Internet of Things devices, like the Nike Fuelband, Jawbone Up, Nest Labs thermostats and others that connect 'meatspace' with cyberspace, are ripe for programmers' plundering. With all of these devices – including the Web still – comes a bevy of new frameworks and platforms, as well as application-delivery methods like cloud.
Fragmentation of frameworks and platforms
Frameworks, broadly, are the supporting stacks of libraries, SDKs and tools used by developers to more easily create applications on top of the lower-level programming languages and runtimes. In the Web application era, the notion of a framework oligarchy – often Java and .NET – was eroded as new frameworks like PHP, Ruby on Rails, and derivatives of Java (like the Spring Framework) took over. Fueled by forces like the mainstreaming of open source development and social coding at GitHub, this Web framework fragmentation has only multiplied.
Coupled with growth in frameworks, cloud computing has introduced numerous new platforms used to deliver applications to market. Rather than just relying on traditional Web servers and hosting, cloud platforms like the IaaS offerings from Amazon AWS, Rackspace Cloud, Joyent and IBM's SoftLayer provide a whole new set of piping, middleware as a service (like databases, queues, etc.) and methods of deploying software beyond the traditional 'three tier' approaches that drove the original Web push. The introduction of PaaS like Heroku, EngineYard, and first passes of Google App Engine and Microsoft Azure added curious, tight couplings of frameworks to these new cloud runtimes that met with mixed reception.
In a fragmented landscape, courting developers is key for tech vendors like service providers and systems management vendors. Overall, however, the trend is clear – developers are relying on private and public cloud platforms to build and deploy their software. This creates a pull for hosters, service providers, and even traditional ISV vendors to provide more developer-friendly platforms, services and marketing to sway developers to use their offerings. A healthy developer relations program is now de rigueur for those looking to cater to developers.
The curious effect of cloud on software development has been a datacenter land-grab by developers. System administrators previously drove the requirements for 'production,' but now developers are soaking in production concerns like packaging their applications for deployment with configuration management tools, such as Chef, Puppet and ever-emerging tools like Docker. Developers have even begun to relish tasks like log management and performance management, areas that were previously left to the system administrators. Wont to turn every task into programming, developers are helping make infrastructure more programmable, as we see with software-defined networking, storage and anything else that a programmer can wrap an API around to avoid shell scripts.
A renewed focus on design driven by catering to end users, not just IT buyers
While multiple devices and fragmentation may create headaches driven by too many options, a major benefit has emerged: the ability to deliver a perfectly tuned, pleasing and productive experience to end users. Bending Web applications to a designer's will was always difficult, leaving the user experience and design-obsessed pining for 'the good old days' of desktop GUIs. Often, this refocusing on end users is wrapped in the concept of 'design.' Numerous comparisons between Apple and Microsoft in recent times have diagnosed that, perhaps, the primarily winning strategy for Apple is its focus on design before pricing and feature-packing. This is a dramatic change from the 'speeds and feeds' method of evaluating and procuring software of the past that valued getting a 'good deal' over getting well designed software.
For developers of all types – consumer- and enterprise-centric – this means paying more attention than ever to user experience, a field that was much maligned due to high costs and uncertain ROI in the past. Tracking the success of design choices, and even experimenting with different options, is now easier and cheaper than ever. Because most applications are 'always on,' developers can observe user behavior in aggregate, tracking how their software is used and feeding those observations into the application development feedback loop.
Rather than surveying users or, more expensively, observing them using software through one-way mirrors, developers can now track every action a user performs in an application. Because cloud platforms, coupled with devops practices, allow developers to deploy applications on a weekly, if not daily, basis, developers can experiment on feedback loops tremendously smaller than the 18-month release windows of traditional software. The tool chains and services here are not well-situated yet, leaving most developers with DIY approaches. Vendors like Adobe, Microsoft and IBM would do well to bring standardized approaches for this type of design-centric tracking to their existing developer base, if only to retain those developers.
What this means for the software-development ecosystem
A sea change like the one described here has implications for many parties in the application ecosystem. Briefly, there are three broad areas that are of special interest to developers, vendors and cloud providers.
Developer relations and services
Many companies need to begin catering to developers to gain favor and preference. Each developer is continually looking for new tools and frameworks to better adapt to devices, fragmentation and design; they are looking for good deployment options in both public and private cloud form factors, and they want easy-to-use services through APIs to integrate with their applications, like for payment processing or to interface, for example, with wearable devices. In each of these, building up a good developer-relations practice is vital.
For those companies that don't traditionally think of themselves as technology companies, focusing on methods to retain and attract developers is also important. As software continues to eat everything, more companies are finding that they require talented developers to succeed. This is why you see companies like Etsy and Netflix actively encouraging their developers' involvement in developer communities, often through open source, essentially giving developers the 'soft benefits' of career-building and notoriety that come from a healthy engagement with the developer community. Setting up 'labs' sites to profile the 'cool' exploits and projects of your company can also help with developer staffing, retention and skills development.
Evolving requirements for development and complementary products, especially systems management
The tool chains used by developers, from IDEs to project-collaboration tools and everything else in application lifecycle management (ALM), must match and service the new needs of developers. Many small and midsized vendors have been innovating here, such as Atlassian and Tasktop, but the bulk of new tooling is either open source or custom-built. Supporting continuous development and delivery is an area particularly well suited for many existing ALM vendors that traditionally seek to instrument the process of development. Large incumbents like Microsoft and IBM have been operating here – even acquiring, like IBM's purchase of UrbanCode. This area is still very open to vendors large and small, however.
As the application development stack transforms, so too must the infrastructure management and support software around it. Traditional systems management tools are poorly suited to manage cloud environments, let alone mobile applications. As ever, there are numerous startups tackling this problem, notably in the application performance management (APM) space, where companies like New Relic and AppDynamics have been trailblazing. The rest of the systems management stack still needs to adapt to the new frameworks and runtimes, as well, and especially to evolve the traditional process-think in service management to better align with cloud delivery, namely, devops.
One of the big four, BMC, has actually done a good job talking the talk in this space, and even acquired a company, StreamStep, in 2011, to augment its portfolio. IBM also has potential here – the company seems to have learned a lot through its own cloud transformations internally, and has a ready 'patient zero' community in SoftLayer.
Expect to see many great rewrites with splendid success and failures
With the drive for new types of client form factors, such as mobile, and the refocused attention on the end user as the primary party of import, the endless troves of existing software running on the world's servers is ripe for rewriting. From corporate software used internally and B2C applications like online banking and insurance claims filing, to government-run healthcare and tax systems, the global pool of custom-written software is looking aged in light of the three forces driving software development. Traditional outsourcing firms will need to train their legions of programmers on the new platforms, and buyers will have to keep a close eye on claims of cloud aptitude when selecting outside firms. For ISVs, which represent a smaller pool of software than the custom software pool above, the move to SaaS and mobile is driving a significant amount of rewriting, as well.
Delight users and profit accordingly
While all this heavy lifting may sound ominous, the goal is well-designed software that makes users more productive. Part of bringing IT out of the age of cost-cutting and focusing instead on how to use IT to grow businesses relies on just that: making software that's easier to use and, in the best case, delights users on a daily basis.