Over the past few years, we have experienced a shift in the way we deliver applications. We are being forced to innovate faster and deliver higher quality software. In response to this, the way we approach the design and delivery of software has changed.
— OverOps (@overopshq) May 20, 2018
Most would identify the cloud as a main component of this change, but it is too abstract of a concept to capture the true depth at which we’ve changed. As you unravel the layers of change, what you will find is our entire thinking of how we get our solutions in front of customers has also transformed. The most impactful change is the application of supply chain concepts to software. We have created a software delivery supply chain.
Software Delivery Supply Chain
a method to plan, implement, and control the efficient, effective storage and flow of software from concept to customer.
The supply chain concept identifies process improvement to efficiently transform raw materials into physical product and has evolved over time to include concepts of integration, globalization and specialization. This original evolution (for hard products) took decades.
In software, the supply chain evolution has taken just a few years
The changes over the past few years are phenomenal. There as an almost entirely new set of tools that have changed the way we create, deliver and manage our (software) products. This modern tooling is comprised of open source and proprietary tools and it is important to note that there are countless permutations of tooling that can be employed.
Everything has changed
Everything we build today and the tooling we employ is typically built to an API. This API movement originated as a vehicle to create platforms where others could build on top of a framework. Today, this API movement is more of a complex web of interactions where all the building blocks work together through an agreed upon standard for a common goal. We have building blocks that all work together and this force more than any has contributed to the emergence of the new supply chain that is comprised of a few groups of capabilities, which includes: Workflow, Deployment and Monitoring.
However, before we get into the three groups, we should note that just as there is raw material that enter a physical supply chain (steel, wood, plastic, etc…) there is also a set of raw material that enters our software supply chain. This too has changed over the past few years as open source software (OSS) has become the commodity and the baseline of almost every application. It would be rare to find a code base that didn’t include OSS in some way. And with OSS much of our new raw materials have become abundant and they evolve faster than ever. Now, let’s explore the tools.
Product Creation: Dev Collaboration and Workflow Tools
The same tools that fueled the emergence of OSS are also transforming our dev teams. Pull requests and Jira tickets are the lingua franca for the modern development organizations. Agile development, collaboration and instant information have changed the game. It’s the way software is built. We also have a new way of packaging our apps so we can enjoy maximum portability, containers.
Some of the new tooling in this part of the supply chain includes:
- Source control and collaboration: Github, Bitbucket, Gitlab
- Issue tracking and agile: Jira, ServiceNow
- Packaging: VMWare, Docker
- Alerting: PagerDuty, Pingdom, HipChat, Slack
Product Delivery: Deploying our Applications
A major shift in the arm for speed of delivery of applications has been driven by a need to automate common processes. Building software and deploying infrastructure is typically an exacting and detailed task that many operators had built scripts around. Over the past few years, tooling has emerged to take these common processes and wrap them with tooling. We have even started to deliver new platforms to automate orchestration of workloads.
Some of the key tools here include:
- CI/CD: Jenkins, Travis CI, Circle CI, TeamCity, Codeship, GitLab CI, Bamboo
- Provisioning, Configuration Management: Chef, Puppet, Ansible, Terraform
- Orchestration: Kubernetes, Mesos, Pivotal Cloud Foundry
Product Success: Monitoring and Reporting
Once applications and services are live we must make sure they not only do what they do, but they perform and deliver as advertised. A set of tools has been created and adopted to take care of this. We have log aggregators and log analysis tools that allow us to search through this information to determine what our apps want to tell us. We can also monitor our apps for slowdowns and performance degradation so we can ensure we meet customer demands. And open source stacks have evolved so we can send data to tools and run analytics and dashboards as well.
Some of this tooling includes:
- Log Management: Splunk, SumoLogic, ELK, Graylog, Loggly
- APM: AppDynamics, New Relic, Dynatrace, and open source flavors like Kamon
- And hybrid solutions like Datadog, Stackify, and the Elastic stack who do a mix of all 3
Then and now: monolith and microservices
The main difference between the 20th century “hard” product supply chain and that which we use for software today is we are shifting away from conceptualizing and maintaining a single product, end to end to creating individual components that communicate through an agreed upon language, these APIs. And it’s not just hard goods, it’s how we are breaking own our monolithic application into microservices.
This new approach employs individual and unique blocks of code that are all sourced and maintained within their own paths but ultimately integrate into a whole. It’s as if each lego block in a package is sourced, built and delivered all separately. The line tooling is the same but the raw materials differ and in the end the way we assemble into a final product.
The software supply chain has ushered in this new approach. It is a direct influence in the move to microservices.
Supply chain optimization doesn’t come without challenge
Ultimately, our development teams are being pushed to the limits as we use this new efficient supply chain to push more software into production at an accelerated rate. But this doesn’t come without a cost. As we speed adoption of delivery across smaller components we increase complexity and introduce significant risk. There is an opposing force to speed; and that is reliability. Quality control in the software supply chain is complex.
At OverOps, we exist to address this problem. We believe the delivery of reliable software should be a primitive for every organization. Reliable software should work the way it’s supposed to work, perform as expected and be secure. We help you make sure it does what is it is supposed to do. We help you with functional quality.