DevOps Overview
2022-12-30
0. Intro
Today we’re going to talk about DevOps, the topics we are going to cover are:
- What is DevOps?
- Goal and Key Metric of DevOps
- Core skills required to become a qualified DevOps Engineer
- Certificates and Exam Prep Resources
1. What is DevOps?
DevOps is the union of people, processes, and products to enable continuous delivery of value to our end users.
As Software Engineers, we are familiar with SDLC (Systems Development Life Cycle ), this is basically a process for planning, creating, testing and deploying an information system.
DevOps can be viewed as a superset of SDLC, it emphasizes the union (or integration) of multiple teams, and multiple processes to achieve the most important business goal – deliver value to end users continuously with quality assurance.
Essential DevOps practices include agile planning, continuous integration, continuous delivery, and monitoring of applications.
DevOps is a broad area, doing it right essentially means a mindset shifting of the entire organization and it requires a lot of cross-functional team collaboration and effective communication. That’s why some people also view it as a critical part of the Organization’s Digital Transformation.
Definition of Done
A DevOps Definition of Done is working software collecting telemetry against the intended business goals.
DevOps may HURT at first
Like with any changes, adopting DevOps practice is not an easy journey and things could go wrong in the early stages.
One way to help with the pain is to take baby steps and do some experiments, try to find the practices suitable for your organization and process that can amplify the most significant impact, then try some other practices to cross-train the DevOps muscle and develop synergy.
The more you exercise the new techniques, the easier they’ll become.
It’s not only the tools and processes that are changing, it’s also how people are going to react to these changes, some of them might have an open mind, while others could be on the defensive side that insist “This is not suitable for us”.
Bear in mind that it’s a journey for the organization, so it takes time, effort, communication, persistence and patience to see the result.
2. DevOps Key Metric: Cycle Time
Keep in mind that the business goal is always to continuously deliver value to the end users (and of course, grow/scale the business).
In this regard, cycle time is one of the key metrics to measure continuous delivery because it tells us how long it takes for an idea from being worked on till it has reached the end users.
Once it has reached the end users, we need to keep monitoring and collecting feedback to facilitate new changes and values to be delivered.
How to Measure Cycle Time?
Start with the release pipeline. How long does it take to deploy a change of one line of code or configuration? Ultimately, that is the brake on your velocity.
Q: Speed sacrifices Value?
However, some people might argue that faster delivery might impact the value being delivered, for example:
- what if we are developing a useless feature that no one likes?
- what if we are sacrificing quality for speed and the application becomes unstable, impacting a large number of users?
- ….
These are valid points because we definitely won’t feel satisfied by just looking at the low cycle time number while a business disaster is caused by “running too fast and carelessly”.
So how can we do it in the right way? How can we reduce our cycle time while maintaining the right level of quality and confidence?
More Changes, More Risks
If you think about any past deployments, it is obvious that a major release with 10 major changes is more likely to break something than a minor release with 2 bug fixes.
DevOps practices encourage more frequent deployment by:
- Working in smaller batches.
- Using more automation.
- Improving telemetry.
Essentially, DevOps Engineers observe the current business processes, identify key phases and bottlenecks, build an end-to-end pipeline with integrated tools and automation to increase the delivery frequency without sacrificing the quality.
Sounds too good to be true? Let’s keep exploring!
3. DevOps Process
Planning
- Process: Scrum (a way to do agile development)
- Permission Management: define team member’s roles and responsibilities, assign least privileged permissions
Development
- Version Control: Git-based repository (Distributed)
- Dependency Management: Azure Artifacts (feeds)
- Branching Strategies: Git Flow (Main, Develop, Feature, Release, Hotfix)
- Branch Policies: Work Item Traceability, Automated Checks, Peer Review Policies
- Testing: Unit Testing, Manual Testing
Continuous Integration - Build Pipeline
- Input: code
- Output: compiled packages
- Trigger: Manual; Automated - Code is pushed to certain branches (or do it in batches)
- Integrate Pipelines with Tools: automated pre-PR checks, static code analysis, automated tests, dependency scanning, security scanning, code coverage, check open bugs…
- Implement Quality Gates: define rules and policies to gate-keep the release (CD) pipeline
- Versioning Strategy: Semantic versioning (SemVer 2.0 )
With testing built into your CI pipeline, every committed code change triggers a build, and then the build runs tests to ensure the changes pass all tests, policies, and code compliance standards you established for your application. With this in place, bugs are identified earlier and with greater context to simplify their resolution, your teams can deploy more frequently and confidently, and you minimize manual testing and reworking late in the process.
Continous Delivery - Release Pipeline
- Input: package from CI
- Output: package deployed on different environments
- Trigger: Manual; Automated - CI finished, scheduled
- Infrastructure as Code: Provision and configure target environments
- Deployment Validations: UI Testing, Load Testing, Availability Testing
- Deployment Strategies: Blue/Green, Canary, Rolling
- Deployment Policies & Gates: Testing Pass Rate, Manual Approval…
When continuous delivery is done well, your software release processes become boring — that is, they are low-risk, consistent, and repeatable. Then you can confidently plan release processes and schedules, automate infrastructure and deployments, and manage your cloud resources more effectively.
Continuous Monitoring
- Tools: telemetry log, application insights, alert rules, dashboards
- Outcome: Trigger communication & actions, Blameless-retro Culture, Knowledge Sharing
Communication & Collaboration
-
Communication Rules:
-
Avoid noise (communicate message ONCE in context)
-
Keep communication actionable (deliver to the right audience)
-
-
ChatOps: Chat driving processes is a good thing, processes driving chat is even better.
-
Tools: Dashboard, Alerts, Notifications
4. DevOps Engineer Skills
Soft Skills
- Communciation & Collaboration
- Growth-mindset & Iterative learning
- Documentation & Knowledge Sharing
Popular Tools
- Process: Jira, GitHub Boards
- Version Control: Git
- OS Systems & Linux Basics
- Networking & Security
- CI/CD: Jenkins, Octopus, GitHub Actions
- Static Code Analysis: SonarQube
- Dynamic Code Analysis: OWASP Zap
- Dependency Analysis: WhiteSource Bolt, Dependabot
- Cloud Provider: AWS
- Scripting Language: Python
- Container: Docker, Docker Hub
- Container Orchestration: Kubernetes
- Monitoring: Prometheus
- Infrastructure as Code:
- Infrastructure Provisioning Tool: Terraform
- Configuration Management Tool: Ansible
Note: It’s important for DevOps Engineers to learn how to use these technologies in combination to create DevOps processes.
Azure Common Tools
- Process: Azure Boards
- CI/CD: Azure Pipeline
- Scripting Language: PowerShell
- Container: Azure Container Instances (ACI), Azure Container Registry (ACR)
- Container Orchestration: Azure Kubernete Services (AKS)
- Monitoring: Azure Monitor, Application Insights, App Center (Mobile)
- Infrastructure as Code:
- Infrastructure Provisioning Tool: ARM, Bicep
5. Popular Certificates
AWS
Azure
AZ-104/AZ-204 + AZ-400 = DevOps Engineer Expert
- Non-fundamental Azure Certificates expire after 1 year
- You’ll receive a renewal notification within 6 months of expiry date
- You can renew it for another year after passing the renewal exam (and renew again when the renewed certificate is expiring)
- Good news! The renew exam is easy, free, time-unlimited and open-book.
Exam Preparation Resources
- Microsoft Learning Path: https://learn.microsoft.com/en-us/training/browse/?resource_type=learning%20path&terms=AZ-400
- PluralSight Certification Prep Path: https://app.pluralsight.com/paths/certificate/designing-and-implementing-microsoft-devops-solutions-az-400
- Free Labs: https://www.azuredevopslabs.com/
- Microsoft Enterprise Monthly Free Credit: https://azure.microsoft.com/en-au/pricing/member-offers/credit-for-visual-studio-subscribers/
- Mock Exam Questions:
6. DevOps In Practice
Shifting security left (DevSecOps)
Successful DevOps need to consider security and build it into end-to-end automation.
Security testing can be integrated into the CI pipeline so code is continually tested.
Example of early-stage security testing:
- Static Application Security Testing
- Dynamic Application Security Testing
- Container and cluster image scanning
- Dependency Scanning
- Secret detection
- …………….
Site Reliability Engineer (SRE)
One of the DevOps principles is to release quality code fast.
SRE and DevOps Engineers are both practical roles that implement the DevOps process, if a company has both roles, then DevOps Engineer is more focused on the speed of delivery, and Site Reliability Engineer is more focused on the reliability of the system to keep it stable while allowing for fast changes.
7. Summary
By using automation, shifting security to the left, and making processes repeatable and measurable, a DevOps platform leads to better software and reduces the time between designing new, higher-quality features and rolling them out into production. And that maximizes the overall return on software development.
- DevOps is a journey of organization transformation, it requires human buy-ins, cross-team collaborations and experiments to build momentum.
- The goal of DevOps is to continuously deliver value to the end users, reliability, security, speed are equally important to make a successful DevOps process.
- With various tools and integrations, DevOps Engineers and SREs work with other teams (Dev, Test, Ops, and Business stakeholders) to identify blockers in the delivery cycle to reduce the cycle time.
- DevOps doesn’t eliminate human interaction, it focuses on using the human brain to do the things that machine/automation cannot do for us, for example: providing actionable information for the right audience and configuring manual approval gates.