Comments by "Daniel Sandberg" (@ddanielsandberg) on "Continuous Delivery"
channel.
-
36
-
24
-
I think that a lot of gamers and game devs misses the point here a little bit. I have heard this many times from colleagues: "If management/designers could just make up their minds and leave us alone for 3 months...". Sigh. It's supposed to change!
The entire purpose of agile, scrum, etc. was to provide consistency and enable the business to adapt and change as new information, ideas, market issues arises. Continuous Delivery takes the premise from agile and the technical practices from extreme programming and turns it all up to 11. Neither agile nor CD is simply a "developers issue", it's an challenge for the entire organization. Everyone must change, from the CIO/CDO to the people translating the text, to the people making the website and promotions.
If your way of working does not allow for change, including changing dates and/or dropping other features, the result will not be pretty and it will hurt. If the processes, the technical people and management are all pulling in different directions; developers afraid of speaking up, the organization being siloed (it's not my job to test, I just write the code attitude), and managers being incapable to actually manage, it doesn't matter what you do or what you call it.
So no "their constant changes" does not waste a lot of time. Crappy development and management practices does.
23
-
14
-
14
-
13
-
A lot of the issue in University settings is that you often get a "problem of the week" to solve, then the next week there is a new problem.
You mostly work alone and you seldom have to live with the solution you made in the past.
I wish that the majority of tasks was a two person tasks and that most built on top of each other. That would teach the students code hygiene, refactoring, unit testing, documentation, version control and collaboration.
At the time of dinosaurs, when I was in University, we only had two courses in year 4 that could constitute a "real project". One was building a controller and data collector for hardware SPI laboratory equipment. And the other was building a microcomputer simulator in Java (registers, memory/page handling, interrupts, and all), an microkernel operating system to run on said microcomputer and a compiler/linker to go with it. That was really messy since we never had properly learned to be software engineers and work as a team.
10
-
9
-
This is kind of what is called "staying in the problem-space before going to the solution-space". Focus on the problem, the underlying assumptions, the whom and why, not the what or how.
One of the biggest problems with us programmers is that we tend to jump directly to the solution-space. Some business person starts talking about a problem and 15 seconds later our brains have stopped listening to the problem and is instead filled with arrows, boxes, patterns and whatever technology we know. Five minutes later we don't even remember what the problem was, just that we get to write a bunch of code using nodejs, k8s, docker and NoSQL (because I don't have NoSQL on my resumé yet). What if the correct solution is to remove a feature, delete 10,000 lines of code and add a new step to an existing process/feature?
Code is not an asset. Every line of code is a cost, a liability. It has to be built, tested, maintained, deployed, configured and operated. Programmers and code do not exist for the sake of their own existence even if we like to believe that we are. We are not getting paid to do our hobby, sitting in a corner with our headphones on, listening to "programmer focus synthwave mixes".
Sorry.
8
-
7
-
7
-
7
-
6
-
5
-
5
-
4
-
4
-
In OSS there may be 1000s of untrusted contributors all over the world and only a few trusted committers. This is where feature branching and PRs shines, this is what feature branching/pull requests were made for.
Doing feature branching when co-located, pairing, having whiteboards, talking, discussing, working with each other for years and a shared vision is in most cases just bureaucratic silliness! Unless of-course the "team" consists of individual "pros" that thinks that working in IT is sitting alone in their corner, wearing headphones, getting paid to do their hobby.
Want to bet that most programmers with less than 5 years of experience has never seen anything but FB/PR? Ever wondered what we did before that (and git) was around? It has become dogma, almost in a biblical sense. This is the way, the only way, you heathen! So us old developers that are doing TBD are viewed as heretical and unprofessional hobby programmers by the new guys. Kinda funny actually.
Context is everything and most developers has no such understanding, nor perspective.
4
-
4
-
Do you mean "trying TBD" just like organizations "tries Scrum" by replacing requirement with user-story, milestone with sprint, deadline with release-trains, keep assigning devs by "percentage utilization" and doing annual budgets, writing 500 pages of "design" for a year before starting, developers communicating using Jira-tickets instead of talking, and nothing really changes, culturally or behavioral, et. al?
You can't just decide to "go CI/TBD" unless you have the practices and tools that makes it possible. Like pairing, TDD, good fast build system, being able to make changes by many-many small steps without ripping 23 modules apart for 3 days and a culture that supports it.
CI/TBD is a skill, and just like everything else it takes time to get there. I would suggest to start by setting up some things:
1. Make sure there are high-level tests covering the most important happy-paths.
2. Make these tests fast, unfragile, and make them/the system runnable on developers machines.
3. No changes gets committed without some kind of test covering the code being changed.
4. No feature branch lasts for more than 1 day.
5. Get a build system that tests main at every merge/commit.
6. If the build turn red - you have 10 minutes to fix it, or revert it.
7. If you can't get the build back to green EVERYBODY in the team stops what they are doing and do anything they can to get it back to green. There is no situation where a failing build is OK to just leave and keep doing other work (which usually happens with branches - habits matters).
Getting these things into place, and making them a habit will go a long way.
Next we turn the heat up:
8. Pair program as much as possible. Not every change requires a review (and some changes can be reviewed after the fact. See "Ship / Show / Ask" by martin fowler).
9. Try to slice changes into small parts that takes a couple of hours or less.
10. No feature branch lasts longer than a couple of hours.
11. TDD and refactor mercilessly. Make changes easy.
12. Make it hot: Any changes that has made it into main may be deployed 30 minutes after you commit, when you're on lunch. Now code-hygiene, care and practices really starts to matter.
13. Start doing some changes without a feature branch/PR. Evaluate, practice, habit.
14. Remove organizational/cultural issues: The entire team succeeds or fails together. As long as everything is driven by "individual tasks" (and associated rewards), stupid "sprint commitments", and "done" that means - "I coded it, it's done, right? Then a separate QA-department is trying to inspect quality in after the fact and it's someone elses problem after that" - nothing you do will matter. Culture eats strategy and intentions for breakfast.
15. Expect things to go wrong, learn to deal with it and improve.
It takes years for a team to get good at CI/TBD. It takes even more years for an organization to get good at CD.
3
-
3
-
3
-
@OlleHellman The reason I'm asking is that basically everyone in this comment section seems to think that CI is a tool, running "checks" on a build server against your feature branches.
That is not CI, by definition.
You don't "run through CI".
You don't "have CI".
You don't "use CI".
You either do CI or you don't!
CI is a practice and a social contract *not* just a build server even though tool-vendors would like us to think so.
Everyone integrates their changes into the *mainline* , at least once per day (preferably more).
Unless it's merged into main and automatically verified, it's not integrated.
If you can't run a majority of these checks on you local environment before committing, you *can't do* CI.
Running checks against a feature branch is not CI.
Sorry, but I'm adamant about this because our industry has gotten into this bad habit of redefining things to mean "whatever we are doing right now".
- We installed Jira and have burnups, burndowns and user stories - we're agile.
- We created a new team called "DevOps" and hired "DevOps engineers" - we're DevOps.
- One write code, then someone else looks at the code, then we change the code, then we PR the code - we're pair programming.
- We are running Jenkins against "whatever we are doing with branches right now" - we're doing CI.
3
-
3
-
3
-
3
-
3
-
3
-
In part. Continuous Integration is a human practice where it is expected that the developers pulls the latest changes and run the unit tests, lint, checkstyle, findbugs, (and component integration tests) before pushing. Those same checks are then run at push, which is called a "commit build" and should give you a high confidence that everything works. In CD there are other kinds of tests and checks downstream that takes different aspects of the entire system into account.
As I understand it, the embargo shown here is seldom used. Again, CI is a human practice and a social contract: if the build is not fixed within 10 minutes, anyone can revert the commit that broke it. The key here is to be disciplined and don't break things all the time, and the few times it breaks we do whatever we can to get back to a working state. Shit happens, learn from it and move on.
Part of the issue with feature branches is that it optimizes for individual developers working alone in their corner with headphones on. This is a bad thing. Sure, you can use short-lived feature branches if you must; where (in my view) short means a couple of hours. But if most only live 30 minutes to a few hours what's the point of branching in the first place?
Look, I'm just an opinionated a-hole that's tired of arguing with people and also thinks that part of the popularity around FB/PR is an amazing feat of social engineering by the tool-vendors that has even the oil and tobacco companies in awe. So I'm just going to give up and drop this Twitter thread by Jez Humble: https://twitter.com/jezhumble/status/982988370942025728
3
-
3
-
3
-
3
-
I hope Dave doesn't mind me plugging one of Dan Norths presentations "Why agile doesn't scale and what you can do about it".
It is primarily about how "agilists" and business talk past each other and how developers need to stop focusing on technical things and "standups, burndowns, burnups, userstories, storypoints and ..." and taking a more holistic view and understanding of the business.
Organizations tend to end up in a vicious cycle where they restructure the organization and hire a new CEO/CIO/CTO every three years and start over, while all the senior developers, PMs, etc just gives up, sighs and leaves for "something better" and all the lessons learned are forgotten. Agile (and CD) affects the entire organization, from the sales people to CEO/CTO/CIO to developers, HR and training. Without this understanding, no process or methodology will make much of a difference in the end.
https://www.youtube.com/watch?v=F4b_MckXea0
https://www.youtube.com/watch?v=1MedZStiAPg (better audio)
3
-
3
-
2
-
2
-
2
-
2
-
I must agree. Every single "git tutorial" starts with "branching" and then too many developers think that that is the goal - to follow some branching strategy perfectly - missing the point of version control and software to begin with. The number of times I've been told I'm unprofessional because I avoid FB/PR and code-reviews and prefer CI, TDD, pair-programming and good tooling that tells me if I'm particularly dumb today. I can't even change jobs anymore because "FB/PR/CR" is mandatory everywhere and I've even been to interviews where they never heard the term "Continuous Integration".
I have seen similar effects with for example agile; "we are doing burn-up, burn-down, user stories, story points, sprints and planning poker; therefore we are agile" - and then people end up fighting over "who is doing story points right". They don't understand agile. Oh, and don't get me started on "DevOps Engineer" - if your company have DevOps Engineers and DevOps Teams you don't understand DevOps either, but I digress. I had another experience where the developers wanted to go nuts with versioning; alpha, beta, rc1, rc2, final, gold and when I asked why the answer was basically "it's what the big companies are doing" - so essentially they wanted a complex release process because it "sounded professional", not understanding that all that complexity is most likely a symptom of bad software engineering. Complexaholics all around. Bring the pain!
Now I need my benzos again.
2
-
You state that CI is based on knowing the right solution the first time. It's the opposite!
CI and most "agile" practices recognizes that we can't know the correct solution nor the right implementation, or even properly understand the problem to begin with, and optimizes for being able to turn around and adapt over time.
FB/PR on the other hand optimizes for doing it once, one chance, most of the time with an "all or nothing" mentality. Get it right the first time by working on it and polishing it for some time, merge it, then jump to the next feature and don't look back. With PB/PR the granularity of a "commit/push" is "a task".
With CI the granularity of a commit/push is "small change that works and takes us closer to the goal, but sometimes we have to turn around and change it, even when it has reached production" (it still works, it's not broken sh*t code, it's just not quite right yet).
This way of working with CI practices scares people that have never tried it and that often are measured and rewarded based on some kind of productivity metric or risk ending up on some kind of PIP.
2
-
2
-
2
-
2
-
2
-
2
-
2
-
First of all, don't get hung up on the "18 months" thing. That was just an example how bad it can get, i.e a slippery-slope scenario.
The first thing (that is painfully obvious in this comment section) is that apparently everyone are working alone, everyone is focusing on finishing "their" tasks, their changes. This is so wrong it hurts. The second worst scenario you can have in an IT organization is a bunch of developers working alone in their corners with headphone on, thinking they are a team. The absolute worst scenario you can have is a lead/manager assigning tasks to individual developers and keeping some kind of tabs/score and handing out gold stars like in first grade. Combining these two scenarios will just create a culture where Bob won't give a sh*t if Anna is in trouble and need help, because he "got to get his tasks done on time or the manager will get angry at him". And then if there is some kind of (merge) conflict it will escalate into a personal conflict. Why not also start paying testers for the number of bugs found, and developers for the number of lines of code written? Wait, we tired that in the 80's - disaster!
As a developer you are supposed to refactor. If you are making changes to a module anyway and spot something that's a bit off, just fix it, now! It's there, right in-front of you, you know how it should be, pair up and just do it. This is a really good habit. Look up the term "refactor mercilessly".
Regarding OSS: Feature branching and pull-requests were invented for and by the open source community. A situation where you have maybe 1000s of untrusted contributers and just a few trusted committers. This is where FB/PR shines. If you do not find yourself in that situation, there are better ways, like TBD and CI.
2
-
2
-
2
-
2