Comments by "Daniel Sandberg" (@ddanielsandberg) on "ThePrimeTime" channel.

  1. Continuous delivery is a way of working (with a lot of automation) to ensure only good builds makes it to production. It is a falsification process where the "bet" is that the build is a release candidate that will pass the crucible and make it to production - but most builds won't. CD is the codification of the organization's processes and SDLC cycle so that every change is verified and has an audit trail instead of process theatre where everyone pretends the process is followed and done doesn't actually means done. Continuous Deployment is the same as Continuous Delivery but the final human "approval gate" is automated. Continuous Delivery is not "every commit goes straight to production".
    108
  2.  @ChrisPatti  Yes, context matters. The issue is when organizations optimizes for "open source, zero-trust workflows" and "we have juniors, must control them" and "we can't have seniors to hold their hands" and end up implementing processes and workflows based on the lowest common denominator. And even worse, they tend to implement it across the entire organization by edict, again missing the different contexts. Working with CI/TBD is not "unprofessional" just because someone made a video on youtube saying "Use git like a professional". But it's at that level we are as a software industry. And it's sad.
    19
  3. Not that I like Uncle Bob, but he has been saying for a decade that we SWEs need to level up and expect higher standards, or when something really bad happens and 10000 people die, the governments of the world will. They will tell us what languages, tools and processes we must use. They will demand certifications for everything and require us to sign a form liability for every release. Because we didn't set higher standards ourselves.
    17
  4. Good OO is very close to FP in many aspects. Well constructed objects are nothing but a set of partially applied functions. Typical OO in the wild tend to be more class oriented programming than OO. And isn't a Monad a special kind of object implementing the monad interface?
    14
  5. "discover footguns as quickly as possible by firing them as many times as possible *before we hit production* systems". Fixed it for you. Continuous Delivery is not Continuous Deployment.
    11
  6. Velocity is a measurement, not a target nor a promise. So how can your "velocity be off"? The point is to tell you some kind of real rate, not the wished, hoped or estimated rate. That there are other things taking time, unpredictable things, is normal but should be managed to not overburden. Of course if you've been doing 5x more support than expected it may mean that the way you are working (development practices, testing, documentation, refactoring, monitoring, logging, planning, etc) need to change so that you don't drown in support. Something about a self-inflicted wound. I find the argument about mixing dev + test time as very odd. Untested, unfinished, undeployable features are worth nothing. There's no separate point-system for devs and testers. It's either done or not. I loathe developers that declared "I'm done", throws it to a tester and starts working on the next thing, and it doesn't even work. If you have a culture where there is a blame game of "you didn't meet your commitment" or "who estimated wrong" the organisation have not understood anything and you should either change your organization or change your organization. Tip: Remove the columns "ready for test" and "in test" from your board and only have todo, doing, done. If it's not tested it's not done and NO ONE gets any points, gold stars or pats on the back until it's actually done! The point of agile is not about processes, story points, burn-up, burn-down, sprints, etc, stuff. It's about giving everybody a healthy dose of reality so that the team and organization can adapt and change early, instead of hope that whatever "wishful plan" people have made will actually happen.
    7
  7. You don't "have CI". You practice CI. CI is much more than "a build management system that checks stuff". Can we please stop redefining CI to mean "whatever we're doing with branches and Jenkins" on this project!? The problem is that vendors say "buy our build tools to have CI" and people don't know what that means, just like people say "refactor" when they mean "two weeks rewrite" - not the same thing. If someone pushes broken crap without running the full build and test cycle locally first, they broke the social contract - don't push broken shit. If you do end up breaking something the build system tells you that, you have 10 minutes to fix it or revert it. if you ignore the issue, anyone else has the right to revert your changes. If you can't make it work again - the entire team stop what they are doing, huddle around and fix the problem. It is more important to keep the system in a good state and improve daily work than to work on more feature on-top of a broken system. But but but but... master will be broken all the time! No it won't because everyone will have to learn code hygiene and work in many small atomic changes. With branches you are actively ignoring that problem by isolation, branches are often broken for hours or days and no-one learns code hygiene.
    5
  8. FB/PR opinions reminds me of a cult. See how that works?
    5
  9. If everyone is working on a bunch of feature branches and only occasionally merges back to master - how will a rebase off master help keep all these feature branches up to date with each other? They won't, you'd be rebasing nothing, because all the changes are isolated - which flies in the face of CI. And he does not work alone. Read up on LMAX and what he and the team did there, without branches or PRs.
    4
  10. When I was younger I thought HR meant "Human Relations". Imagine my confusion once I found out it meant "Human Resources". If you think about it, resources are things that are USED UP. Are you being used up? That is why May 1st is "No Resource Day" and when if a manager calls you a resource you should call them "overhead". :)
    4
  11.  @robertminardi4268  Ok, you made it sound like there was a lot of finger-pointing going on. I might be projecting a bit. PTSD. I agree with using something more like kanban. However, the point of measuring velocity is to have some kind idea how much you got done each timebox. Now, I'm not a fan of estimations (too much WAGS and micromanagement), but if the team keep cramming more into the next sprint in hopes that they will catch up, they're not taking into account the velocity from the previous sprints (which kind of tells you how fast you're really going - i.e. capacity). Reminds me of the problem we had with trains about 10 years ago - the trains were late in the winter due to snow, the estimates on the boards was based on the faulty idea they would be able to "catch up". Of course that never happened, every station was a few minutes later than the one before and the end destination was 20-30 minutes late. I have a feeling you know this instinctively already. It's frustrating when it feels like it's the bosses that's causing the chaos... PS. I like this video, it's a great resource when wrestling with the dichotomy between work and management: "XP Days 2013: Dan North - Why Agile doesn't scale and what you can do about it" PSS. The noestimates movement was big 5-10 years ago.
    3
  12.  @ruperth.  "these people"... great argument. Why don't you approach Dave directly and ask him about his experience and what kind of project and software he has worked on instead of sitting here throwing out nonsense that he has never done any real work. Like the other 1241 in this thread that has made the same non-argument so that they can continue to clap their ears and yell la la la. And no it's not nonsense. I have worked with TBD/CI for almost 20 years in highly regulated markets. We have on occasion been forced to do FB/PR and it always resulted in worse outcome, more people leaving and slower development. So do not tell me it is nonsense. You know what is nonsense? All the people praying at the altar of FB/PR, denigrating us that has found a different way of working and yet I bet most proponents of branch-based development have never not worked with branches and as such has no other argument than ignorance to stand on.
    3
  13.  @MaxYoung-Maxinfet  ​ I think you misunderstood. I am squarely in Daves "CI/CD with no branches" camp and absolutely abhor branch-driven development; especially environment branches. A merge to master is NOT a deployment to production. What I meant is that I have never deployed a branch and that I cannot understand where the idea that "master represents production" even comes from. I deploy an immutable versioned artifact. I have never treated master as production. I treat production as production. I use tags, references and versions. Although I do treat master as if someone may deploy the latest build to production while I'm out on lunch.
    2
  14. Sounds like a bunch of programmers not knowing what they were doing and as usual the bad experience gets turned into "OOP is bad". If developers spent even half the time, they spent on complaining about things they don't understand, on actually understanding the why of things this industry might stop being such a sh*tshow.
    2
  15. You are the 10000s person to argue that point. And it's incorrect. Let's say you have 30 developers, each with their own feature branch. Even if all of them rebases from master every hour - how many changes are integrated to the branches? None? They stay continuously isolated and you won't know which branches that conflicts with which other branches until one of them is merged into main. CI/TBD is about exposing conflicts early by continuously integrating into mainline. It's nothing strange. What is strange is the obsession with branches to isolate changes and hide systemic issues.
    2
  16. And you'd be so wrong. Like so many others in this comment section you're engaging in ad hominem, red herring and bulverism. Just because you probably have always worked branch-driven does not make him wrong. Maybe not attacking people because they show you a different way of working. And a way of working that has been around for over 30 years at that.
    2
  17.  @ruperth.  Wait? What? Who is hurting whom, how?
    2
  18.  @ruperth.  What methods? You mean corporate scrum (which is not agile)? You mean corporate devops (which is not devops)? You mean corporate CI/CD (which is not CI nor CD)? You mean the incompetent management that just want to slap those terms on top of whatever they are already doing? You mean developers that haven't spent a second actually trying to understand anything beyond technology masturbation and sitting alone in their corner getting paid to do their hobby? You mean those people? You're hating the people that do not want to learn and then blame the "thing" for their failure.
    2
  19.  @ruperth.  Because people takes the path of least resistance. They don't want to spend time understanding. People inherently don't want to change and will find any excuse not to. And the result is junk. Not because of the ideas but because of people being people. The hardest part of learning something new is not learning the new - it's to unlearn the old. All of these things requires a deeper understanding in how work works. They require that the organization actually change. Again, we're not being paid to do our hobby and yet that is how most people in the software industry acts. My grandpa refused to try pizza his entire life. A few weeks before he died we tried to have him try it. His answer? "Why would i pay for pancakes with ketchup?" And that is the same misunderstanding people have about all this XP, agile, Devops, CD... they do all the cargo-culting motions with burnups, burndowns, story points, standups, sprints, etc and then a few years later all the developers are going "I hate agile". Not understanding that they did it to themselves.
    2
  20. Inherent complexity vs. Accidental complexity. Accidental complexity = technology complexity * technology m*sturbation. We are very good at j*erking off!
    2
  21. No. 357 person using the same argument. My response is the same as always: It's very simple. Git is everywhere, every tool, every integration, every provider, course and tutorial is using git. Of course we'll use it. "We're using git, therefore we must use branches" is a ridiculous (and is quite frankly an ignorant) position to have.
    2
  22.  @cronnosli  Yes. But he was working on the largest, most mission critical, open source project of all time. With 1000s of more or less untrusted contributors all over the world. If you find yourself in a situation where collaboration and trust is that low in your team and organization you should probably ask yourself why that is and work on that instead. Optimizing for individuals to work in isolation is not a great way to manage a team or maintain a codebase.
    2
  23. That is because you weren't practicing CI, and by the sounds of it - didn't pay attention to the feedback and had a "team" of individual tasks over a team that collaborated. I have lead a team of six that managed 600 applications in a highly regulated environment where we practiced CI, PP, TDD and never used a branch.
    2
  24. TDD is about figuring it out as you go along, not upfront.
    2
  25. The problem is not "not using branches". The problem is that people do not know how to work together in a code-base without breaking things all the time. Adding branches to the work often just hides the problem - it's a band-aid for low trust organizations. It does have it's place in OSS and when bringing new people on. But to force it upon everybody irregardless of experience and skill level is how organizations slow down and experienced people ends up leaving, then being replaced by juniors, entrenching the need for branches and rules. It's a self-fulfilling spiral. PS. I have working in highly regulated iGaming for 15 years. We have always worked on main and the times we have been forced (by management edict) things did not go well. It IS possible and CI/TBD does NOT makes us irresponsible cowboy developers that "has never worked with other people on real projects".
    2
  26. The point is that when you have 30 developers, each working on their own branches for days on end, what does it help rebasing from master? You still won't know if anyone else's changes conflicts with all other changes until it's merged into master.
    2
  27. You mean that old guy that lead the team that made one of the highest performant online trading systems in the world (LMAX) with more than a year in production before the first production bug; by using these XP/CI/CD techniques, in java 1.6, over 10 years ago? What you just did (not the first person in this comment section I might add) is a form of "poisoning the well" in an attempt to preemptively discredit a person so you can dismiss anything presented.
    1
  28. I'll just copy paste from my other answer: Dave used to work for a company called LMAX. They practiced CI and CD and built one of the highest performant real-time trading system at the time, in java 1.6. They had to adhere to lots of laws and compliance while doing so, all while handling a large amount of transactions. They worked in a single repo, on a single branch and released to production once per week; it took a year for the first minor production bug to appear. Many of the devs has split off after the company was sold and started a new company (TransFICC) to continue on the same path, using the same CI/CD practices. Dave promotes whole team involvement, pair-programming and good acceptance tests in addition to TDD. Your arguments makes the assumption that CI/CD is risky and can only work in a narrow context; which would be so if the team/organization does not practice good software engineering. Maybe you are right in the sense that most organizations and people in SWE are mediocre at best and have collectively decided to keep it that way. This makes me sad.
    1
  29. huh?
    1
  30.  @TheBrazilRules  What does that have anything to do with TDD? I'm actually confused by your comment. Who have said anything about large up-front design? Did I miss part of the video?
    1
  31. You misunderstand. CI means that everyone's changes is merged and integrated into mainline at-least once per day (preferably much more often than that). If people are sitting with local changes for days on end, they are screwing up. I.e. your scenario is null and void.
    1
  32.  @markpaterson2260  So what did you exactly mean by "the dudes ISP dies or his harddrive goes to shit"?
    1
  33. What is your definition of CI?
    1
  34. Push to master/main is not deployment to production.
    1
  35. I wouldn't say he (Dave) is mixing anything. From his perspective there can be no Continuous Delivery without CI, and at the same CI is not enough. While Continuous Delivery evolved in multiple places at about the same time Dave and Jez gave it a name and codified it in their book. They also named things like "deployment pipeline" which may confuse things for people not read up on the subject.
    1
  36. An answer would be an entire book. However there is a large misconception about CI/CD here... For the 437th time: Push to master does *not* mean deployment to production. CI/CD requires really good engineering, test automation, security and compliance automation so that the majority of "functional issues" *cannot* reach production. And the goal is to prove that the build is *not* ready for production as fast as possible.
    1
  37. What is this obsession with "deploy a branch"? What does that even mean?
    1
  38. ​ @scanrecs  With CI (and CD) we don't do that. Don't use branches to keep track of what should be in each environment. We build it once from main, stamp a version/tag/commit-ref on it and then don't change it. We then move that immutable version between environments, stages and approvals. i.e. what moves through the pipeline for the different environments is a version/artifact/tag/commit-ref, not different branches. There is an article on Hackernoon called "A Guide to Git with Trunk Based Development" that describes one way of doing it that way. Quote: "What version of each application is in staging? The answer is no longer “whatever is in the development branch for each project”. It’s 0.0.75, 3.15.1, 0.0.20, and 0.0.55. It’s explicitly defined in an environment repository. Meanwhile your production environment might be on www version 0.0.50 and content on 0.0.74. Also explicitly defined in code."
    1
  39. And when you have 30 developers, each working on their own branches for days/weeks on end, what does it help rebasing from master? You still won't know if anyone else's changes interferes with all other changes until it's merged into master. And that is always fun to discover 2 days before the deadline. So it creates even more issues because "what goes into a release is managed by merges" instead of manage it with opt-in/feature flags/configuration/etc. Feature branch workflow optimizes for isolation. CI practices optimizes for early detection of conflicts and issues by Continuously Integrating.
    1
  40. What happens with all that production code that needs to be changed, refactored, redesigned and removed when the requirements change? Oh, there is no difference, both are code that changes for the same reason! It's a false dichotomy! You're just making the classic argument that it's "bad/waste"; when the real reason is simply that you're looking for an escape hatch so that you don't have to do that new scary thing you don't want to do.
    1
  41. Because we're testing the expected behavior. Not that" the code we wrote is the code we wrote".
    1
  42.  @ghun131  I am not suggesting to "avoid doing support". I'm suggesting to figure out if/why a lot of time is spent doing support. Why would managers tell you how to do your job? As a team you need to figure out where the time is best spent, for the short term and the long term.
    1
  43. Dave used to work for a company called LMAX. They practiced CI and CD and built one of the highest performant real-time trading system at the time, in java 1.6. They had to adhere to lots of laws and compliance while doing so. They worked in a single repo, on a single branch and released to production once per week; it took a year for the first minor production bug to appear.
    1
  44. Dave pushes directly to main whenever possible.
    1
  45. Who has said anything about deploying to prod every day? Dave used to work at LMAX (a high performance online trading platform, so lots of financial laws and compliance). They had over 48 hours worth of tests (that's a lot!) that they were able to run in under an hour by optimizing and parallelizing the pipeline. It took them a year before they had their first production bug. They used CI/CD as described.
    1
  46. That is not CI. What you are talking about is continuous build. Continuous Integration is something different; it's a practice, not a tool, irregardless what vendors are trying to make us believe.
    1
  47.  @nefthy  Not really what I meant. Unless it's merged into main, built and passes automated tests, it's not integrated.
    1
  48. One of the main points of CI is that everything is always integrated in the main branch so there is no "integration cost prioritization" at the end. You can still have the code integrated without releasing to the customers. Either by build profiles (exclude features in the build), skip deployment of those components (if you have a more modular plug-in like architecture) or feature flags (where the features are included, but are disabled/hidden/opt-in by configuration or license).
    1
  49. In which way is he wrong? Where does he say that there are no other environments?
    1
  50. Don't enable the unfinished features using feature flags! All other changes in the code shall work, which is why automated testing is important. We do this all the time in Java/Kotlin/Go and it works just fine. Perfect when rewriting a component; just a flag if it shall use old or new implementation and flow. If the new implementation/feature doesn't work - just disable it. Need an emergency fix? Just make it and deploy! It's always deployable, even with unfinished features committed. In CI/CD the emergency hotfix process is exactly the same as any other change (except that we may skip days of performance testing). I can ask a question back: What if you have a bunch of features in the development branch. There is a deadline, but 3 out of 10 features that are merged are not in a releasable state due to finding bugs late? What do you do? 1. Revert a bunch of merge-commits at the last minute and have to re-test everything overnight? - Which may cause breakages in other code that had to be modified/refactored due to changes made to accommodate these 3 features. - Leading to people becoming afraid of refactoring and then process is added so that everyone has to ask for permission, time and budget to refactor. - Leading to increase in tech debt, stress and whatnot. 2. Taking cherry-picked commits from dev to master? And ending up with a master with a combination of commits that has never been tested together? - Even harder with the anti-pattern of doing squash merges from feature branches. - And all the other problems from (1). 3. Making another "release branch" and cherry-pick so that it at least can be tested in isolation. - But now ending up with yet another "truth" of the system? Have fun merging back and forth like it's 1997. All of this just creates complexity. The organization ends up hiring release managers, starts doing a lot of planning, processes, gatekeeping, Gantt charts, inter-team dependency management, etc. CI and TBD removes 90% of the problems. But it requires engineering beyond implementing the task and moving on to the next.
    1