Comments by "TheEvertw" (@TheEvertw) on "Continuous Delivery" channel.

  1. The behaviour of Fujitsu is shocking. And as they could have predicted, this has now come out, and will cost them in the order of a billion pounds in damages. I never understand how some people will protect their own image & ego in the short term, while ensuring its destruction in the long term. The managers that made the choice to keep this bug under the rug may well face criminal prosecution.
    63
  2. Many managers complain that software is the only discipline in engineering that can't make good estimates. But in all other disciplines, designing is about 10% of the final cost. The rest is manufacturing, logistics, purchasing etc. In software, designing is 100% of the cost. There are no (almost no) "manufacturing" costs. The other disciplines can not estimate how much "design" costs either. If they do estimate it, they are often wrong by a factor of 2. But they can make pretty good estimates for manufacturing from a preliminary, rough design, and the variance in design costs is negligible compared to manufacturing. That is apart from the other disciplines dumping all their big problems on software. "Oh, software will fix that" has doomed many projects. This include inter-department warfare that software is somehow to bring to an end -- the bane of many government SW projects.
    60
  3. While software developers have a professional duty to do their best, I feel the main responsibility lies with their principal. This team did not have the skills & knowledge to build this system. Otherwise it would have instituted processes, tests and mechanisms to ensure ACID behaviour. But I don't think you can blame people for something they did not know. The final responsibility is with the principal who trusted them, not the developers. Fujitsu is not a small company and has a LOT of experience with critical systems. In a normal situation, in a healthy company, the pattern of faulty transactions would have been picked up quite quickly, remedied and its victims compensated. There must have been logs from which the behaviour of the system could have been reconstructed, and the culprit identified. The Post Office didn't do its due dilligence in find the root cause of the problem, but instead assumed the office managers were at fault. Without proof. And it is a dire indictment against the UK legal system, that they followed the Post Office in its findings, without demanding proper proof that the SW system was not at fault. Courts must be aware that SW can be at fault.
    29
  4. The key step for achieving Continuous Delivery is the automation of the deployment pipeline beyond running unit tests. When the pipeline also automatically runs integration tests and a set of acceptance tests, and then deploys the software if all is well, that is when you start getting the benefits of this style of working. Most teams are scared to automate the final parts of the pipe line, and have some sort of manual QA holding things up. Those teams will never see the benefits of QA. And most will have an over-worked Architect or QA team struggling to keep up with the changes.
    22
  5. The principles still work. You can make "proof of concepts" very quickly even for very complex systems. That is what stuff like LabView and Simulink is for. Risk-driven prioritization is even more important for those projects. And management needs to know and be OK with the fact that early in the product cycle, estimates for software can be off by a factor 2. If they are not OK with that, they need to manage the project differently, spending more time in small POC projects before going all-out. If you give a detailed estimate for a complex software system when there still are technical risks, you are doing your company a disservice.
    14
  6. I recently left a team because of the frustration of not being able to merge into `master`. Twice I had to spend two weeks of merging, catching up with other people, remerging, waiting for approval from the Architect, etc, and hoping to find a moment when nobody else was checking stuff in. At one point I got angry and demanded my changes got accepted without review and nobody merge anything into master until I got my changes merged into it. That was a 12-man team heavily using feature branches. I did stay until I felt I had done enough to fulfill my contract, then quit. I am all for committing directly into main (after some sanity checks & local unit tests). These complex acceptance procedures give a false sense of security, but in effect only raise frustration and stress levels.
    11
  7. Yet another excellent presentation. This is by far my favourite Software Engineering channel.
    8
  8.  @0LoneTech  "accepting absolutely no responsibility" The software industry is perfectly willing to accept responsibility -- except that almost no-one is willing to pay us sufficiently for us to take responsibility. So, in order to save money, we are forced to cut corners. Writing software is the most complex activity humanity is capable of. We can do a lot for a price that is acceptable. But the costs for doing a perfect job are tenfold (if not more) of doing a "mostly OK" job. Ask the medical and military industries.
    7
  9. I have interviewed people who expected a senior role because they spent 5+ years at their first programmer job. I had to disappoint them and offer a Junior role. Don't get stuck in a single gig!
    6
  10. Absolutely right in saying that developing software is not the same as running a factory. Developing software is the same class of work as developing a new product. No-one tries to develop new product in a "lean" way. And we shouldn't try to develop software like that either. It isn't called software development for nothing.
    6
  11. Great move! I always though "Continuous Delivery" was too limiting a name. According to me, you are by far the highest quality SW engineering channel, and I am honour that you are sharing your knowledge & information with me.
    5
  12. The reward structure around these projects is what dooms them from the start. Starting with politicians who are rewarded for looking decisive and strong by kicking off mammoth projects, weak and ineffective by taking a smaller, incremental approach. As long as politicians are rewarded for looking busy these problems will persist, and some software contractors will continue to make billions in profits for not doing anything useful.
    5
  13. ​ @DumbledoreMcCracken  Touchée! Truth is, requirements need to be written by those who know what needs to be built, not how to build it. Though some understanding of the realities of SW engineering can be very helpful when writing requirements. As a Unix guru once noted, "it is better to built the right solution the wrong way, than the wrong solution the right way". Or as someone else said, "It is better to be approximately right than absolutely wrong".
    4
  14.  @georgehelyar  I have seen plenty of "waterfall" project succeed. But if you look closer, those projects were done by mature developers and managers who knew how to handle surprises and manage risk. Which is what being "agile" boils down to.
    4
  15.  @ForgottenKnight1  Also, every 10 years the industry decides to no longer use the nuts 'n bolts of the previous product line, switching to something as radically different as e.g. using only glue or only velcro.
    4
  16. I missed the questionaire, but I do think that Rust programming language is an import development. It being the first language that is both efficient and has fundamentally solved memory leaks and race conditions. And that it has solved these issues simply by using the features of a powerful type system is mind-blowing. It will take some time to grow in popularity, but I think Rust has a very bright future ahead.
    4
  17.  @stephendgreen1502  Dude, what have you been smoking? I want some. OF COURSE there are over-arching designs, the various teams do not work in TOTAL isolation. They are guided by the overall design for the API's of the services, and by the requirements for the overall system. The users (consumers) of the various API's are stakeholders in the design, development & testing of each service, as is the Principal who is paying for the system, and the users he wants to sell the system to. That initial over-arching design of the API's is what the video was talking about when he said some things require more thought up-front than with monolithical systems. You don't have to get them perfect, but you do need have something to start with before you can delegate work to the teams. And during the project, each team needs to frequently present their service to the other teams (its users) to determine whether it is fit for purpose or needs improvement. The other teams do have the right to tell a team that they have made a mistake and need to change things. And each team has the responsibility to support each version of the API for as long as it is used by the others. They can't just go a different direction and break the system.
    3
  18. If you are a successful SW engineer (successful as in making successful solutions), I can guarantee that you have incorporated most if not all of the Agile Manifesto in you way of work. Do you listen to your users and incorporate their feedback, or do you doggedly continue building what was agreed years ago even when users are dissatisfied? Do you write reams of documentation no-one will read and even when they do, they discover it is out of date? When a colleague comes with a question, or you see him struggling, do you refer to the documentation system or the Design, or do you sit down to work with him? Does your Design change when you discover that it must? Did you ever talk with a User or other non-SW person to discover you misunderstood something about the solution, and then changed the SW accordingly? If you answer these with Yes, you are Agile.
    3
  19. @@retiredbore378 Thanks for the insult, dude. I'm an MSc (EE) with 30 years experience developing firmware.
    3
  20. If the legacy classes are testable, there is no reason not to do TDD. If the legacy classes are not testable, you could work towards making them testable, especially those parts that are subject to most change. That effort will soon earn itself back.
    3
  21.  @retagainez  There was insufficient trust from Management, it wasn't a happy work environment. I was glad to leave, though I made excellent money.
    3
  22. QA is a totally unnecessary burden to the company. Until it isn't. Let's hope that CrowdStike is bankrupted by a slew of criminal negligence suits filed against it. That will open some eyes to the need of QA.
    3
  23. When I interview people for a developer job, I usually get them to talk about stuff they built earlier, and how easy or difficult it is for them to explain it to me. I'll try to get them to draw a few diagrams to summarize the system. Because I believe the most important skill for a developer is to take a step back from the details and create some overview / abstraction. And this test also gives me insight into how they will function in a team, and the level of complexity the person can handle.
    3
  24. Software development is a GIGO process. Garbage In, Garbage Out.
    3
  25. Often it is necessary to work through the happy flow first because it is not totally clear what you need to make. In those cases, working through edge cases and failure modes is a waste of time. True engineers often do this as well. First make a Proof of Concept, then properly engineer the product. I find that SW projects far too often skip the PoC phase, leading to disaster. PoC comes before MVP (Minimum Viable Product).
    3
  26. ​ @Tony-dp1rl  Companies that do complex systems are aware of the uncertainties in software estimates and are skilled in risk management. Or they won't be in business for long.
    2
  27. Fully agree with your analysis of the failure of UML-driven design. However, I do believe in Model Driven Design where the model is a Domain Specific Language.
    2
  28. I LOVE TDD. It is the only way to write professional software. There is one situation in which it shouldn't be used: when exploring a solution space. But as soon as the design starts to solidify, the paradigm should shift to TDD.
    2
  29.  @ContinuousDelivery  to focus on design while still exploring the solution space will lead to premature choices you will regret. It is better to make throw-away proof-of-concepts in that phase. Just be sure to actually throw them away...
    2
  30. I believe in metrics. Metrics like test coverage, Cyclomatic Complexity, Hot Spots for bugs & code modifications, etc. NOT in metrics that would "measure" the contribution of a human being to the team. But I agree there shouldn't be hard limits set them, but they should help prioritize the work.
    2
  31. "Functional Programmers think of Programming as Maths" I think most functional programmers think of program as a series of data transformations. I that is what you mean with "maths", then you are correct. I also think that most OO programmers think of a program as a collection of entities that "exist" and share state with each other. Both have merit, though the functional paradigm has the advantage of being composable, a very nice attribute that makes testing and re-use relatively easy. You can glue functional code together to create a new function, much easier than you can glue objects together. For me, functional programming is an acquired taste, that I have learned to appreciate!
    2
  32.  @ForgottenKnight1  Nah. I had the privilege of re-designing a bit on the core of the system that had major impact on the rest of the system. In order to fix design errors that kept the system from fulfilling its scalability requirements. So basically every other checkin by anyone else required me to update my branch -- which required new approval by the QA people, etc. Business-wise, my update was THE most important that was being done: the product was not viable without it.
    2
  33. I remember a nice article by Jack Ganssle on the need for watchdogs. He describes a system that was crashing all the time yet continued to perform as desired because of a good watchdog. In fact, the customers never knew the system was buggy.
    2
  34. I wouldn't want to work for a company that lets me be interviewed for a programming job by a non-programmer. Or for a management job by a non-manager.
    2
  35. Dilbert had such a long runtime because of this. Companies don't learn as long as MBAs are given to people with big egos instead of big hearts or big brains.
    2
  36. The problem with sprints is that it was never intended for constant use throughout the project. They didn't call it a SPRINT for nothing. You can only sprint once in a while, NEVER continuously. Continuous Delivery is a MUCH better way of working, even if not completely applicable in all SW projects, because the user feedback element is not always an automatic result of continuous delivery. So sometimes you need to organize events to garner user feedback and demonstrate progress to the stakeholders. And with CD you can organize these events anytime without putting pressure on the team, because the SW is always ready to be demonstrated.
    2
  37.  @SufianBabri  Exactly! It isn't like the tooth shape of an M-10 thread changes every time the relevant committee has a meeting. But this does happen with our tools and even our programming languages.
    2
  38. One thing underpinning the Waterfall model is that development happens at many different levels of abstraction. As a person can only grasp a limited spread from high to low abstraction, the steps in Waterfall are often done by different people. The fault in the Waterfall model is that it assumes people working at higher levels of abstraction can predict with 100% confidence the needs for the lower levels and are isolated from any discrepancy. This feels very safe for managers, who are thus isolated from the nitty-gritty details in a project, but is not how the world operates.
    2
  39.  @Axel_Andersen  There are few people who understand the different types of software better than this channel, and myself, having worked for 30 years in developing mainly firmware, but also other types of software.
    2
  40.  @mikevidzdev  All cooks are great chefs -- until another person tastes their creations. Likewise, software needs to be actually used to evaluate whether it is fit for purpose. The quicker feedback, the easier to fix problems. That is the theory behind CD.
    2
  41. Developers shouldn't give their managers / product owners etc a choice between features and quality. That is not something they are qualified to micro-manage. This is one reason why a task list should be off-limits to these peoples, or why "technical debt" tasks shouldn't be on the storyboard.
    2
  42. Personally, I use comments like section headings in a document. It helps readers get an overview of what is being done. Including me. I usually write these comments before I write the code. Very rarely, I use it when I have done something clever, though I try hard to avoid that. This means that comments are only needed when functions have several sections.
    2
  43. After some thought, my bet is that the development of this software was scaled up prematurely by managers wanting to make a deadline. I.e. before the problem was thoroughly analyzed and an architecture defined that could deliver the reliability demands required. What tips me off is the bitching about the "useless" developers. It shows that the architect / lead designer of the software did not have the project planned & designed to a point where less capable developers can work without doing damage. Those developers probably drained his time & energy so he never got round to tackling the hard problems in this project. I have seen it happen in other projects. Projects with a lot of technical risk (like this one) need to be started with a core team of high-quality developers who focus on the hard problems and tackle them. Only after the technical risk is acceptably low can less capable developers be shoe-horned into a project.
    2
  44. Another great video! People offering quick fixes or methodologies for making the outcome of SW projects more predictable, and are not truly Agile, are no better than snake-oil salesmen. They offer false security. The problem is that in SW, all the repetitive work is automated, and thus doesn't cost anything. Whereas in the other engineering disciplines the repetitive work costs an order of magnitude (often several orders of magnitude) more than the non-repetitive, creative, design work. So any methodology that borrows from other engineering disciplines simply doesn't work in SW development. SW development is inherently more complex and difficult than most other engineering work. As someone said, there is no silver bullet. (Systems Engineering is also inherently complex. Thus system engineering methodologies borrow heavily from SW engineering and vice-versa.)
    2
  45. Sadly the hiring process is managed by HR, who know diddly-squat about SW engineering or learning. They just want to keep their plushy, overpaid jobs.
    2
  46. Issues like this is why I do not give my credentials for e.g. pushing changes to my repo to a tool like my IDE. I will use the git CLI for pushing code to servers, thank you very much. Security starts with limiting the number of parties you trust with your credentials. I will not entrust an IDE with my credentials, and I think it is wrong of Jetbrains, in this instance, to ask for them. Same for SSH private keys. Of course, the danger from CI / CD pipe-lines is even greater, as these need to have credentials to access the production pipeline. I hope this episode will teach people to be more careful with what they entrust their tools with.
    1
  47. You can be Agile without calling yourself agile.
    1
  48. Me, I do not think the cloud is inherently good. In fact, quite the opposite. I think it is a new way in which the large providers try to lock us into their over-priced services.
    1
  49. I have built an event-driven platform 15 years ago that was very successful (i.e. still being used right now in two companies). I found that the hardest thing is to robustly implement the request-response semantic. We still used it, sparingly, but that is my single regret. We should have done without it completely.
    1
  50.  @chrispeng5502  What projects that start with a deadline need, is a risk / uncertainty driven management process, like true agile, to ensure that at least there is something that more-or-less works at that deadline. As all PMs know, you can estimate project cost, duration and quality (match to requirements), and pick any two of those to meet. But never all three. In SW projects, two of those need flexibility, as you can not pump money (i.e. more developers) into a project to make it faster. As someone said, you can't spread the process of having a baby over 9 women and expect the baby to be ready in 1 month. Pumping developers into a SW project that is late will make it even later.
    1