Comments by "Tiago D\x27Agostini" (@tiagodagostini) on "Continuous Delivery" channel.

  1. Estimations are only valid if you already did something very very similar. That said, from a owner of a software company POV now, they are needed, because money allocation depends on that. So we really need to know a low and a high bar for an expected release.
    9
  2.  @lukasdolezal8245  There are several fields that would call insanity to not do any end to end test. Would you deploy a telephony system without doing a single real call before delivery? It is not only deemed stupid, regulatory agencies demand register and proofs of a large set of REAL conditions tests . You may test every single part of your system and each signal degardation step may be within parameters but when you put everythign together the interaction along those degradations creates an unusable result. So it is common practive to automatically generate a large number of calls on a large number of routing situations and compare the entropy between the signals in the input and in the output.
    9
  3.  @Ash31415  Sincirely I think that is the problem with some of his videos. The title of a book gives the whole tone of the reading experience.
    8
  4. This, your answers puts it much better . There are things you NEED E2E tests, so do it for those. Just try to use smaller scale for things that can be caught on that scale. A simple example is system performance, e2E testign is the only one that will give you a REALISTIC measure of the performance on a system that is asynchronous, parallel and with several dozen stages.
    6
  5.  @THEMithrandir09  Well For starters, Open Source does not imply free of charge. It is free to be changed. You could distribute your Open Source content behind a paywall. THe logic way is to do that with peripherical modules mainly.
    5
  6. That is the classic middle management agile. They think agile is something FOR THEM, that makes Them more important. They are the ones that forget that a company direction is given by directors, the ones that MOVE things there are the workers. management is there to HELP the workers, not the other way around.
    5
  7. The problem of partitioning code for business is that you enforce the result that your architecture will be a copy of the organization internal department scheme and that usually is HORRIBLE to mantain and horrible to scale. Develoeprs do need to understand the business but the oens that define where to partition things shoudl be senior DEVELOPERS.
    4
  8. Well I never trusted. On all places I Worked we NERVER used a release newer than 3 months. Your development environment must be more stable than that to keep a good quality output. If things change everyday you are in a wild sea.
    4
  9. Yet that is not enough. For a client the product is the interaction of the code with the environment. So systems that operate in complex scenarios that cannot be isolated (telecom for example) are usually in need of E2E testing in live scenario to evaluate extraneous factors.
    3
  10. There are proper mathemathical tools that are way more effective than TDD exactly because they are not sampling, see TLA+ for a good example. It grants a much higher level of assurance than any TDD.
    3
  11.  @johnjackson6262  It is still the ONLY form to ensure performance. And perfromance is part of quality! It is not a good test to get operation failures but it is the best test to evaluate how yoru changes impact load, latency, tasks paralelism etc...
    3
  12. The thing I often not be used enough is hierarchical test. In your example you can have a few tests that if they pass everything works. Only if they fail the more detailed tests need to be run (automatically) to find where. Before anyone screams with me.. real world is not always web development, there are systems where some tests take HOURS because they operate over a Terabyte of data each time. It is important to not waste time and server costs to run something that is not needed.
    2
  13.  @enginerdy  Let me put you a simple example that corresponds to 50% of the software companies in world.. : ONLY 1 DEVELOPER...
    2
  14.  @TrackedHiker  I Know MORE about development than any of my employees. I had your mentality yet as ALL business owners I understood that it cannot be that way. Simple as that. There is a reason why all developers that start their own companies change their opinions... because it is not an opinion, it is a fact. I bet you would be angry if your boss did not accept to say how much you will earn in your month until after the month ended. Yet we also do not know how much money will enter the company before hand.. but we HAVE TO predict! We can lose a client, we can fail to get new ones, lots of things can happen. The business feature is the one that pays the bills, be sure if ONE facet of companies that will not change is that. The business is the goal, development is just a tool. You do not change the objective just to make the tool easier.
    2
  15.  @pawelhyzopski6456  That is not as easy to say when your field for eaxmple is healthcare.. and not opoenign meansw peopel will die in emergency waiting for the PACS or HIS to proccess their data. There are some moments where you NEED to do what is needed. And as soon as that is done you start doing what you SHOULD have done.
    2
  16.  @mariusg8824  Sound very cool until your company needs to pivot into a field and the new field is completely incompatible with the way your organization is structured. I have seen companies crumble exactly due to that. Try to make an organization match with an operating system.. I bet microsoft suffered a lot with the Conway's effect.
    2
  17. Well it is better that they do not get to production than if they got to production being unfit for it :)
    2
  18. well that although is not a proof. I spent 12 years on a quite large project (1.4 MIllion code lines of C++ when I left). The measured time fixing bugs was roughly 4% of the team development time. We did not use TDD.. so one cannot assume that not using TDD generates buggy code. Quality of code comes from quality of the team, nor a process. TDD is a way to solve the issue of how to force a minimum standard on teams on another angle of approach.
    2
  19.  @ContinuousDelivery  The "always deliverable".. or as close to it as possible is something I agree, but I do not know if you heard this complain from more people, but at least in my experience, I have faced a lot of people that thinks the important part is the DELIVERY itself, not the software being always deliverable. The always in deliverable state I see a great correlation on what I learned on painting ( a hobby of mine).. a great practice is.. work from general to detail.. and try to keep your painting always in a consistent state that you could consider.. complete (even if in a lower detailed state). You add details in iterations. Always pushing the whole painting into that next stage before you advance further . The painting is always consistent. The contrast would be to render a corner of the image to user detail before you do anything else... and you fall in the trap that if you suddenly dislike your composition.. it is too late do change anything.
    2
  20. I used to be only developer, but then started a new company and now I understand why something alike Jira is NEEDED. You cannot ever justify anything to your investors without it.
    2
  21.  @kurt7020  Unfortunately the ways are not obvious and the community is very .. reactive to people asking how to change some behavior (real zealotry). Everything would be easy to fix (on my pov) if I could overload the assignment operator (=), with specific signatures for move and copy (just like C++). Each one could make easily code that behaves as they seem fit. I am currently making a project to re learn Rust (I passed a few years without looking at it), or an n-body orbital solver supporting a generative AI to learn to make orbital transfers using the least fuel. I am writing 2 version one in C++ and one in Rust. The C++ one has much less typing and is clearer to read (mostly because math and simulation code is much biased towards copy behavior). Rust can have some gains on other areas but paying that price make its appeal diminish.
    2
  22.  @-Jason-L  Should, but you cannot! If you are making a website that sells tshirts you can risk it, but if you are making systems where a single minute of downtime can cost millions or worse, LIVES, then you CANNOT TRUST ANYONE word.
    2
  23. Using fixed versions is basic common sense.
    2
  24.  @GalaxyCat001  I remember a coleague that when the new manager asked what model of development we used (already preparing to change everything) he answered "steady and ongoing"... and completed..."it is like scrum .. but witht he stupid part replaced by the "turtle and the hare" lessons.
    2
  25. Well be ready to get a lecture how company management, resource allocation marketing fund allocation works in return. I have done both things and I will say, software development is not harder than navigating business and government regulations that emand beforehand estimation of cash flow..
    2
  26. What I hate more than anything.. besides Javascript, .... and standup meeting.., is OPEN SPACE OFFICES!!!! Developers do not work or think as people in HR or other departments!!!
    2
  27. As everything in live the middle ground is usually the best place.
    2
  28. Yes, but both things should live side by side. Worst thing is when you find a method whsoe name implies it does A, but in fact it does B.
    2
  29. Before that there is the step of EXPLAINING your code to someone else. IFyou cannot make another person understand.. you probably need to throw it away and re do it.
    2
  30.  @octomancer  My own opinion is not so positive of unit testing. In my 30 year career I have seen so many mistakes created by unity testing focus (instead of thinking in the problem, the developers think in tests only and fail to see real complex interactions, that are the SERIOUS issues in several type of systems). I do near zero UNITY testing a the very definition, yet my systems have operated for decades in healthcare with very high reliability in healthcare. You need testing, automated AND manual (depending on the tool type) but not always unity testing is the way to make the best tests. IF your field of operation tranlates well over a math base, a lot that unit test aims is already acomplished by the underlying reliability of a math model.
    2
  31. Well One to blame in this debacle was EU. They Forced microsoft to expose their kernel to third part vendors. Operatign systems is for the operatign systems develoeprs and politicians have no place into forcing dangerous things like that.
    1
  32. But that is standard practice in game market. You do not get funding for your projects if you do not do that. Apple does not need external funding for a project. Their mistake was the most common one on game industry. They did not have a real game view, they had a hyperbolic dream and advertised that. Anyoen that knew anything about game development knew, back at the first announcement that it would NOT succeed on what it promised.
    1
  33. Yet is is not how math works. Math is a truth. It can define things that are not modeled. Several things were discovered because math demanded them to exist.
    1
  34. With the classic sole exception.. a system with only 1 part.
    1
  35. Java script is to programming languages what IA32 was to processors. Stupid idiotic, bot so spread that no one dares not not focus on it.
    1
  36. The 10 lines of code although must be taken with a pinch of salt (i.e it is not for 100% of cases). For example if you are writing a code that has a directly mapped meaning, split it can cause confusion. One example is math code. If you take a invert matrix code and start to fragment it you are splitting a well defined and known concept and that very likely will cause MORE confusion (because you start to create arbitrary cut point that do not exist in the concept)
    1
  37. Well in pure theory it is possible to measure productivity. You just need a company with only 1 developer. You check how much money the company makes with its product and how much it costs and you have productivity. Problem is that this scenario is not very useful.
    1
  38. But that is why I am against the hardcore TDD. Development should not be DRIVEN by test, should be SIDE TO SID with it. When something other than understanding the domain drives the development, you end up with BAD design. TEst is part of development, not the reason (Driven means is is the thing that pulls all your moves into the develoment, i.e the reason) of the development.
    1
  39. I think you underestimate things. This happens exactly the same in a hospital. The only difference is that people are very very careful at saying life A is worth more than B.
    1
  40. As I heard a wise dude say once, the only contract you can trust to never change is math, for the rest is all about your naivety.
    1
  41. I will say most companies do return to the origin any bug fixes they find, and that is their obligation. If you want more than that you need a license that goes into that direction.
    1
  42.  @gronki1  yes, when you are writting a stop gap feature whose lifetime is expected to be 2 weeks and only need to convert some data from system A to B 1 time per day when a secretary push a certain button and the worst it can happen is someone having to do something by hand. Then it is not monetarily worth to spend resources into achieving the same safety that you should employ in a system that controls something in a passengers plane.
    1
  43.  @piotrd.4850  Yup, My company is in healthcare and it is a NO-NO to deploy ANY software piece, be it a library or your OWN software without a few MONTHS of enclosed testing.
    1
  44. The main cause of that is that ideally notebooks shoudl have some form of specialized version control. Git is problematic because it cannot differentiate automatically the CODE lines from the execution lines. THat result in frequent conflicts and makes everything painful.
    1
  45. I find funny that there is so much HYPE on continuous delivery when during my 24 year experience (most of it in healthcare and communication backbone systems), most often than not the clients COMPLAIN of continuous delivery. I all the fields I worked they start to complain heavily if you make weekly deliveries. . I had cases where we lost clients because they felt they were being used as testers and they wanted at MOST 2 releases per year. This was so critical that they pushed the regulatory agencies in my country to FORBID such practices. For example I cannot deliver/ sell a software component in healthcare that had ANY change at all after it had been deployed in test field environment for 28 days.
    1
  46.  @ContinuousDelivery  I do not agree with the policy, but the regulatory agencies here have setup rules that make the effort mostly moot. They demand a grace period where no change at all was made in the software before delivering it... that makes the Always at deliverable state almost irrelevant(at a businness level) , since you cannot use its most immediate gain (the only remainign gain is that you cannot take yourself too far from a stable situation and that indeed avoid going into a deep hole). Indeed there is a mix of the cocnept of deploy with delivery.. maybe here it is enhanced sicne in portuguese Deploy and deliver are the EXACT same word.
    1
  47. The issue is, there are not even close the the worst kind of errors neither the most common. A system that looks like it is working but it is resulting in wrong values or actions is far far more dangerous that one that crashes. I agree we should not ever be complacent on mathematical rigor, but the worst problems usually arise of bad system modelling, not a bad language. Most errors caused by languages shortcomings cause immediate and critical failures (although I know there are some exceptions) My view is not that languages need to evolve is that we have too many people that THINK they are capable developers when they are NOT, and unfortunately I think the trend of new " I will hold your hand" languages contributed to the spread of so many bad developers. So while I am all for using safer languages on products, I think learnign shoudl be in a harsher environment.
    1
  48.  @kurt7020  The zealots are what kill the language for me (that and the default behavior of MOVE instead of copy that make writing math code horribly verbose)
    1
  49.  @johnjackson6262  It is not the answer, but NOTHING is the answer anywhere in software development. Everything are tools and tools are to be used when they are deemed the correct tool for that specific moment. The simplest example is ensuring the performance of the system is still within acceptable parameters. You can only be sure about the performance in an asynchronous parallel system when you run the system in a complete state.
    1
  50.  @johnjackson6262  My point, to be more clear, is by your statement , I consider you a CRAP software engineer. Is it more clear now?
    1
  51. The classic terminology from the 60's, ORTHOGONAL requirements is the best one. They are not laid upon the axis of actuation of the system, but they are Still requirements.
    1
  52. The classic term from software engineering are ORTHOGONAL REQUIREMENTS. because they are orthogonal to the task the system performs (in the very algebric sense of the word). That terminology is way better.
    1
  53.  @RaMz00z  What kill companies is market , management, regulation and govnnrment shenannigans. Abstraction cannot do that. Bad abstractions make a company SLOW. Alone they are not enough to kill a company. Obviously you want good ones, but engineers tend to overestimate how much their critical path is ALL the critical path of a company.
    1
  54.  @madmanX1314  I have 20 years on software development and took me only 3 years managing a company to understand why the pressure comes for certain things. The problem usually arises in middle, then middle management do not understand either side and create procedures to pretend to achieve what he THINKS his boss asked. Will give you an example, here in the fiscal mode MY company works( vary on each country and each modality) we need to PREDICT the expenditure on the semester in paychecks used for new products and maintenance (Because the taxation is different).. so we need to have a reasonable prediction if something is deliverable before a clear cut day of the semester or not. If we make it wrong, we pay more taxes... believe, managing a company is NOWHERE simpler than writing code (on both cases due to difficulties created by other humans).
    1
  55. The end result is the same. The strongly typed language has a chaperone that points when you make a mistake. But the mistake is still you that make. It has been years sicne last time I had an error in python of an entity of wrong type reaching a location. But that is mostly the developer developing the paranoia needed for that tool (i.e the damn parameters indicate very clearly what should be passed). Strong typing is very useful helper tool, but all mistakes are always made by a developer.
    1
  56. I have been watching all your videos in the last 2 weeks or so, and the more I read people comments and examples, the more I realize we as "development field" have a problem. We, frequently think as development as a domain , when in reality it is a function upon different domains . An example was your statement in this video... "it is a fairly complex system and take sup to 25 minutes to build from scratch". I come from a background where a compilation from scratch takes like 3 hours, Our test datasets had more than 700 GB of data and the testing as a whole took yet a few more hours (and COSTS quite a bit of money to execute) . That just raises a point for me, we usually take OUR reality as the reality of development and that is as much a mistake as is when marketing and other departments take their realities and try to shoehorn it into development. There is no such thing as the reality of development, each company and field will have very different conditions that will result in different needs!
    1
  57.  @ContinuousDelivery  It is production data because there is statistical and AI inference about data in the product and we need to test if in a real workload it keep within certain boundaries (and Tomosynthesis exams have a few GB each one). Sure probably there is along path of optimization but you need to survive and hold on while you trail that path. Each domain has its different minefield to navigate. I bet Space X teams would love if testing a rocket engine was cheaper :)
    1
  58. That is specially true because in a legacy system you not always knwo what is expected at each point or component output, but you know with a larger degree of certainty what is expected at the end of a LARGER group of components or system.
    1
  59. Specially because in several fields the clients DEMAND proof of end to end testing. When someone says do not do what your clients DEMAND, there is something wrong.
    1
  60.  @grrr_lef  well if I have no time limit and the pay is really good I might try it :P Also more on the subject.. you cannot tell your clients to get lost. That is not how successfull companies work, besides what holywood might think .
    1
  61. That said a modern engine has built in self monitoring. That is what unit test would be in that scenario. That said , it is true, You cannot write ALL the tests before you advance a bit in your development, it is as bad as writtign ALL the code before writting any test.
    1
  62. OOP is so BAD that any other paradigm is superior just for the fact that it keeps people away from the church of the absurd object hierarchies.
    1
  63. The greatest advantage of having independent systems is that you can test and evaluate (not only feature test, but business test) a new approach for something more easily. Obviously the limit is that you cannot have the tool (the architecture in this case) cause more effort than the job itself.
    1
  64. On your example at 14:06.. no way in hell. The first example ios EXPLICIT.. Explicit is always more clear (and is superior in avoiding someone stick something mixed with it (that is very easy to do in that loop)
    1
  65. That is the root of my whole problem with a practice that puts TEST as driving the development. Tests are important but are not the DRIVE factor or you deturpe the objective of your development. People start to skew the design just so they can get better Test coverage numbers and not solve the client problem in the best way.
    1
  66.  @jangohemmes352  Git is super problematic with notebooks because it does nto understand which lines are code and which are execution results (they are there intermixed). So when you make a pull over one of your own versiosn that was running in other data.. it nearly always result is HUGE number of conflicts and you spending half your day trying to fix it. We need a version control tool that knows how to handle notebooks safely, until then we need to believe everyoen using the notebook remembered to erase every single cell of execution before a commit and that is well like believing in santa claus
    1
  67.  @antoruby  Yes there are, but the more pieces you need to put in a system to work, the more chance something would go wrong. I really wish git had soemthing embedded for that in the sense that it ignores certain things in merges.
    1
  68.  @a0flj0  But that has always been my critic.. TDD preachers draw it as a holy salvation that by itself can assure nothign ever goes wrong. I hate when people put techniques in front of the main objective.. delivering a good product.
    1
  69.  @Immudzen  That is the correct approach (unless you are designign a library as a product, then forethinking on the interface is warranted)
    1
  70.  @a0flj0  Some 30 years of career here.. and I never found a TDD preacher that did not treat it as the holy path of salvation, at the level that "junior developers that do TDD do better code than senior ones without TDD." I blame the universities here, they teach content as if it were gospel and not technique. Sad byproduct of bad education system in places where universities are public and the teachers cannot ever be fired.. so they never update and keep repeating the same last thing they learned jsut before they go their job.
    1
  71.  @a0flj0  Well my 20+ years in the field have been basically only repetitions fo these anecdotals. I never seen anything different from that (I know they do exist, but my point is that it is NOT TDD that makes it work.. it is a set of more relevant maturity and education aspects).
    1
  72. The development branch being almost a mirror of the master is indeed useless. But branching when an experimental large change is being made is critical. If you are trying something that you might want to throw away you should NEVER EVER do it in a place where other people receive that before you can decide if you want to go forward.
    1