Comments by "Immudzen" (@Immudzen) on "Continuous Delivery" channel.

Exactly this. By making developers write test it forces you to use your own code and that forces you to face the design problems and fix them. This is what I have encountered with all the junior developers. It has helped them become much better developers.
2
@danielli3288 Lots of people idolize him but it looks like not many highly qualified people capable of programming at the level required do.
2
One of the thing you can do is for functions that take a dictionary of arguments or return a dictionary of outputs is to create a dataclass instead. You can then type based on the dataclass and your input and output is then a contract that is far easier to test. You can then also put types on all of the variables involved. It also makes it easier to test and document.
2
I tend to use a hybrid of OO and functional code. I use features like python dataclasses or attrs to create objects that mostly just hold data. I then have functions that depend on that interface and return objects of an interface. I found this made the code easier to test and more flexible because the dependencies move to the interface. So plotting can take a result object, a simulation evaluation function can depend on a parameter set objects and return a set of results etc. It has made the code more testable and also easier to teach to people because they can see from the interfaces how the functions can be chained together.
2
@JanVerny It is nowhere near the level of waterfall but it is not like you can just jump in and start coding when you don't even know what kind of problem you need to solve. Once you have a general idea you typically have to read papers to figure out what kinds of solutions even exist and we can discuss what would even work for us. If one of the methods requires equipment we don't have then other methods need to be looked at first. I don't see a way to just start coding without these ideas figured out at least a little bit first.
2
I find trunk based development interesting and a kind of aspirational goal that I keep the teams moving towards. We have gone from merging once a year with some branches lasting more than a year to merging once a month to now most branches live for about 1-5 days before they are merged in. Coding practices are getting much better and so are coding skills. We don't actually do TDD yet but we do make sure all the code that is merged in is tested and most of the time now people write one function and then a test for it and then use it to verify the function is correct and fix bugs in it before moving on. You videos have definitely been helpful in getting the teams to move forward. Still a lot of work to do but I keep learning more from your videos.
2
@e2e23e Technically it was a combination of TDD and unit tests. For most of the new changes the tests where written at least partially first. Unittests where also added for older code in order to refactor the code. Tests for the newer functions to be created where often written first.
2
@DudeWatIsThis That is probably true. I have even read research papers that these AI coding assistants provide the most help to senior programmers and can even lower productivity on newer programmers and marginal gains for juniors. The part I find odd that I just can't explain is that the phd engineers and scientists I work with that are junior programmers at best DO benefit from it. I explained about writing small functions with a very specific purpose and they have not had issues with that.
2
@awmy3109 I do group coding sessions with vscode where you can all code on the same thing at the same time. Involving engineers and scientists with the programming like that is highly productive. You can ask the experts about any issues as they come up and if something impacts multiple expertises they can discuss the right approach. Doing this through pull requests and meetings is MUCH slower.
1
@retagainez That could be. I want to be clear that most of the time we are coding alone and just talk over chat. If we run into something more complicated then we will have a group coding session with the people needed to solve the problem.
1
These tools are useful but you have to remember they are REALLY stupid. They have no understanding of what they are writing. They pass college level standardized exams but they fail at highschool and gradeschool exams. They can do certain common programming tasks but they routinely fail at anything less common and the reason for both is the same. These things heavily rely on the solution to the question being in the training data. It can mix and match things together but it doesn't understand. It solves all the problems on stack overflow BECAUSE they are on stack overflow. It can solve standardized exams because there are thousands of online study guides for them. If these models killed stack overflow the companies would have to save the data to continue to use it for training because they can't be trained without it. There have already been studies that if you train an LLM on the output of an LLM it gets dumber fairly rapidly because you feed too many mistakes back into the system. Most of coding is not very novel when it comes down to most of the individual functions and so these tools can help but they are not very good at coding larger pieces. I think we are going to rapidly hit a point where these models don't get much better than they are now. They will still be very useful and an be tuned for specific tasks but I don't think they will actually get much better. We have to come up with some fundamentally different kind of model. I will also note that this is common in AI models. Look at self-driving, that has basically stalled for close to 10 years because the remaining problems are so difficult to do. If you build your own classification or regression models you can see they quickly get close to right but that further improvements are incredibly difficult.
1
What I have found with the DORA metrics is that releasing frequency is something that is often really difficult with scientific software. Sometimes it takes us a month to deliver a feature because of all the papers that have to be read and then implementing the math. While we do add the feature in small pieces and unit test all of it none of it works until it is all done. You can't really use just part of a new physics model without it being complete.
1
@manishm9478 We have a CI/CD system and full unit testing. I would just not consider it released until it actually works. We keep it in a constantly working state but what value would a release have for just part of a pysics model?
1
@jasondbaker We share the code right away and we have tests to make sure the fragments work. I would just not consider it released if it is just a model fragment. You can't simulate part of a physics model. It has no value to anyone and can't be used but the code is there and tested.
1
AI are already used for criminal sentencing and hiring/firing decisions. It is evil and the systems are often held as unaccountable.
1
@jimmyhirr5773 Yeah I have heard of mob programming and I think this is similar. This has been extremely useful to help newer coders, get people up to speed on a project and teach better practices.
1
Alternatives are difficult because it depends on what you are doing. For example I have never seen someone use Typescript. I have heard of the language but nothing beyond that. All of my work is on science and engineering applications and the alternatives would be things like MATLAB (proprietary) or Julia (still very new and not many people know it). Don't use MATLAB just don't. If you are doing machine learning you have Python and you have rounding errors for usage in other languages.
1
When working on any kind of simulator the one thing I think you have to do upfront design on is the actual math. What are the actual equations you are going to implement and why are they the right ones? This is such a foundational issue that it ends up touching a lot of the stuff the software does and while you can change it later it can also be quite painful.
1
@lprimak Look at pyTorch, Tensorflow, Numpy, Scipy, etc. They are all high performance libraries that are faster than Java because it is easy to write a Python interface to a high performance library. Java exists in a weird inbetween level where it is faster than Python but not remotely as fast as C/C++ and also harder to optimize. While Python is slower it is very easy to code the performance critical parts in a low level language. Since most low level tasks are pretty common most of those tools already have low level high performance versions. For example install numpy in a conda environment and it will link to MKL by default.
1
@berkes That has not been my experience. I have been coding Python since version 1.5. All of my work is science and engineering based modeling. We have nearly 100% unit test coverage and have been using type information since it was introduced. The type information doesn't help much with bugs but it is great for helping the IDE and more recently AI assist. There are so many tools for science and engineering code it is hard to justify not using Python because of the lack of libraries for other platforms. The way I see it you are dependent on tests no matter what because a type system won't save you and if you need to test everything then Python doesn't cause more problems than languages like C# does.
1
AI is bad at coding. Also, coding is the part I find enjoyable in life.
1
@passerbyp8531 I see unions working very well. Unions have done a lot to protect me and make working better. Jobs covered by unions also tend to pay more over time.
1
Sometimes where is also useful. If you are implementing something from a research paper then please put a link to the paper you are implementing it from so that 5 years later when we have to update the model we can find what you based it on.
1
I typically break machine learning down into two distinct types. You have regression type systems which approximate an unknown function. These systems are well understood and have mathematical proofs on how and why they work. They also make small mistakes as a result of us using these tools in a well understood way. The second type of things like image classification, driving, etc. type of tasks. These are problems we have built giant networks for and they kind of work. The problem is that when these systems fail they fail in extreme ways and we don't have a theory on how they actually work. These are the types I am most worried about. They get most of the press, they get the most deployments, and they are the least safe. Companies already use these to make decisions which is extremely worrying.
1
Overall I like the KANBAN approach and that is what my teams normally use. However, we have something closer to multiple queues. I work with a lot of research based code so a great number of the features are only implementable by a subset of the developers. If you need to improve the mathematical stability of a complex algorithm only people with the kinds of skills needed for that problem can work on it. So when people take tasks they take the next highest priority task that they are capable of doing. I have to admit I don't hold completely to that rule though. If I have a day with a bunch of meetings where I don't have time to really get into a hard task but I see a bunch of trivial tasks that I can just get through I will grab a bunch of those. I know it violates the idea of taking the next highest priority task but at least this way tasks get done. If I only have one hour between meetings I just can't deal with mathematical stability, designing a neural network, debugging something complicated, etc.
1
@litjellyfish These are scientific models and they only need fairly simple internal web interfaces so that others in the company can use them. Trying to get UI involved earlier has not really worked because the models can take years to develop and there is no real way to know ahead of time how long it will take or even what parameters the model will have. You start with a basic math simulator, see if it can explain the data, find out where it is deficient, model more of the physical process, and repeat.
1
@trsshowstopper9518 I think we have a very different idea of what agile is. Of course that is understandable given that agile seems to mean almost anything at this point. What I think of is what this channel talks about where you have CI/CD, rapid feedback, unit testing, tooling to support best practices and give rapid feedback on mistakes and try to engineer a system to prevent many classes of mistakes. So at work in Python we have started defining some of the function interfaces using dataclasses. It dramatically lowers the defect rates and based on evidence. I would consider all of this agile and also all part of an engineering approach. Having code that is well tested and in an always releasable state is also much easier to change as requirements change.
1
@trsshowstopper9518 The advice given in this channel are based on actual tested knowledge. Dave constantly refers to the research behind the advice given here. These are the engineering best practices based on data and this is the education format. I have no idea why most schools don't teach this stuff in computing classes. The point of this channel is you don't have to discover it yourself.
1
Aren't humans particularly bad at guessing their own accuracy? I think that is the basis of the dunning-kruger effect.
1
@JorgetePanete I remember a UK facial recognition system the police used. They reported it was about 95% accurate during testing but once deployed the news papers reported it was less than 5% accurate. Most AI systems tend to fail pretty hard in actual usage.
1
In our team we have modified the meetings quite a lot. Just giving a status update does not seem very useful because gitlab already captures that. Instead we focus more on if anyone is stuck, what resources they need, that kind of stuff. Since I do science and engineering programming sometimes people have actual hard issues and having a bunch of brains to help can really cut through the issues quickly. We do CI/CD and a KANBAN style of working. I sometimes wonder if the way we have modified agile just would not work for more normal software teams.
1
The problem I have run into is how do I test code that depends on large math data structures without having a fair bit of setup? For instance I was doing work on a solver and it takes in a 4D array that is several GB in size. While I can make it much smaller that also ends up not testing the solver correctly due to the error it introduces. Saving the data structure to disk can also be a problem in terms of then trying to keep it with the code and not having the checkouts be too large. What I mostly end up with is a complex setup for some of these kinds of tests and also separate tests for every single piece I can test independently.
1
@Savvynomad225 some scope changes occur due to understanding the problem better at the scope narrowed to something that is more realistic to deliver. In other cases there are external needs that drive changes in scope. In all cases we use CI/CD to make a continuously working system and we work with the various stakeholders at every step. If they do want to make a chance they are kept informed in the entire process and what that change will cost in terms of time. Since they work directly with the developers we often find ways to cut that time down because the stakeholders can be informed of the price of their various options.
1
I think we need some trust breaking. The companies in this market have merged or destroyed each other and this has made the system more fragile. We would have a more resilient system if we had a wider range of vendors.
1
I work in a heavily regulated environment. While we do test every piece independently we also deploy to a test environment that is a clone of production and a group of people has to manually check to make sure everything is correct with a well documented test script to follow. With lives on the line for making a mistake the process is slow to prevent any accidents.
1
@ContinuousDelivery Somehow we need to convince the regulators of that. We do test every piece independently with automated test suites and normal CI/CD systems. It is just that at some point a decision is made to release to production and all of that stuff is moved to a test server. We already know all the tests have passed but people need to go through and check many things manually before it can be released to production.
1
AI technology is being used to help people right now. It is being used for diseases and finding new types of treatments. That stuff just gets almost no news coverage.
1
I would have used return [word for word in dictionary if len(word) == 5] I find it simpler than both of those and easy to explain.
1
@sokoo1978 It does a very good job at that. I have still found that it doesn't seem to help junior developers very much. I don't know the reason why but I see it helping senior developers, engineers, and scientists far more. I suspect there is something different about how junior developers are approaching the problem and how copilot helps that is causing problems for learning.
1
@DudeWatIsThis I work with scientists that use it for python but the idea is pretty similar. They use it for modeling various reactions and it really does a good job with assisting this kind of code. Also once they have their system in code with tests I can then have someone that is a better programmer improve it but first it has to work correctly. As a senior developer I also find that these tools help me prototype much more rapidly. I was able to test a number of different neural network designs and realized they did not work for our problem and why along with documenting what I learned. It took about a day to do it and would have taken a few days to do it without an AI.
1
For all of our projects we use conda and create a yaml file to make it easy to create the environments. Especially for science, engineering, machine learning applications this will take care of your dependencies and link to high performance libraries with libraries like numpy linked to openblas or mkl and these provide a large performance increase.
1
@berkes Automated testing is designed to catch issues, if you don't have it you are screwed anyways. I have not had Python become less maintainable over time due to language issues. I have had poor coders make bad code and that becomes less maintainable but I see that in any language. That is something senior developers need to enforce and mentor to fix.
1