Winter Rant

We are entering an era in which AI can write code, what are the concerns of software written by AI? [Threads] [Wayback Archive]

Great question! And I wanted to couch my answer in some basic outlook I have about coding: Coding is simultaneously an act of expressing and comprehending human needs. That is, we write code to meet a real need that someone, somewhere is facing. This is basic to programming. As long as we remember that when using AI to write code, there is little to be concerned about.

I have been thinking about your question a lot, and I think I have a top-three list for this question. But for each of these concerns, I am also very excited about the possibilities that AI brings when writing code, and creating products.

First, I am concerned that we think that AI can write both our Code and (unit) Tests for us. This is IMHO dangerous thinking. I think a human needs to be in the loop somewhere to ensure that code is correct.Second, I am always thinking about how much more AI generated code there will be in the world, and what far reaches of this world and human life will it touch. We have code in cars for instance, and that makes me nervous; would I be more nervous if that was AI written code — hell yes!

… Just like driving, I am always thinking about what other parts of my life AI will control: Background checks? Resume sorting? Education and schooling?Third, i am always thinking about liability and responsibility. When I write faulty code, I am directly responsible for it; and I take ownership of fixing the bug. What happens when AI ends up making mistakes? Who is responsible for fixing them.

… Industry and economy work when we are able to identify parts of a system that are not performing and work to improve them. I am not sure that we are there with AI yet.