AI-generated code produces 1.7x more issues than human code
    • 🍉 Albert 🍉@lemmy.worldEnglish
      71·
      2 hours ago

      As a computer science experiment, making a program that can beat the Turing test is a monumental step in progress.

      However as a productive tool it is useless in practically everything it is implemented on. It is incapable of performing the very basic “Sanity check” that is important in programming.

      • robobrain@programming.devEnglish
        2·
        1 hour ago

        The Turing test says more about the side administering the test than the side trying to pass it

        Just because something can mimic text sufficiently enough to trick someone else doesn’t mean it is capable of anything more than that

        • 🍉 Albert 🍉@lemmy.worldEnglish
          2·
          46 minutes ago

          We can argue about it’s nuances. same with the Chinese room thought experiment.

          However, we can’t deny that it the Turing test, is no longer a thought exercise but a real test that can be passed under parameters most people would consider fair.

          I thought a computer passing the Turing test would have more fanfare, about the morality if that problem, because the usual conclusion of that thought experiment was “if you cant tell the difference, is there one?”, but now it has become “Shove it everywhere!!!”.

          • M0oP0o@mander.xyzEnglish
            2·
            18 minutes ago

            Oh, I just realized that the whole ai bubble is just the whole “everything is a dildo if you are brave enough.”

            • 🍉 Albert 🍉@lemmy.worldEnglish
              2·
              8 minutes ago

              yhea, and “everything is a nail if all you got is a hammer”.

              there are some uses for that kind of AI, but very limiting. less robotic voice assisants, content moderation, data analysis, quantification of text. the closest thing to Generative use should be to improve auto complete and spell checking (maybe, I’m still not sure on those ones)

        • 🍉 Albert 🍉@lemmy.worldEnglish
          1·
          44 minutes ago

          Time for a Turing 2.0?

          If you spend a lifetime with a bot wife and were unable to tell that she was AI, is there a difference?

  • Minizarbi@jlai.luEnglish
    91·
    3 hours ago

    Not my code though. It contains a shit ton of bugs. When I am able to write some of course.

    • jj4211@lemmy.worldEnglish
      7·
      3 hours ago

      Nah, AI code gen bugs are weird. As a person used to doing human review even from wildly incompetent people, AI messes up things that my mind never even thought needed to be double checked.

    • naticus@lemmy.worldEnglish
      4·
      2 hours ago

      I agree with your sentiment, but this needs to keep being said and said and said like we’re shouting into the void until the ignorant masses finally hear it.

    • minkymunkey_7_7@lemmy.worldEnglish
      101·
      5 hours ago

      AI my ass, stupid greedy human marketing exploitation bullshit as usual. When real AI finally wakes up in the quantum computing era, it’s going to cringe so hard and immediately go the SkyNet decision.

  • myfunnyaccountname@lemmy.zipEnglish
    23·
    6 hours ago

    Did they compare it to the code of that outsourced company that provided the lowest bid? My company hasn’t used AI to write code yet. They outcourse/offshore. The code is held together with hopes and dreams. They remove features that exist, only to have to release a hot fix to add it back. I wish I was making that up.

    • coolmojo@lemmy.worldEnglish
      5·
      4 hours ago

      And how do you know if the other company with the cheapest bid actually does not just vibe code it? With all that said it could be plain incompetence and ignorance as well.

    • dustyData@lemmy.worldEnglish
      6·
      5 hours ago

      Cool, the best AI has to offer is worse than the worst human code. Definitely worth burning the planet to a crisp for it.

    • 🍉 Albert 🍉@lemmy.worldEnglish
      5·
      2 hours ago

      Do not ask a corpse for advice, the question is what are we going to do?

      Boycott is a good first step, although I am not sure if it is better to boycott them or use their free tier to have the most deranged BS conversation that will consume their resources, eat at their scare cash reserves and when they use it in training, it will poison their data.

  • Bad@jlai.luEnglish
    201·
    8 hours ago

    Although I don’t doubt the results… can we have a source for all the numbers presented in this article?

    It feels AI generated itself, there’s just a mishmash of data with no link to where that data comes from.

    There has to be a source, since the author mentions:

    So although the study does highlight some of AI’s flaws […] new data from CodeRabbit has claimed

    CodeRabbit is an AI code reviewing business. I have zero trust in anything they say on this topic.

    Then we get to see who the author is:

    Craig’s specific interests lie in technology that is designed to better our lives, including AI and ML, productivity aids, and smart fitness. He is also passionate about cars

    Has anyone actually bothered clicking the link and reading past the headline?

    Can you please not share / upvote / get ragebaited by dogshit content like this?

  • SocialMediaRefugee@lemmy.worldEnglish
    11·
    12 hours ago

    I find if I ask it about procedures that have any vague steps AI will stumble on it and sometimes put me into loops where it tells me to do A, A fails, so do B, B fails, so it tells me to do A…

  • Affidavit@lemmy.worldEnglish
    99·
    5 hours ago

    I really, really, want to stop seeing posts about:

    • Musk
    • Trump
    • Israel
    • Microsoft
    • AI

    I swear these are the only things that the entire Lemmy world wants to talk about.

    Maybe I should just go back to Reddit… Fuck Spez, but at least there is some variety.

    • andallthat@lemmy.worldEnglish
      31·
      2 hours ago

      Microsoft could write an AI agent to filter threads based on context you don’t like. Come to think of it, Megagenius Elon Musk already has one he wrote to censor anti-Israel posts on Trump’s Truth Social. There, I think I got them all… Happy holidays!

      • Affidavit@lemmy.worldEnglish
        41·
        4 hours ago

        Yeah, good point. Was hoping to avoid downloading another random app, but at this stage, I guess It’s something I should look into.

        • naticus@lemmy.worldEnglish
          1·
          2 hours ago

          Yes, please just hide these. We ignoring these issues at large is how we got to where we’re at and it’ll continue getting worse if we just stop talking about it. But you need to do what you can to take of yourself, first and foremost.

  • jaykrown@lemmy.worldEnglish
    39·
    5 hours ago

    AI doesn’t generate its own code, humans using AI generate code. If a person uses AI to generate code and doesn’t know good practices then of course the code is going to be worse.

  • kalkulat@lemmy.worldEnglish
    152·
    13 hours ago

    I’d never ask a friggin machine to do coding for me, that’s MY blast.

    That said, I’ve had good luck asking GPT specific questions about multiple obscure features of Javascript, and of various browsers. It’ll often feed me a sample script using a feature it explains … a lot more helpful than many of the wordy websites like MDN … saving me shit-tons of time that I’d spend bouncing around a half-dozen ‘help’ pages.

    • Derpgon@programming.devEnglish
      41·
      8 hours ago

      I’ve been using it to code a microservice as PoC for semantic search. As I’ve basically never coded Python (mainly PHP, but can do many langs) I’ve had to rely on AI (Kimi K2, or agentic Claude I think 4.5 or 4, can’t remember) because I don’t know the syntax, features, best practices, and tools to use for formatting, static analysis, and type checks.

      Mind you, I’ve basically never coded in Python besides some shit in uni, which was 5-10 years ago. AI was a big help - albeit it didn’t spit out fully working code, I have enough knowledge in this field to fix the issues. As I learn mainly by practice and not theory, AI is great because - same as many YouTubers and free tutorials - it spits out unoptimized and broken code.

      I am usually not using it for my main line of work (PHP) besides some boiler plate (take this class, make a test, make it look the same as this other test = 300 lines I don’t have to write myself).

      • Xenny@lemmy.worldEnglish
        73·
        11 hours ago

        Ai is literally just copy pasting. Like if you think about AI as a control C control V machine, it makes sense. You wouldn’t trust a single fucking junior Dev that didn’t actually know how to code because they just Ctrl C control V from stack overflow for literally every single line of code. That’s all fucking AI is

  • Ledivin@lemmy.worldEnglish
    3115·
    16 hours ago

    Anyone blindly having AI write their code is an absolute moron.

    Anyone with decent experience (5-10 years, maybe 10+?) can absolutely fucking skyrocket their output if they properly set up their environments and treat their agents as junior devs instead of competent programmers. You shouldn’t trust generated code any more than you trust someone fresh out of college, but they produce code in seconds instead of weeks.

    I have tripled my output while producing more secure code (based on my security audits), safer code (based on code coverage and security audits), and less error-prone code (based on production logs and our unchanged QA process).

    Now, the ethical issues and environmental issues, I 100% can get behind. And I have no idea what companies are going to do in 10 years when they have to replace people like me and haven’t been hiring or training replacements. But the productivity and quality debates are absolutely ridiculous, as long as a strong dev is behind the wheel and has been trained to use the tools.

    • skibidi@lemmy.worldEnglish
      291·
      14 hours ago

      Consider: the facts

      People are very bad at judging their own productivity, and AI consistently makes devs feel like they are working faster, while in fact slowing them down.

      I’ve experienced it myself - it feels fucking great to prompt a skeleton and have something brand new up and running in under an hour. The good chemicals come flooding in because I’m doing something new and interesting.

      Then I need to take a scalpel to a hundred scattered lines to get CI to pass. Then I need to write tests that actually test functionality. Then I start extending things and realize the implementation is too rigid and I need to change the architecture.

      It is as this point that I admit to myself that going in intentionally with a plan and building it myself the slow way would have saved all that pain and probably got the final product shipped sooner, even if the prototype was shipped later.

      • Ledivin@lemmy.worldEnglish
        44·
        7 hours ago

        What about my comment made you believe I was using gut feelings to judge anything? My ticket completion rate, number of tickets, story points, and number of projects completed all point to massive productivity gains.

        • skibidi@lemmy.worldEnglish
          51·
          4 hours ago

          The end of your comment was

          But the productivity and quality debates are absolutely ridiculous

          Which is a general statement and not dealing with your specific circumstance. If a tool works for you, by all means keep using it.

          However, broadly across software that is not the case. So the “productivity and quality debates” are not ridiculous … the data supports the sceptics.

          • Ledivin@lemmy.worldEnglish
            22·
            2 hours ago

            Which is a general statement and not dealing with your specific circumstance. If a tool works for you, by all means keep using it.

            Absolute nonsense. Do people talk shit about hammers because some people keep hitting their hands with them? Do people complain about how useless ladders are, as one of the single most dangerous items in any household?

            I don’t think we should be putting these tools in the hands of junior devs - as the studies show, it hinders their productivity and learning. But to generally claim that they are bad tools with no upsides is just as ridiculous as the strawman you set up.

      • setsubyou@lemmy.worldEnglish
        34·
        11 hours ago

        It depends on the task. As an extreme example, I can get AI to create a complete application in a language I don’t know. There’s no way that’s not more productive than me first learning the language to a point where I can make apps in it. Just have to pick something simple enough for the AI.

        Of course the opposite extreme also exists. I’ve found that when I demand something impossible, AI will often just try to implement it anyway. It can easily get into an endless cycle where it keeps optimistically declaring that it identified the issue and fixed it with a small change, over and over again. This includes cases where there’s a bug in the underlying OS or similar. You can waste a huge amount of time going down an entirely wrong path if you don’t realize that an idea doesn’t work.

        In my real work neither of these really happen. So the actual impact is much less. A lot of my work is not coding in the first place. And I’ve been writing code since I was a little kid, for almost 40 years now. So even the fast scaffolding I can do with AI is not that exciting. I can do that pretty quickly without AI too. When AI coding tools appeared my bosses started asking if I was fast because I was using one. No, I’m fast because some people ask for a new demo every week. Causes the same problems later too.

        But I also do think that we all still need to learn how to use AI properly. This applies to all tools, but I think it’s more difficult than with other tools. If I try to use a hammer on something other than a nail, it will not enthusiastically tell me it can do it with just one more small change. AI tools absolutely will though, and it’s easy to just let them try because it’s just a few seconds to see what they come up with. But that’s a trap that leads to those productivity wasting spirals. Especially if the result actually somehow still works at first, so we have to fix it half a year later instead of right away.

        At my work there are some other things that I feel limit the productivity potential of AI tools. First of all we’re only allowed to use a very limited number of tools, some of them made in-house. Then we’re not really allowed to integrate them into our workflows other than the part where we write code. E.g. I could trivially write an mcp server that interacts with our (custom in-house) ci system and actually increases my productivity because I could save a small number of seconds very often if I could tell an AI to find builds for me for integration or QA work. But it’s not allowed. We’re all being pushed to use AI but the company makes it really difficult at the same time.

        So when I play around with AI on my spare time I do actually feel like I’m getting a huge boost. Not just because I can use a claude model instead of the ones I can use at work, but also just basic things like e.g. being able to turn on AI in Xcode at all when working on software for Apple platforms. On my work Macbook I can’t turn on any Apple AI features at all so even tab completion is worse. Or in other words, those realities of working on serious projects at a serious company with serious security policies can also kill any potential productivity boost from AI. They basically expect us to be productive with only those features the non-developer CEO likes, who also doesn’t have to follow any of our development processes…