> This year is going to be different. PostgreSQL 19 most likely won’t be released until November… […] So, what happened? The short answer is that LLMs happened. […] There’s a huge amount of code, and that code contains a huge number of bugs. […] What’s different now is that it’s become vastly easier to find those bugs.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
Did you honestly need a /sarcasm tag in the article?
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
Yes, exactly. The only question I have is whether the delay is attributable to high-quality bug reports or the burden of triaging low-quality ones. It's not completely clear which situation they're experiencing.
That's kind of what I was afraid of. Both is really bad news. If you think of it in terms of average speed over ground, working on low quality reports is like spending a bunch of time at v=0.
I did a little AI-assisted bug-hunting in Postgres this summer in between jobs, and the haul was staggering. Multiple serialization anomalies (eg [0] [1] [2]), foreign key integrity issues ([3] [4]), planner correctness bugs such as [5]. Various CVE-worthy vulnerabilities. An unfixable timezone bug [6]. The list got so long that I had to triage what I even had time to report, and settled for a dozen or two out of hundreds of candidates. Now imagine the entire world doing this.
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
I will add more example links in the top-level comment, but here is the thread for that bug. [0]. tl;dr timestamp vs timestamptz comparisons aren't consistently ordered in a pathological edge case, which means that index scans can return the wrong answer. There's no way to fix it without either redefining the operators in a surprising way or disabling index scans in this case.
I am pretty sure we have always been able to add holes faster than we can find them and fill them, and that this has not changed with the advent of faster hole finding tools that are also faster hole producing tools.
I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify
"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.
That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer.
This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”
I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though!
I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc
The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.
I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money.
It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.
I guess the real challenge is figuring out how to scale verification, cause discovery isn't the bottleneck anymore. Generating more bug reports doesn't necessarily translate into faster fixes if the same small group of maintainers still have to validate the reports, review fixes, etc.
Is no one entertaining the possibility that some contributions were lower quality due to AI? I'm surprised to have only seen arguments that it's review.
So it seems there as a certain level of safety in obscurity that was acceptable / fine which has been lost and therefore the focus is shifting towards churning through the slop.
I'm not very positive on the social, technical, or intellectual impact of AI on software development. Actually, that's understating it. I think AI is basically cancer for the mind. That said, this framing gave me pause:
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
Signal quality... what is that even? The bugs may be valid, but they may also be obscure, irrelevant, low priority. A sibling thread mentioned that they had found CVE-level bugs in Postgres, and those require attention in some version of "immediately". But beyond that... not every bug needs fixing; that's a trap.
Seems like a lot of these bugs have not affected end-users or tripped up maintainers.
They could be trade-offs that helped get other features over the line even?
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
I have found major bugs in late beta's of postgres long before AI came along. I found a bug where malformed json would segfault the entire server (a segfault in a server process forces the entire cluster to shut down by design) when using jsonb in a release candidate once. They fixed it rather than ripping the entire feature out.
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
Maybe its more a issue that X amount of people have more work to fix discovered issues. So the new features have not undergone the same level of testing as before.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
Are the features bad by design? Like are they bad even if they work perfectly because they add bad abstractions to pg's SQL syntax, or are just broken in how they are supposed to work? If so the process is dysfunctional to let them get this far before noticing.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
To be honest I would love for someone to work on the structuring of pg_dump for example, which is entirely driven by regexes and binary structures. It’s awful and absolutely not programmable - a real shame.
I work on pgrust. We shouldn't have submitted the preview when we did: it was still in active development, and its SQL coverage was far too narrow, though there was nothing in it giving it an unfair performance advantage
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
- Per-query kernels did not serve the benchmark queries[0]
- The preview's fast repeat LIKE queries came from a dictionary filter cache, not a cached query result[1]
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]
Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
postgres is mature database used the world over allegedly about to be on major version 19. how has the time not been spent already to get rid of the bugs?
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
"We found a bunch of ways to make our product safer and better. We need more time to ship a higher quality product."
PICKET SIGNS
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
We will hopefully get over this wave and be able to move forward with much more secure software, but it's crazy right now.
LLMs finding unpatched vulnerabilities faster than they can be reasonably patched, bringing all other software development to a halt.
https://www.vulncheck.com/blog/anthropic-glasswing-receipts
[0] Write skew observed under serializable isolation: https://www.postgresql.org/message-id/flat/CA+COZaBo+ZpKgMvx...
[1] Serializable anomaly: duplicate primary keys: https://www.postgresql.org/message-id/flat/CA+COZaBOiiRPmEfX...
[2] SERIALIZABLE can commit write skew after SSI summarization: https://www.postgresql.org/message-id/flat/CA+COZaCtK=UQbeQw...
[3] Fix temporal foreign key validation of pre-existing rows https://www.postgresql.org/message-id/flat/CA+COZaChq_ukm2o7...
[4] Lost lock during toasting allows FK violation https://www.postgresql.org/message-id/flat/CA+COZaC-2-GQabq+...
[5] Partition pruning can incorrectly exclude a RANGE default partition https://www.postgresql.org/message-id/flat/CA+COZaDXrfTaBjLE...
[6] timestamp vs. timestamptz comparisons inconsistently ordered under DST : https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
question is, how long do you think this persists before there's only a long tail of minor exploits left?
“Find the issues in this code” vs “write this feature” are different problems.
The first one is easier because the feedback loop is llm to code/test and trivially scalable.
The second one is llm to human which is going to be slower because it can’t read your mind.
The tighter and more deterministic the loop the more llms excel.
make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.
if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
NP-Hard, even[1]!
[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...
AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.
Postgres did not have a slop issue and it’s wrong for people to insinuate it.
any kind of pattern matching / transformation is going to be an ideal task for LLMs
I missed when AI became accountable for Postgres releases, when did that happen?
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
this doesn't seem good attitude for critical software like database
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
https://www.postgresql.org/message-id/20140818195634.2543.56...
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
Remove pgrust: the entry is cheating (per-query kernels and result caching) (197 points, 10 days ago, 166 comments) https://news.ycombinator.com/item?id=50205816
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
[0] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[2] https://benchmark.clickhouse.com/
[3] https://github.com/ClickHouse/ClickBench/pull/2751